Sectores
calendar icon
Noticia

The Sistema Nacional de Publicidad de Subvenciones y Ayudas Públicas (SNPSAP) is a tool that contributes to the transparency, dissemination and re-use of data related to public subsidies and grants. This system centralises all the information on calls for applications and concessions of subsidies and public aid approved by the General State Administration, the autonomous communities and local entities.

Origins of the project

We have to go back to 2014 to find the beginning of this project. That year saw a reform of the 2003 General Law on Subsidies (Law 38/2003) with a huge impact in two significant respects:

  • On the one hand, the National Subsidies Database, which had been created by the aforementioned Law in 2003, became available for full consultation by the managing bodies of subsidies and grants of all public administrations and those others legally authorised.

  • Moreover, the National System of Publicity of Public Subsidies and Grants was created as a public website, with free and unrestricted access for all citizens. This complied with the requirements of publicity and transparency of all public subsidies and aid granted in Spain, especially calls for applications and awards, with identification of the beneficiaries. This space began operating on 1 January 2015, offering data from the state administration. The following year it was extended to regional and local administrations, thus covering the entire spectrum of the public sector.

Initial data formats and functionalities

From its origin, there was a serious support for the re-use of data for the society that originates it. Already initially the website allowed  downloading of data in different formats (CSV, XLSX and PDF) that allow their reuse, although limiting the size of the downloads to 10,000 records for performance and technical capacity reasons.

In addition, a alert subscription mechanismwas available from the outset. A citizen or company could - and can - register as many alerts as they need so that the system automatically notifies them when any call of interest to them is published, sending them a link to it. The tedious need to consult the dozens of official newspapers published in Spain on a daily basis to find out about calls for grants and subsidies was thus eliminated at a stroke.

A project in constant evolution to cope with growth

Regulatory developments in the field of aid and subsidies in the European Union, and Spain's accession to the United Nations Open Government Partnership initiative, shaped the growth of the website in successive years, increasing the offer of specific data views (State aid and minimis, large beneficiaries, political parties, etc.), which made it easier for citizens and data reusers to access them.

At the dawn of the pandemic, the system was already supporting 1.3 million visits per year, serving 3.3 million pages of grants and calls for proposals. This posed a challenge in terms of performance, as volumes never foreseen in the initial technical designs were achieved. A deep technological reform was needed to support the high demand for information and service level.

The reform was approached not only from a technological point of view, but also taking into account:

  • The new features established in the Royal Decree 130/2019 regulating the National Grants Database.
  • The socially demanded need to enable a API-REST interface for downloading information in reusable JSON format to overcome technical limitations.

Characteristics of the current platform

The new website was put into production at the end of November 2023, offering multiple views of calls for proposals (520,000 as of December 2024), grants (27,700,000), State aid awards (5.000,000), grants of minimis (3,190,000), strategic grant plans (1,341), very serious infringements (4), grants to parties (7,580), and grants to large beneficiaries (145,000).

All this information is accessible today via screen, downloadable in PDF, CSV, XSLX and API-REST interface formats in JSON and XML, being freely reusable by infomediary companies and citizens, with no restrictions other than those established by law.

Figure 1. Capture of the website of the National System for the Publicity of Public Subsidies and Grants (SNPSAP).

Since it went into production 11 months ago, the new system has received 7.5 million visits from citizens and businesses. And through the API-REST interface, thousands of downloads are made daily, making a powerful contribution to the dissemination of "raw" subsidy information for reuse by society for all kinds of analyses, studies, etc. In addition, an average of 35,000 e-mail alerts are issued daily to citizens and companies to inform them of new calls for subsidies.

Advantages of SNPSAP

The social dissemination and reuse of all this information eliminates asymmetries and frictions in the markets, and allows operators and citizens to work more efficiently and productively, resulting in higher levels of welfare for society.

The publication of open data allows citizens and organisations not only to know how public funds are distributed, but also to identify new opportunities. This system ensures that all individuals and organisations have equal access to information, regardless of their size or resources, contributing to a more equitable distribution of public support.

calendar icon
Blog

One of the main requirements of the digital transformation of the public sector concerns the existence of optimal interoperability conditions for data sharing. This is an essential premise from a number of points of view, in particular as regards multi-entity actions and procedures. In particular, interoperability allows:

Interoperability also plays an important role in facilitating the integration of different open data sources for re-use, hence there is even a specific technical standard. It aims to establish common conditions to "facilitate and guarantee the process of re-use of public information from public administrations, ensuring the persistence of the information, the use of formats, as well as the appropriate terms and conditions of use".

Interoperability at European level

Interoperability is therefore a premise for facilitating relations between different entities, which is of particular importance in the European context if we take into account that legal relations will often be between different states. This is therefore a great challenge for the promotion of cross-border digital public services and, consequently, for the enforcement of essential rights and values in the European Union linked to the free movement of persons.

For this reason, the adoption of a regulatory framework to facilitate cross-border data exchange has been promoted to ensure the proper functioning of digital public services at European level. This is Regulation (EU) 2024/903 of the European Parliament and of the Council of 13 March 2024 laying down measures for a high level of public sector interoperability across the Union (known as the Interoperable Europe Act), which is directly applicable across the European Union from 12 July 2024.

This regulation aims to provide the right conditions to facilitate cross-border interoperability, which requires an advanced approach to the establishment and management of legal, organisational, semantic and technical requirements. In particular, trans-European digital public services, i.e. those requiring interaction across Member States' borders through their network and information systems, will be affected. This would be the case, for example, for the change of residence to work or study in another Member State, the recognition of academic diplomas or professional qualifications, access to health and social security data or, as regards legal persons, the exchange of tax data or information necessary to participate in a tendering procedure in the field of public procurement. In short, "all those services that apply the "once-only" principle for accessing and exchanging cross-border data".

What are the main measures it envisages?

  • Interoperability assessment: prior to decisions on conditions for trans-European digital public services by EU entities or public sector bodies of States, the Regulation requires them to carry out an interoperability assessment, although this will only be mandatory from January 2025. The result of this evaluation shall be published on an official website in a machine-readable format that allows for automatic translation.
  • Sharing of interoperability solutions: the above mentioned entities shall be obliged to share interoperability solutions supporting a trans-European digital public service, including technical documentation and source code, as well as references to open standards or technical specifications used. However, there are some limits to this obligation, such as in cases where there are intellectual property rights in favour of third parties. In addition, these solutions will be published on the Interoperable Europe Portal, which will replace the current Joinup portal.
  • Enabling of sandboxes: one of the main novelties consists of enabling public bodies to proceed with the creation of sandboxes or controlled interoperability test areas which, in the case of processing personal data, will be managed under the supervision of the corresponding supervisory authority competent to do so. The aim of this figure is to encourage innovation and facilitate cooperation based on the requirements of legal certainty, thereby promoting the development of interoperability solutions based on a better understanding of the opportunities and obstacles that may arise.
  • Creation of a governance committee: as regards governance, it is envisaged that a committee will be set up comprising representatives of each of the States and of the Commission, which will be responsible for chairing it. Its main functions include establishing the criteria for interoperability assessment, facilitating the sharing of interoperability solutions, supervising their consistency and developing the European Interoperability Framework, among others. For their part, Member States will have to designate at least one competent authority for the implementation of the Regulation by 12 January 2025, which will act as a single point of contact in case there are several. Its main tasks will be to coordinate the implementation of the Act, to support public bodies in carrying out the assessment and, inter alia, to promote the re-use of interoperability solutions.

The exchange of data between public bodies throughout the European Union and its Member States with full legal guarantees is an essential priority for the effective exercise of their competences and, therefore, for ensuring efficiency in carrying out formalities from the point of view of good administration. The new Interoperable European Regulation is an important step forward in the regulatory framework to further this objective, but the regulation needs to be complemented by a paradigm shift in administrative practice. In this respect, it is essential to make a firm commitment to a document management model based mainly on data, which also makes it easier to deal with regulatory compliance with the regulation on personal data protection, and is also fully coherent with the approach and solutions promoted by the Data Governance Regulation when promoting the re-use of the information generated by public entities in the exercise of their functions.


Content prepared by Julián Valero, Professor at the University of Murcia and Coordinator of the Research Group "Innovation, Law and Technology" (iDerTec). The contents and points of view reflected in this publication are the sole responsibility of its author.

calendar icon
Noticia

The 2024 Best Cases Awards of the Public Sector Tech Watch observatory now have finalists. These awards seek to highlight solutions that use emerging technologies, such as artificial intelligence or blockchain, in public administrations, through two categories:

  • Solutions to improve the public services offered to citizens (Government-to-Citizen or G2C).
  • Solutions to improve the internal processes of the administrations themselves (Government-to-Government or G2G).

The awards are intended to create a mechanism for sharing the best experiences on the use of emerging technologies in the public sector and thus give visibility to the most innovative administrations in Europe.

Almost 60% of the finalist solutions are Spanish.

In total, 32 proposals have been received, 14 of which have been pre-selected in a preliminary evaluation. Of these, more than half are solutions from Spanish organisations. Specifically, nine finalists have been shortlisted for the G2G category -five of them Spanish- and five for G2C -three of them linked to our country-.The following is a summary of what these Spanish solutions consist of.

Solutions to improve the internal processes of the administrations themselves.

  • Innovation in local government: digital transformation and GeoAI for data management (Alicante Provincial Council).

Suma Gestión Tributaria, of the Diputación de Alicante, is the agency in charge of managing and collecting the municipal taxes of the city councils of its province. To optimise this task, they have developed a solution that combines geographic information systems and artificial intelligence (machine learning and deep learning) to improve training in detection of properties that do not pay taxes. This solution collects data from multiple administrations and entities in order to avoid delays in the collection of municipalities.

  • Regional inspector of public infrastructures: monitoring of construction sites (Provincial Council of Bizkaia and Interbiak).

The autonomous road inspector and autonomous urban inspector help public administrations to automatically monitor roads. These solutions, which can be installed in any vehicle, use artificial or computer vision techniques along with information from sensors to automatically check the condition of traffic signs, road markings, protective barriers, etc. They also perform early forecasting of pavement degradation, monitor construction sites and generate alerts for hazards such as possible landslides.

  • Application of drones for the transport of biological samples (Centre for Telecommunications and Information Technologies -CTTI-, Generalitat de Catalunya).

This pilot project implements and evaluates a health transport route in the Girona health region. Its aim is to transport biological samples (blood and urine) between a primary health centre and a hospital using drones. As a result, the journey time has been reduced from 20 minutes with ground transport to seven minutes with the use of drones. This has improved the quality of the samples transported, increased flexibility in scheduling transport times and reduced environmental impact.

  • Robotic automation of processes in the administration of justice (Ministry of the Presidency, Justice and Relations with the Courts).

Ministry of the Presidency, Justice and Relations with the Courts has implemented a solution for the robotisation of administrative processes in order to streamline routine, repetitive and low-risk work. To date, more than 25 process automation lines have been implemented, including the automatic cancellation of criminal records, nationality applications, automatic issuance of life insurance certificates, etc. As a result, it is estimated that more than 500,000 working hourshave been saved.

  • Artificial intelligence in the processing of official publications (Official Gazette of the Province of Barcelona and Official Documentation and Publications Service, Barcelona Provincial Council).

CIDO (Official Information and Documentation Search Engine) has implemented an AI system that automatically generates summaries of official publications of the public administrations of Barcelona. Using supervised machine learning and neural networkstechniques, the system generates summaries of up to 100 words for publications in Catalan or Spanish. The tool allows the recording of manual modifications to improve accuracy.

Solutions to improve the public services offered to citizens

  • Virtual Desk of Digital Immediacy: bringing Justice closer to citizens through digitalisation (Ministry of the Presidency, Justice and Relations with the Courts).

The Virtual Digital Immediacy Desktop (EVID) allows remote hearings with full guarantees of legal certainty using blockchain technologies. The solution integrates the convening of the hearing, the provision of documentation, the identification of the participants, the acceptance of consents, the generation of the document justifying the action carried out, the signing of the document and the recording of the session. In this way, legal acts can be carried out from anywhere, without the need to travel and in a simple way, making justice more inclusive, accessible and environmentally friendly. By the end of June 2024, more than 370,000 virtual sessions had been held through EVID.

  • Application of Generative AI to make it easier for citizens to understand legal texts (Entitat Autònoma del Diari Oficial i Publicacions -EADOP-, Generalitat de Catalunya).

Legal language is often a barrier that prevents citizens from easily understanding legal texts. To remove this obstacle, the Government is making available to users of the Legal Portal of Catalonia and to the general public the summaries of Catalan law in simple language obtained from generative artificial intelligence. The aim is to have summaries of the more than 14,000 14,000 existing regulatory provisions adapted to clear communication available by the end of the year. The abstracts will be published in Catalan and Spanish, with the prospect of also offering a version in Aranesein the future.

  • Emi - Intelligent Employment (Consellería de Emprego, Comercio e Emigración de la Xunta de Galicia).

Emi, Intelligent Employment is an artificial intelligence and big data tool that helps the offices of the Public Employment Service of Galicia to orient unemployed people towards the skills required by the labour market, according to their abilities. AI models make six-month projections of contracts for a particular occupation for a chosen geographical area. In addition, they allow estimating the probability of finding employment for individuals in the coming months.

You can see all the solutions presented here. The winners will be announced at the final event on 28 November. The ceremony takes place in Brussels, but can also be followed online. To do so, you need to register here.

Public Sector Tech Watch: an observatory to inspire new projects

Public Sector Tech Watch (PSTW), managed by the European Commission, is positioned as a "one-stop shop" for all those interested - public sector, policy makers, private companies, academia, etc. - in the latest technological developments to improve public sector performance and service delivery. For this purpose, it has several sections where the following information of interest is displayed:

  • Cases: contains examples of how innovative technologies and their associated data are used by public sector organisations in Europe.
  • Stories: presents testimonials to show the challenges faced by European administrations in implementing technological solutions.

If you know of a case of interest that is not currently monitored by PSTW, you can register it here. Successful cases are reviewed and evaluated before being included in the database.

 

calendar icon
Blog

The strong commitment to common data spaces at European level is one of the main axes of the European Data Strategy adopted in 2020. This approach was already announced in that document as a basis, on the one hand, to support public policy momentum and, on the other hand, to facilitate the development of innovative products and services based on data intelligence and machine learning.

However, the availability of large sectoral datasets required, as an unavoidable prerequisite, an appropriate cross-cutting regulatory framework to establish the conditions for feasibility and security from a legal perspective. In this regard, once the reform of the regulation on the re-use of public sector information had been consolidated, with major innovations such as high-value data, the regulation on data governance was approved in 2022 and then, in 2023, the so-called Data Act. With these initiatives already approved and the recent official publication of the Artificial Intelligence Regulation, the promotion of data spaces is of particular importance, especially in the public sector, in order to ensure the availability of sufficient and quality data.

Data spaces: diversity in their configuration and regulation

The European Data Strategy already envisaged the creation of common European data spaces in a number of sectors and areas of public interest, but at the same time did not rule out the launching of new ones. In fact, in recent years, new spaces have been announced, so that the current number has increased significantly, as we shall see below.

The main reason for data spaces is to facilitate the sharing and exchange of reliable and secure data in strategic economic sectors and areas of public interest. Thus, it is not simply a matter of promoting large datasets but, above all, of supporting initiatives that offer data accessibility according to suitable governance models that, ultimately, allow the interoperability of data throughout the European Union on the basis of appropriate technological infrastructures.

Although general characterisations of data spaces can be offered on the basis of a number of common notes, there is a great diversity from a legal perspective in terms of the purposes they pursue, the conditions under which data are shared and, in particular, the subjects involved.

This heterogeneity is also present in spaces related to the public sector, i.e. those in which there is a prominent role for data generated by administrations and other public entities in the exercise of their functions, to which, therefore, the regulation on reuse and open data approved in 2019 is fully applicable.

Which are the European public sector data spaces?

In early 2024, the second version of a European Commission working document was published with the dual objective of providing an updated overview of the European policy framework for data spaces and also identifying European data space initiatives to assess their maturity and the main challenges ahead for each of them.

In particular, as far as public administrations are concerned, four data spaces are envisaged: the legal data space, the public procurement data space, the data space linked to the technical "once only" system in the context of eGovernment and, finally, the security data space for innovation. These are very diverse initiatives which, moreover, present an uneven degree of maturity, so that some have an advanced level of development and solid institutional support, while other cases are only initially sketched out and have considerable effort ahead for their design and implementation.

Let us take a closer look at each of these spaces referred to in the working paper.

1. Legal data space

It is a data space linked to legislation and jurisprudence generated by both the European Union and the Member States. The aim of this initiative is to support the legal professions, public administrations and, in general, to facilitate access to society in order to strengthen the mechanisms of the rule of law. This space has so far been based on two specific initiatives:

  • One concerning information on officially published legislation, which has been articulated through the European Legislation Identifier-ELI. It is a European standard that facilitates the identification of rules in a stable and easily reusable way as it describes legislation with a set of automatically processable metadata, according to a recommended ontology.
  • The second concerns decisions taken by judicial bodies, which are made accessible through an European system of unique identifiers called ECLI (European Case Law Identifier) that is assigned to the decisions of both European and national judicial bodies.

These two important initiatives, which facilitate access to and automated processing of legal information, have required a shift from a document-based management model (official gazette, court decisions) to a data-based model. And it is precisely this paradigm shift that has made it possible to offer advanced information services that go beyond the legal and linguistic limits posed by regulatory and linguistic diversity across the European Union.

In any case, while recognising the important progress they represent, there are still important challenges to be faced, such as facilitating access by specific precepts and not by normative documents or, among others, the availability of judicial documents on the basis of the rules they apply and, also, the linking of the rules with their judicial interpretation by the various judicial bodies in all States. In the case of the latter two scenarios, the challenge is even greater, as they would require the automated linking of both identifiers.

2. Public procurement data space

This is undoubtedly one of the areas with the greatest potential impact, given that in the European Union as a whole, it is estimated that public entities spend around two trillion euros (almost 14% of GDP) on the purchase of services, works and supplies. This space is therefore intended not only to facilitate access to the public procurement market across the European Union but also to strengthen transparency and accountability in public procurement spending, which is essential in the fight against corruption and in improving efficiency.

The practical relevance of this space is reinforced by the fact that it has a specific official document that strongly supports the project and sets out a precise roadmap with the objective of ensuring its deployment within a reasonable timeframe. Moreover, despite limitations in its scope of application (there is no provision for extending the publication obligation to contracts below the thresholds set at European level, nor for contract completion notices), it is at a very advanced stage, in particular as regards the availability of a specific ontology which facilitates the accessibility of information and its re-use by reinforcing the conditions for interoperability.

In short, this space is facilitating the automated processing of public procurement data by interconnecting existing datasets, thus providing a more complete picture of public procurement in the European Union as a whole, even though it has been estimated that there are more than 250,000 contracting authorities awarding public contracts.

3. Single Technical System (e-Government)

This new space is intended to support the need that exists in administrative procedures to collect information issued by the administrations of other States, without the interested parties being required to do so directly. It is therefore a matter of automatically and securely gathering the required evidence in a formalised environment based on the direct interconnection between the various public bodies, which will thus act as authentic sources of the required information.

This initiative is linked to the objective of addressing administrative simplification and, in particular, to the implementation of:

4. Security data space for innovation

The objective here is to improve law enforcement authorities' access to the data needed to train and validate algorithms with the aim of enhancing the use of artificial intelligence and thus strengthening law enforcement in full respect of ethical and legal standards.

While there is a clear need to facilitate the exchange of data between Member States' law enforcement authorities, the working paper emphasises that this is not a priority for AI strategies in this area, and that the advanced use of data in this area from an innovation perspective is currently relatively low.

In this respect, it is appropriate to highlight the initiative for the development of the Europol sandbox, a project that was sponsored by the decision of the Standing Committee on Operational Cooperation on Internal Security (COSI) to create an isolated space that allows States to develop, train and validate artificial intelligence and machine learning models.

Now that the process of digitisation of public entities is largely consolidated, the main challenge for data spaces in this area is to provide adequate technical, legal and organisational conditions to facilitate data availability and interoperability. In this sense, these data spaces should be taken into account when expanding the list of high-value data, along the lines already advanced by the study published by the European Commission in 2023, which emphasises that the data ets with the greatest potential are those related to government and public administration, justice and legal matters, as well as financial data.


Content prepared by Julián Valero, Professor at the University of Murcia and Coordinator of the "Innovation, Law and Technology" Research Group (iDerTec). The contents and points of view reflected in this publication are the sole responsibility of the author.

calendar icon
Blog

There are a number of data that are very valuable, but which by their nature cannot be opened to the public at large. These are confidential data which are subject to third party rights that prevent them from being made available through open platforms, but which may be essential for research that promotes advances for society as a whole, in fields such as medical diagnosis public policy evaluation, detection or prosecution of criminal offences, etc. 

In order to facilitate the extraction of value from these data, in compliance with the regulations in force and the rights attached to them, researchers have been provided with researchers secure processing environments, known as safe rooms, have been made available to researchers. The aim is to enable researchers to request and subsequently use and integrate the data contained in certain databases held by organisations to carry out scientific work in the public interest

All in a controlled, secure and privacy-preserving manner . Therefore, researchers and institutions having access to the data are obliged to maintain absolute confidentiality and not to disseminate any identifiable information.

In this context, the National Statistics Institute (INE), the State Tax Administration Agency (AEAT), various Social Security bodies, the State Public Employment Service (SEPE) and the Bank of Spain have signed an agreement with the National Statistics Institute (INE) have signed an agreement to boost controlled access to this type of data. The agreement is in line with the European Union's strategy and the Data Governance Regulation, as we explained in this article. One of the advantages of this agreement is that it facilitates the cross-referencing of data from different organisations through Es_Datalab.

Es_Datalab, joint access to multiple databases

ES_DataLab is a restricted microdata laboratory for researchers developing projects for scientific and public interest purposes. Access to the microdata takes place in an environment that guarantees the confidentiality of the information, as it does not allow the direct identification of the units, coming from different databases.

To access this environment, you must make an application as described here and access will only be valid for the specified period of the research. The process is as follows:

  1. The researcher must be recognised as a "research entity".  There is currently a register of entities (universities, research institutes, research departments of public administrations, etc.) which will be expanded as new organisations apply to join.
  2. Once accredited, the entity must apply for access to microdata, which requires the submission of a research proposal .

Through Es_ Datalab, it is possible to access to several microdata, collected at this link. In this sense, ES_Datalab facilitates the cross-referencing of databases of the participating institutions, maximising the value that the data can offer to the development of research.

Below are some examples of the data provided by each of the agencies, either through ES_datalab for cross-checking with other sources, or in their own secure processing environments.

The National Institute of Statistics

It currently makes available microdata relating to INE datasets, including:

  • Results of surveys that collect information on the labour market insertion of university graduates, the wage structure, the active population, living conditions, health in Spain, etc.
  • Statistics on various social and economic aspects, such as marriages or deaths, environmental protection activities, subsidiaries of companies abroad, etc.
  • Censuses, both general population and by economic activity (e.g. agricultural census).

The INE, in turn, has its own secure own secure room which facilitates access to confidential data for statistical analysis for scientific purposes in the public interest.

State Tax Administration Agency

The microdata relating to the databases provided by the AEAT include detailed information on:

  • Data on the main items contained in various forms, such as form 100, relating to the annual personal income tax return, form 576, on vehicle registrations, or form 714, on wealth tax, among others.
  • Foreign trade statistics, with both total data and data segmented by sector of activity.

Also noteworthy is the contribution of the Institute for Fiscal Studies, which draws on data from the State Tax Administration Agency (Agencia Estatal de la Administración Tributaria). Linked to the Ministry of Finance, it has made available to the public a Statistics Area of the Institute for Fiscal Studies (Instituto de Estudios Fiscales) as well as its own secure room. Its databases include, for example, personal income tax samples, household panels, income panels and the Spanish sector economic database (BADESPE). The product description and data request protocol is available here here.

Social Security

The Social Security grants access to microdata referring to databases such as:

  • The Continuous Sample of Working Lives (MCVL), which includes individual, current and historical data on contribution bases, affiliations (working life), pensions, cohabitants, Personal Income Tax (IRPF), etc.
  •  Social Security affiliates with monthly information on labour relations, by dates of registration and deregistration of companies, type of contract, collective, regime, province, etc.
  • Benefits recognised in the previous year, including retirement, permanent disability, temporary disability and childbirth and childcare pensions.
  • Other databases such as various budget settlements, temporary redundancy procedures (ERTE) by COVID-19, medical examinations by theSocial Marine Institute (ISM), and the medical examinations of the Social Marine Institute (ISM) or data on maritime or data on student maritime training .

The social Security secure rooms available in Madrid, Barcelona and Albacete, allow the processing of this and other protected information by offering access to a series of secure workstations with various programmes and programming languages (SAS, STATA, R, Python and LibreOffice). Remote access is also allowed through secure devices (called "bastioned devices") that are distributed to researchers.

Thanks to these data, it has been possible to carry out studies on the impact of the retirement age on mortality o the use of paternity leave in Spain.

Bank of Spain

We can also find in Es_Datalab microdata related to the Bank of Spain and databases such as: 

  • Company databases, containing information on individual companies, consolidated non-financial corporate groups or the structure of corporate groups.
  • Macroeconomic data, such as public sector debt or loans to legal entities.
  • Other data relating to sustainability indicators or the household panel.

BELab is the secure data laboratory managed by the Banco de España, offering on-site (Madrid) and remote access. Its data have enabled the development of projects on the effects of the minimum wage on Spanish companies, technology management in the textile sector and machine learning applied to credit risk, among others. You can find out about all projects here both those that have been completed and those that are still in progress.

Boosting the re-use of data through the Data Governance Regulation

All these measures are part of the harmonised approach and processes carried out in implementation of the provisions of the Data Governance Regulation (DGA) to facilitate and encourage the use for scientific research purposes of data held by public sector bodies in the public interest. Likewise, in order to encourage the re-use of specific categories of data held by public sector bodies, the Single National Information Point has been set up at datos.gob.es and managed by the Directorate General for Data.

The aim is to contribute to the advancement of scientific research in our country, while protecting the confidentiality of sensitive data. Safe Rooms are an important resource for the re-use of protected data held by the public sector. They enable controlled processing of information, preserve privacy and other data rights, while facilitating compliance with the European Data Governance Regulation.

calendar icon
Blog

The Open Government Guide for Public Employees is a manual to guide the staff of public administrations at all levels (local, regional and state) on the concept and conditions necessary to achieve an "inclusive open government in a digital environment". Specifically, the document seeks for the administration to assume open government as a cross-cutting element of society, fostering its connection with the Sustainable Development Goals. 

 It is a comprehensive, practical and well-structured guide that facilitates the understanding and implementation of the principles of open government, providing examples and best practices that foster the development of the necessary skills to facilitate the long-term sustainability of open government.

What is open government?

The guide adopts the most widely accepted definition of open government, based on three axes: 

  • Transparency and access to information (vision axis): Refers to open access to public information to facilitate greater accountability.
  • Citizen participation (voice axis): It offers the possibility for citizens to be heard and intervene to improve decision-making and co-creation processes in public policies.
  • Collaboration (value axis): Focuses on cooperation within the administration or externally, with citizens or civil society organizations, through innovation to generate greater co-production in the design and implementation of public services.

This manual defines these axes and breaks them down into their most relevant elements for better understanding and application. According to the guide, the basic elements of open administration are:

  • An integrity that cuts across all public action.
  • Data are "the raw material of governments and public administrations" and, for this reason, must be made available to "any actor", respecting the limits established by law.  The use of information and communication technologies (digital) is conceived as a "space for the expansion of public action", without neglecting the digital divide.
  • The citizenry is placed at the center of open administration, because it is not only the object of public action, but also "must enjoy a leading role in all the dynamics of transparency, participation and collaboration".
  • Sustainability of government initiatives.

Adapted from a visual of the Open Government Guide for Public Employees. Source: https://funcionpublica.hacienda.gob.es/Secretaria-de-Estado-de-Funcion-Publica/Actualidad/ultimas-noticias/Noticias/2023/04/2023_04_11.html

Benefits of Open Government

With all this, a number of benefits are achieved:

  • Increased institutional quality and legitimacy

  • Increased trust in institutions

  • More targeted policies to serve citizens

  • More equitable access to policy formulation

How can I use the guide?

The guide is very useful because, in order to explain some concepts, it poses challenges so that civil servants themselves can reflect on them and even put them into practice. The authors also propose cases that provide an overview of open government in the world and its evolution, both in terms of the concepts related to it and the laws, regulations, relevant plans and areas of application (including Law 19/2023 on transparency, the Digital Spain 2025 agenda, the Digital Rights Charter and the General Data Protection Regulation, known as RGPD). As an example, the cases he mentions include the Elkar-EKIN Social Inclusion Plan of the Provincial Council of Gipuzkoa and Frena La Curva, an initiative launched by members of the Directorate General of Citizen Participation and the LAAAB of the Government of Aragon during COVID-19.

The guide also includes a self-diagnostic test on accountability, fostering collaboration, bibliographical references and proposals for improvement.

 In addition, it offers diagrams and summaries to explain and schematize each concept, as well as specific guidelines to put them into practice. For example, it includes the question "Where are the limits on access to public information? To answer this question, the guide cites the cases in which access can be given to information that refers to a person's ideology, beliefs, religious or union affiliation (p. 26). With adaptation to specific contexts, the manual could very well serve as a basis for organizing training workshops for civil servants because of the number of relevant issues it addresses and its organization.

The authors are right to also include warnings and constructive criticisms of the situation of open government in institutions. Although they do not point out directly, they talk about:

  • Black boxes: they are criticized for being closed systems. It is stated that black boxes should be opened and made transparent and that "the representation of sectors traditionally excluded from public decisions should be increased".
  • Administrative language: This is a challenge for real transparency, since, according to a study mentioned in the guide, out of 760 official texts, 78% of them were not clear. Among the most difficult to understand are applications for scholarships, grants and subsidies, and employment-related procedures.
  • The existence of a lack of transparency in some municipalities, according to another study mentioned in the guide. The global open government index, elaborated by the World Justice Project, places Spain in 24th place, behind countries such as Estonia (14th), Chile (18th), Costa Rica (19th) or Uruguay (21st) and ahead of Italy (28th), Greece (36th) or Romania (51st), among 102 countries. Open Knowledge Foundation has stopped updating its Global Open Data Index, specifically on open data.

In short, public administration is conceived as a step towards an open state, with the incorporation of the values of openness in all branches of government, including the legislative and judicial branches, in addition to government.

Additional issues to consider

For those who want to follow the path to open government, there are a number of issues to consider: 

  • The guide can be adapted to different spheres and scales of public. But public administration is not homogeneous, nor do the people in it have the same responsibilities, motivations, knowledge or attitudes to open government. A review of citizen use of open data in the Basque administration concluded that one obstacle to transparency is the lack of acceptance or collaboration in some sectors of the administration itself. A step forward, therefore, could be to conduct internal campaigns to disseminate the advantages for the administration of integrating citizen perspectives and to generate those spaces to integrate their contributions.

  • Although the black box model is disappearing from the public administration, which is subject to great scrutiny, it has returned in the form of closed and opaque algorithmic systems applied to public administration. There are many studies in the scientific literature -for example, this one- that warn that erroneous opaque box systems may be operating in public administration without anyone noticing until harmful results are generated. This is an issue that needs to be reviewed.
  •  In order to adapt it to specific contexts, it should be possible to define more concretely what participation, collaboration and co-creation are. As the guide indicates, they imply not only transparency, but also the implementation of collaborative or innovative initiatives. But it is also necessary to ask a series of additional questions: what is a collaborative or innovation initiative, what methodologies exist, how is it organized and how is its success measured?
  • The guide highlights the need to include citizens in open government. When talking about inclusion and participation, organized civil society and academia are mentioned above all, for example, in the Open Government Forum. But there is room for improvement to encourage individual participation and collaboration, especially for people with little access to technology. The guide mentions gender, territorial, age and disability digital divides, but does not explore them. However, when access to many public services, aid and assistance has been platformized (especially after the COVID-19 pandemic), such digital divides affect many people, especially the elderly, low-income and women. Since a generalist guide cannot address all relevant issues in detail, this would merit a separate guide.

Public institutions are increasingly turning to algorithmic decision-making for effective, fast and inclusive decision making. Therefore, it is also increasingly relevant to train the administration itself in open government in a digitized, digitized and platformized environment. This guide is a great first step for those who want to approach the subject.


Content prepared by Miren Gutiérrez, PhD and researcher at the University of Deusto, expert in data activism, data justice, data literacy and gender disinformation. The contents and views reflected in this publication are the sole responsibility of the author.

calendar icon
Blog

The regulatory approach in the European Union has taken a major turn since the first regulation on the reuse of public sector information was promoted in 2003. Specifically, as a consequence of the European Data Strategy approved in 2020, the regulatory approach is being expanded from at least two points of view:   

  • on the one hand, governance models are being promoted that take into account the need to integrate, from the design and by default, respect for other legally relevant rights and interests, such as the protection of personal data, intellectual property or commercial secrecy, as has happened in particular through the Data Governance Regulation;   

  • on the other hand, extending the subjective scope of the rules to go beyond the public sector, so that obligations specifically aimed at private entities are also beginning to be contemplated, as shown by the approval in November 2023 of the Regulation on harmonized rules for fair access to and use of data (known as the Data Act). 

In this new approach, data spaces take on a singular role, both in terms of the importance of the sectors they deal with (health, mobility, environment, energy...) and, above all, because of the important role they are called upon to play in facilitating the availability of large amounts of data, specifically in overcoming the technical and legal obstacles that hinder their sharing. In this regard, in Spain we already have a legal provision in this regard, which has materialized with the creation of a specific section in the Public Sector Procurement Platform.  

The Strategy itself envisages the creation of "a common European data space for public administrations, in order to improve transparency and accountability of public spending and the quality of spending, fight corruption at both national and EU level, and address compliance needs, as well as support the effective implementation of EU legislation and encourage innovative applications". At the same time, however, it is recognized that "data concerning public procurement are disseminated through various systems in the Member States, are available in different formats and are not user-friendly", concluding the need, in many cases, to "improve the quality of the data". 

Why a data space in the field of public procurement?  

Within the activity carried out by public entities, public procurement stands out, whose relevance in the economy of the EU as a whole reaches almost 14% of GDP, so it is a strategic pole to boost a more innovative, competitive and efficient economy. However, as expressly recognized in the Commission's Communication Public Procurement: A Data Space to improve public spending, boost data-driven policy making and improve access to tenders for SMEs published in March 2023, although there is a large amount of data on public procurement, however "at the moment its usefulness for taxpayers, public decision-makers and public purchasers is scarce".  

The regulation on public procurement approved in 2014 incorporated a strong commitment to the use of electronic media in the dissemination of information related to the call for tenders and the awarding of procedures, although this regulation suffers from some important limitations: 

  • refers only to contracts that exceed certain minimum thresholds set at European level, which limits the measure to 20% of public procurement in the EU, so that it is up to the States themselves to promote their own transparency measures for the rest of the cases;  

  • does not affect the contractual execution phase, so that it does not apply to such relevant issues as the price finally paid, the execution periods actually consumed or, among other issues, possible breaches by the contractor and, if applicable, the measures adopted by the public entities in this respect;  

  • although it refers to the use of electronic media when complying with the obligation of transparency, it does not, however, contemplate the need for it to be articulated on the basis of open formats that allow the automated reuse of the information. 

Certainly, since the adoption of the 2014 regulation, significant progress has been made in facilitating the standardization of the data collection process, notably by imposing the use of electronic forms for the above-mentioned thresholds as of October 25, 2023. However, a more ambitious approach was needed to "fully leverage the power of procurement data". To this end, this new initiative envisages not only measures aimed at decisively increasing the quantity and quality of data available, but also the creation of an EU-wide platform to address the current dispersion, as well as the combination with a set of tools based on advanced technologies, notably artificial intelligence. 

The advantages of this approach are obvious from several points of view:   

  • on the one hand, it could provide public entities with more accurate information for planning and decision-making;   

  • on the other hand, it would also facilitate the control and supervision functions of the competent authorities and society in general;   

  • and, above all, it would give a decisive boost to the effective access of companies and, in particular, of SMEs to information on current or future procedures in which they could compete. 

What are the main challenges to be faced from a legal point of view?  

The Communication on the European Public Procurement Data Space is an important initiative of great interest in that it outlines the way forward, setting out the potential benefits of its implementation, emphasizing the possibilities offered by such an ambitious approach and identifying the main conditions that would make it feasible. All this is based on the analysis of relevant use cases, the identification of the key players in this process and the establishment of a precise timetable with a time horizon up to 2025.  

The promotion of a specific European data space in the field of public procurement is undoubtedly an initiative that could potentially have an enormous impact both on the contractual activity of public entities and also on companies and, in general, on society as a whole. But for this to be possible, major challenges would also have to be addressed from a legal perspective: 

Firstly, there are currently no plans to extend the publication obligation to contracts below the thresholds set at European level, which would mean that most tenders would remain outside the scope of the area. This limitation poses an additional consequence, as it means leaving it up to the Member States to establish additional active publication obligations on the basis of which to collect and, if necessary, integrate the data, which could pose a major difficulty in ensuring the integration of multiple and heterogeneous data sources, particularly from the perspective of interoperability. In this respect, the Commission intends to create a harmonized set of data which, if they were to be mandatory for all public entities at European level, would not only allow data to be collected by electronic means, but also to be translated into a common language that facilitates their automated processing. 

Secondly, although the Communication urges States to "endeavor to collect data at both the pre-award and post-award stages", it nevertheless makes contract completion notices voluntary. If they were mandatory, it would be possible to "achieve a much more detailed understanding of the entire public procurement cycle", as well as to encourage corrective action in legally questionable situations, both as regards the legal position of the companies that were not awarded the contracts and of the authorities responsible for carrying out audit functions. 

Another of the main challenges for the optimal functioning of the European data space is the reliability of the data published, since errors can often slip in when filling in the forms or, even, this task can be perceived as a routine activity that is sometimes carried out without paying due attention to its execution, as has been demonstrated by administrative practice in relation to the CPVs. Although it must be recognized that there are currently advanced tools that could help to correct this type of dysfunction, the truth is that it is essential to go beyond the mere digitization of management processes and make a firm commitment to automated processing models that are based on data and not on documents, as is still common in many areas of the public sector. Based on these premises, it would be possible to move forward decisively from the interoperability requirements referred to above and implement the analytical tools based on emerging technologies referred to in the Communication. 

The necessary adaptation of European public procurement regulations  

Given the relevance of the objectives proposed and the enormous difficulty involved in the challenges indicated above, it seems justified that such an ambitious initiative with such a significant potential impact should be articulated on the basis of a solid regulatory foundation. It is essential to go beyond recommendations, establishing clear and precise legal obligations for the Member States and, in general, for public entities, when managing and disseminating information on their contractual activity, as has been proposed, for example, in the health data space.  

In short, almost ten years after the approval of the package of directives on public procurement, perhaps the time has come to update them with a more ambitious approach that, based on the requirements and possibilities of technological innovation, will allow us to really make the most of the huge amount of data generated in this area. Moreover, why not configure public procurement data as high-value data under the regulation on open data and reuse of public sector information? 


Content prepared by Julián Valero, Professor at the University of Murcia and Coordinator of the Research Group "Innovation, Law and Technology" (iDerTec). The contents and points of view reflected in this publication are the sole responsibility of its author.

calendar icon
Blog

After several months competing, on 20 October the open data contest organised by the EU came to an end. The EU Datathon is a contest that gives data developers and scientists the chance to demonstrate, through their creativity, the potential of open data.

Although in this post you can find out in detail about the winning projects, in this case we would like to highlight the participation of two Spanish developers whose initiatives were chosen as semi-finalists from amongst the 156 proposals submitted at the start.

In an edition that broke the attendance records, both in terms of the number of contestants and the countries of origin, Antonio Moneo and Manuel Jose García represented Spain with two projects which stood out for their innovative nature regarding the reuse of open data.

Using Artificial Intelligence to optimally solve public tenders

Manuel José García has a PhD in Telecommunications Engineering from the University of Oviedo and currently works as a data scientist at the technology consultancy NTT Data. After scooping up first prize in the Euskadi Open Data contest in 2020, García decided to take part in the European hackathon, making the most of what he had learned from the research carried out in his PhD thesis and which gave rise to the project 'Detection of irregular tenders in Spain through big data analytics and artificial intelligence'.

"It is an initiative that uses Big Data and Artificial Intelligence (AI) to analyse the data from public tenders and to automatically recommend those companies that can best undertake the tender. With this in mind, a search engine is created for companies that can carry out a tender and a form is filled out describing the details that characterise the public tender. From that point onwards, the programme seeks the most suitable companies to carry out the project”, describes Manuel Jose García, who adds that the list of companies recommended by tender is achieved thanks to the fact that the AI model has been trained with the history of hundreds of thousands of tenders and winning companies from the past, learning what type of companies win tenders and what characteristics they have.

An essential requirement in order to participate in the European datathon is to use information from the data catalogues that both Europe and Spain make available to the public at a national, regional and local level. In the specific case of Manuel José García, his project has been developed using the public tender data available at the Public Procurement Platforms.

“The project has been developed using two types of data sources. On the one hand, the public and free data of the tenders and, on the other, the business data required to search for and characterise the companies present in the search engine.  In particular, the annual accounts that companies must submit to the Registrar of Companies have been used. These data are public but paid and it is vogue to make them free as they are data managed by a public entity”, comments the data scientist.

In fact, it is precisely this point related with the data from the Registrar of Companies which entailed a challenge to take the project forward: “Getting structured data from public tenders is complicated, since the open data format of the Spanish Procurement Platform is difficult to handle. In addition, it is necessary to carry out thorough cleaning of the data because their quality is low”, he points out.

In his opinion, if public administrations wish to promote the reuse of open data, “They must promote the culture of data. In other words, to be aware of the importance of the data they handle and store and, in turn, be proactive to exploit said data and make them available to third parties”.

Architecture and open data to make the Sustainable Development Goals visible

In addition to being the Director of Change Management and Advanced Analysis at BBVA, Antonio Moneo was also a semi-finalist in the latest edition of the European datathon thanks to a project that merges art with the dissemination of open data.

Tangible Data is an initiative whose goal is to convert emblematic data series into physical sculptures and thereby be able to lend visibility to issues such as climate change, inequality or the transparency of our governments. Against a backdrop of excess information and a growing digital divide, it is essential to rely on the physical environment to explain what is happening in the world", explains Antonio Moneo and he stresses that "representing data in a sculpture allows us to present a challenge from an objective, respectful perspective”.

When selecting the open data sets, Moneo was clear that he wanted to make visible the realities and statistics related with the Sustainable Development Objectives so that his project would fulfil the social purpose that led him to participate in the datathon: “We use open data from reliable sources and properly licensed with Creative Commons or MIT criteria. Sometimes we have used a private data source, but our objective is to enhance the information that is already available. In addition, we usually use the data in the manner in which they are published and only apply some transformations such as the smoothing of the curves with moving averages that allow us to make the sculptures more pleasant to the touch and, self-evidently, techniques to create volumes in three dimensions that are the basis for the sculptures," he observed.

Hence, to carry out Tangible Data it has been necessary, on the one hand, to build a physical structure and, on the other, to make it invite the user to move to the digital realm where, at the end of the day, the information they seek to make visible can be found. “The first step is to design a 3D model in virtual format which we send off to be produced locally, using the FabLabs network. Later, we include a QR code in the sculpture that allows the audience to know in depth the meaning of the data that it reflects”, explained the promoter of the project.

Developing such an ambitious project, both from a physical point of view and from an informative perspective, is no easy task. The thing is, it is not only a matter of circumventing the design of the sculpture as such, but also of finding the necessary data to transfer the reality that it is sought to represent: “Comparability is one of the biggest challenges we have encountered because, in many cases, the most relevant data for measuring the environment are not always comparable. Sometimes we find data at a regional level, sometimes at a national or local level, but it is not always possible to find all the information you need. This is why, in order to solve this challenge, we have invested more time in searching for data and, in many cases, the initial idea about a sculpture has been modified because we could not find data of sufficient quality”, he went on to say.

Moneo also commented that the other major difficulty that has marked the development of the project has been to access updated data. “Updating is always a critical issue, but right now it is even more so. The consequences of COVID, the war in Ukraine and the current energy crisis paint a very different world from the one we encountered in 2015, when the sustainable development objectives were signed. For example, it is estimated that as a consequence of the pandemic, between 70 and 150 million people will fall into the extreme poverty segment (with less than 1.9 dollars a day). This change, bucking the trend of the last three decades, is not reflected in the World Bank statistics yet as they only go up to 2019. It would this seem that that very relevant data reflect a distorted reality”, he concluded.

A positive review of his time in the EU Datathon

Despite not having reached the final that would have allowed them to compete for part of the total prize, which amounted to 200,000 euros, the two participants agreed that their experience in the datathon has been more than positive. So, whilst Manuel José García believes that "the European Commission must continue to commit to these initiatives so that people are aware of the value of data and the challenges that they can solve”, for his part, Antonio Moneo points out that “this type of action motivates the agencies that drive the data and those who are developing to improve the impact of data on society”.

What's more, both participants have managed to stimulate their professional curiosity thanks to this challenge, whilst simultaneously testing the potential and quality of their respective work vis-à-vis European data experts.

calendar icon
Evento

The Open Government Week, organised by the Open Government Partnership in the framework of Open Gov Week, is taking place this year from 27 June to 1 July 2022. The aim of this initiative is to open up and bring public administrations closer to citizens, based on the Open Government principles mentioned above.In Spain, this initiative is promoted by the Directorate General for Public Governance, in collaboration with the Open Government Forum.

As usual, this year's Open Government Week will feature numerous events organised by public administrations throughout Spain. Due to health circumstances, some of these activities will still be held in a virtual or semi-presential format, along with other events that will be held exclusively in-person.

Online webinar on 29 June to learn about the Data Office and the Aporta Initiative

You can watch the presentations of the event here:

Among the types of activities that will take place during this week, we can find some of a diverse nature such as presentations of government plans and programmes, citizen consultations and participatory processes or open days, among others. Of all of them, we would like to highlight the event to be organised by the Ministry of Economic Affairs and Digital Transformation so that citizens can learn more about the Data Office, and also find out more about the Aporta Initiative of datos.gob.es.

This event, which will be held via an online webinar on Wednesday 29 June, from 12:00 to 13:30, will consist of two parts: one dedicated to further information about the Data Office and the other aimed at disseminating the activity of the datos.gob.es portal.

Part One: Data Office

The Data Office, part of the Secretary of State for Digitalisation and Artificial Intelligence, will be the protagonist of this first part of the online event. They will try to explain how they are addressing the main challenges that exist today in the data economy, defining the legal and political frameworks for data sharing and governance.

The European Union has established as one of the next priorities the promotion of the data economy, and one of the measures that our country has carried out to achieve this goal has been the creation of the Data Office. In this webinar you will be able to learn more about the functions of this body and how it can intervene to promote data sharing as a means to boost the data economy.

Specifically, the Data Office is focused on enabling data sharing through the deployment of data spaces. The data space is key to the development of the data economy, enabling its access, exchange and legitimate re-use, positioning data as a non-rivalrous resource, whose utility grows as its use becomes more widespread in a clear example of the network effect. Data spaces go beyond the bilateral exchange of information, constituting in their most advanced version authentic sharing networks where the value of data is realised.

Part Two: Aporta initiative/datos.gob.es

The second part, which will focus on the Aporta initiative, will be aimed at raising public awareness of the value of open data as a key element of open government and what actions are being taken to encourage its publication and use.

Open data is a key element in driving the data economy. Spain is at the top of the European open data maturity index. In this virtual event you will be able to discover what underlies the philosophy and practice of open data, and what services are offered from the datos.gob.es platform as a one-stop space for reusable data offered by the public sector, among other aspects.

You can follow the event through this link.

Other events of interest of the Open Government Week 2022

Although you can consult the full calendar of events that will take place during Open Government Week at this link, below is a small selection of events related to topics such as open data, transparency, open government and citizen science, among others.

  • 30 June. "Reuse of public sector information beyond open data" (online), University of Valencia, PAGODA: this seminar will analyse, among other issues, how the dynamics of traditional Open Data environments raise questions about the purpose of data use, attribution and traceability of such uses.
  • 27 June. Open Government for All (semi-presential), State Secretariat for Public Administration, Madrid: this event will address the project on Collaboration for Inclusive Communication in Open Government, the result of the agreement signed between the Third Sector Platform and the Ministry of Finance and Public Administration.
  • 29 June. “Presentation of the Open Government Digital Portal of the Fuenlabrada City Council” (in-person), Auditorium of the CIFE - Avda. de las Provincias, 33, Fuenlabrada: description of the new sections of the portal: transparency, open data, participation, economic and infrastructure viewers, etc. Thanks to them, the city council will be able to be accountable more accurately to citizens and they will be able to access administrative information of municipal relevance in an easy and understandable way.

In short, initiatives such as Open Government Week allow citizens to increase their knowledge of and participation in public administrations. Remember that this is just a small selection of events, but you can consult them all on this map.

calendar icon