datos abiertos

The Spanish Federation of Municipalities and Provinces (FEMP) approved at the end of 2023 two model ordinances that address progress in two key areas: transparency and data governance. Both documents will not only improve the quality of processes, but also facilitate access, management and re-use of data. In this post, we will analyse the second ordinance drafted within the FEMP's Network of Entities for Transparency and Citizen Participation in its quest to define common reference models. In particular, the ordinance on data governance.

The usefulness and good work of the Model Ordinance on Data Governance in Local Entities has been highlighted by the Multisectoral Association of Information (ASEDIE), which awarded it the prize in the category 'Promoting data literacy' at its 15th ASEDIE International Conference.

Under this premise, the document addresses all elements related to the collection, management and exploitation of data in order to approach them as a commongood, i.e. ensuring their openness, accessibility and re-use. This is a relevant objective for local administrations, as it enables them to improve their functioning, service delivery and decision-making. Data governance is the framework that guides and guarantees this process and this ordinance proposes a flexible regulatory framework that different administrations can adapt according to their specific needs.

What is data governance?

 Data Governance comprehensively addresses all aspects related to the collection, management and exploitation of data, as well as its openness and re-use by society as a whole on an equal basis. Itcan therefore bedefined as an organisational function responsible for being accountable for the effective, efficient and acceptable use of databy the organisation, which is necessary to deliver the business strategy. This is described in the specifications UNE 0077:2023 on Data Governance and UNE 78:2023 on Data Management, which include standardised processes to guide organisations in the establishment of approved and validated mechanisms that provide organisational support to aspects related to the opening and publication of data, for subsequent use by citizens and other institutions.

How was the FEMP Data Governance Ordinance developed?

In order to develop the Model Ordinance on Data Governance in the Local Entity, a multidisciplinary working group was set up in 2022, which included workers from the Public Administrations, private companies, representatives of the infomediary sector, the Data Office, universities, etc. This team set out two main objectives that would mark the content of the document:

• Develop guidelines for municipalities and other public authorities defining the strategy to be followed in order to implement an open data project.
• Create a reference model of datasets common to all public administrations to facilitate the re-use of information.

With these two challenges in mind, in early 2023 the FEMP working group started to establish aspects, structure, contents and work plan. During the following months, work was carried out to draft, elaborate and reach consensus on a single draft.

In addition, a participatory process was organised on the Idea Zaragoza platform to nurture the document with contributions from experts from all over the country and FEMP partners.

The result of all the work was based on the Open Data Charter (ODC), the recommendations issued by the Spanish Government's Data Office and the existing European and national regulations on this matter.

New features and structure of the Data Governance Ordinance

The FEMP's Model Ordinance on Data Governance is in line with the context in which it has been presented, i.e. it recognises relevant aspects of the current moment we are living in. One of the document's salient features is the premise of guaranteeing and enhancing the rights of both natural and legal persons and respecting the General Data Protection Regulation. The regulation places particular emphasis on the proportionality of anonymisation to ensure the privacy of individuals.

Another novel aspect of the standard is that it brings the vision of high-value data defined by the European Commission from the perspective of local government. In addition, the Model Ordinance recognises a single regime for access and re-use of public information, in accordance with Law 19/2013 of 9 December on transparency, access to public information and good governance, and Law 37/2007 on the re-use of public sector information.

Beyond ensuring the legal and regulatory framework, the FEMP Ordinance also addresses the data associated with artificial intelligence, a cutting-edge technological synergy that every day offers great innovative solutions. For an artificial intelligence to function properly, it is necessary to have quality data to help train it. In relation to this point, the ordinance defines quality requirements (Article 18) and metrics for their assessment that are adapted to each specific context and address issues such as accuracy, portability or confidentiality, among others.  The document establishes guarantees that the use of the data will be carried out in a way that respects the rights of individuals.

All these new aspects are part of the FEMP's Model Ordinance on Data Governance for Local Entities, which is organised in the following structure:

1. General provisions: This first section presents data as the main digital asset of Public Administrations as a strategic asset, and the object, principles and right of citizenship.
2. Planning, organisation and tools for data governance: Here the organisation and competencies for data governance are defined. In addition, the importance of maintaining an inventory of datasets and information sources is stressed (Article 9).
3. The data: This chapter recognises the publication requirements and security standards, the importance of the use of reference vocabularies, and the categories of datasets whose openness should be prioritised, namely the 80 typologies referred to by FEMP as most relevant.
4. Life cycle: This section highlights, on the one hand, the collection, opening, storage and use of data; and, on the other hand, the limits, deletion and destruction of data when these actions are required.  when these actions are required.
5. Access, publication and re-use: The fifth chapter deals with issues related to the exploitation of data such as the use of specific licences, exclusive rights, payment for re-use or prior request for access to certain datasets.
6. Liability and guarantees: The last point describes the sanctioning and disciplinary regime and the civil and criminal liabilities of the re-user.

In short, the publication of the Ordinance on Data Governance in Local Entities provides local administrations with a flexible regulation and defines administrative structures that seek to improve management, reuse and the promotion of a data-driven society.

You can access the full document here: Standard Ordinance on Data Governance in the Local Entity

The Centre de documentació i biblioteca del Institut Català d'Arqueologia Clàssica (ICAC) has the repository Open Science ICAC. This website is a space where science is shared in an accessible and inclusive way. The space introduces recommendations and advises on the process of publishing content. Also, on how to make the data generated during the research process available for future research work.

The website, in addition to being a repository of scientific research texts, is also a place to find tools and tips on how to approach the research data management process in each of its phases: before, during and at the time of publication.

• Before you begin: create a data management plan to ensure that your research proposal is as robust as possible. The Data Management Plan (DMP) is a methodological document that describes the life cycle of the data collected, generated and processed during a research project, a doctoral thesis, etc.
• During the research process: at this point it points out the need to unify the nomenclature of the documents to be generated before starting to collect files or data, in order to avoid an accumulation of disorganised content that will lead to lost or misplaced data. In addition, this section provides information on directory structure, folder names and file names, the creation of a txt file (README) describing the nomenclatures or the use of short, descriptive names such as project name/acronym, file creation date, sample number or version number. Recommendations on how to structure each of these fields so that they are reusable and easily searchable can also be found on the website.
• Publication of research data: in addition to the results of the research itself in the form of a thesis, dissertation, paper, etc., it recommends the publication of the data generated by the research process itself. The ICAC itself points out that research data remains valuable after the research project for which it was generated has ended, and that sharing data can open up new avenues of research without future researchers having to recreate and collect identical data. Finally, it outlines how, when and what to consider when publishing research data.

Graphical content for improving the quality of open data

Recently, the ICAC has taken a further step to encourage good practice in the use of open data. To this end, it has developed a series of graphic contents based on the "Practical guide for the improvement of the quality of open data"produced by datos.gob.es. Specifically, the cultural body has produced four easy-to-understand infographics, in Catalan and English, on good practices with open data in working with databases and spreadsheets, texts and docs and CSV format.

All the infographics resulting from the adaptation of the guide are available to the general public and also to the centre's research staff at Recercat, Catalonia's research repository. Soon it will also be available on the Open Science website of the Institut Català d'Arqueologia Clàssica (ICAC)open Science ICAC.

The infographics produced by the ICAC review various aspects. The first ones contain general recommendations to ensure the quality of open data, such as the use of standardised character encoding, such as UTF-8, or naming columns correctly, using only lowercase letters and avoiding spaces, which are replaced by hyphens. Among the recommendations for generating quality data, they also include how to show the presence of null or missing data or how to manage data duplication, so that data collection and processing is centralised in a single system so that, in case of duplication, it can be easily detected and eliminated.

The latter deal with how to set the format of thenumerical figures and other data such as dates, so that they follow the ISO standardised system, as well as how to use dots as decimals. In the case of geographic information, as recommended by the Guide, its materials also include the need to reserve two columns for inserting the longitude and latitude of the geographic points used.

The third theme of these infographics focuses on the development of good databases or spreadsheets databases or spreadsheetsso that they are easily reusable and do not generate problems when working with them. Among the recommendations that stand out are consistency in generating names or codes for each item included in the data collection, as well as developing a help guide for the cells that are coded, so that they are intelligible to those who need to reuse them.

In the section on texts and documents within these databases, the infographics produced by the Institut Català d'Arqueologia Clàssica include some of the most important recommendations for creating texts and ensuring that they are preserved in the best possible way. Among them, it points to the need to save attachments to text documents such as images or spreadsheets separately from the text document. This ensures that the document retains its original quality, such as the resolution of an image, for example.

Finally, the fourth infographic that has been made available contains the most important recommendations for working with CSV format working with CSV format (comma separated value) format, such as creating a CSV document for each table and, in the case of working with a document with several spreadsheets, making them available independently. It also notes in this case that each row in the CSV document has the same number of columns so that they are easily workable and reusable, without the need for further clean-up.

As mentioned above, all infographics follow the recommendations already included in the Practical guide for improving the quality of open data.

The guide to improving open data quality

The "Practical guide for improving the quality of open data" is a document produced by datos.gob.es as part of the Aporta Initiative and published in September 2022. The document provides a compendium of guidelines for action on each of the defining characteristics of quality, driving quality improvement. In turn, this guide takes the data.europe.eu data quality guide, published in 2021 by the Publications Office of the European Union, as a reference and complements it so that both publishers and re-users of data can follow guidelines to ensure the quality of open data.

In summary, the guide aims to be a reference framework for all those involved in both the generation and use of open data so that they have a starting point to ensure the suitability of data both in making it available and in assessing whether a dataset is of sufficient quality to be reused in studies, applications, services or other.

The Open Government Guide for Public Employees is a manual to guide the staff of public administrations at all levels (local, regional and state) on the concept and conditions necessary to achieve an "inclusive open government in a digital environment". Specifically, the document seeks for the administration to assume open government as a cross-cutting element of society, fostering its connection with the Sustainable Development Goals. 

 It is a comprehensive, practical and well-structured guide that facilitates the understanding and implementation of the principles of open government, providing examples and best practices that foster the development of the necessary skills to facilitate the long-term sustainability of open government.

What is open government?

The guide adopts the most widely accepted definition of open government, based on three axes: 

• Transparency and access to information (vision axis): Refers to open access to public information to facilitate greater accountability.
• Citizen participation (voice axis): It offers the possibility for citizens to be heard and intervene to improve decision-making and co-creation processes in public policies.
• Collaboration (value axis): Focuses on cooperation within the administration or externally, with citizens or civil society organizations, through innovation to generate greater co-production in the design and implementation of public services.

This manual defines these axes and breaks them down into their most relevant elements for better understanding and application. According to the guide, the basic elements of open administration are:

• An integrity that cuts across all public action.
• Data are "the raw material of governments and public administrations" and, for this reason, must be made available to "any actor", respecting the limits established by law.  The use of information and communication technologies (digital) is conceived as a "space for the expansion of public action", without neglecting the digital divide.
• The citizenry is placed at the center of open administration, because it is not only the object of public action, but also "must enjoy a leading role in all the dynamics of transparency, participation and collaboration".
• Sustainability of government initiatives.

Adapted from a visual of the Open Government Guide for Public Employees. Source: https://funcionpublica.hacienda.gob.es/Secretaria-de-Estado-de-Funcion-Publica/Actualidad/ultimas-noticias/Noticias/2023/04/2023_04_11.html

Benefits of Open Government

With all this, a number of benefits are achieved:

• Increased institutional quality and legitimacy

• Increased trust in institutions

• More targeted policies to serve citizens

• More equitable access to policy formulation

How can I use the guide?

The guide is very useful because, in order to explain some concepts, it poses challenges so that civil servants themselves can reflect on them and even put them into practice. The authors also propose cases that provide an overview of open government in the world and its evolution, both in terms of the concepts related to it and the laws, regulations, relevant plans and areas of application (including Law 19/2023 on transparency, the Digital Spain 2025 agenda, the Digital Rights Charter and the General Data Protection Regulation, known as RGPD). As an example, the cases he mentions include the Elkar-EKIN Social Inclusion Plan of the Provincial Council of Gipuzkoa and Frena La Curva, an initiative launched by members of the Directorate General of Citizen Participation and the LAAAB of the Government of Aragon during COVID-19.

The guide also includes a self-diagnostic test on accountability, fostering collaboration, bibliographical references and proposals for improvement.

 In addition, it offers diagrams and summaries to explain and schematize each concept, as well as specific guidelines to put them into practice. For example, it includes the question "Where are the limits on access to public information? To answer this question, the guide cites the cases in which access can be given to information that refers to a person's ideology, beliefs, religious or union affiliation (p. 26). With adaptation to specific contexts, the manual could very well serve as a basis for organizing training workshops for civil servants because of the number of relevant issues it addresses and its organization.

The authors are right to also include warnings and constructive criticisms of the situation of open government in institutions. Although they do not point out directly, they talk about:

• Black boxes: they are criticized for being closed systems. It is stated that black boxes should be opened and made transparent and that "the representation of sectors traditionally excluded from public decisions should be increased".
• Administrative language: This is a challenge for real transparency, since, according to a study mentioned in the guide, out of 760 official texts, 78% of them were not clear. Among the most difficult to understand are applications for scholarships, grants and subsidies, and employment-related procedures.
• The existence of a lack of transparency in some municipalities, according to another study mentioned in the guide. The global open government index, elaborated by the World Justice Project, places Spain in 24th place, behind countries such as Estonia (14th), Chile (18th), Costa Rica (19th) or Uruguay (21st) and ahead of Italy (28th), Greece (36th) or Romania (51st), among 102 countries. Open Knowledge Foundation has stopped updating its Global Open Data Index, specifically on open data.

In short, public administration is conceived as a step towards an open state, with the incorporation of the values of openness in all branches of government, including the legislative and judicial branches, in addition to government.

Additional issues to consider

For those who want to follow the path to open government, there are a number of issues to consider: 

• The guide can be adapted to different spheres and scales of public. But public administration is not homogeneous, nor do the people in it have the same responsibilities, motivations, knowledge or attitudes to open government. A review of citizen use of open data in the Basque administration concluded that one obstacle to transparency is the lack of acceptance or collaboration in some sectors of the administration itself. A step forward, therefore, could be to conduct internal campaigns to disseminate the advantages for the administration of integrating citizen perspectives and to generate those spaces to integrate their contributions.

• Although the black box model is disappearing from the public administration, which is subject to great scrutiny, it has returned in the form of closed and opaque algorithmic systems applied to public administration. There are many studies in the scientific literature -for example, this one- that warn that erroneous opaque box systems may be operating in public administration without anyone noticing until harmful results are generated. This is an issue that needs to be reviewed.
•  In order to adapt it to specific contexts, it should be possible to define more concretely what participation, collaboration and co-creation are. As the guide indicates, they imply not only transparency, but also the implementation of collaborative or innovative initiatives. But it is also necessary to ask a series of additional questions: what is a collaborative or innovation initiative, what methodologies exist, how is it organized and how is its success measured?
• The guide highlights the need to include citizens in open government. When talking about inclusion and participation, organized civil society and academia are mentioned above all, for example, in the Open Government Forum. But there is room for improvement to encourage individual participation and collaboration, especially for people with little access to technology. The guide mentions gender, territorial, age and disability digital divides, but does not explore them. However, when access to many public services, aid and assistance has been platformized (especially after the COVID-19 pandemic), such digital divides affect many people, especially the elderly, low-income and women. Since a generalist guide cannot address all relevant issues in detail, this would merit a separate guide.

Public institutions are increasingly turning to algorithmic decision-making for effective, fast and inclusive decision making. Therefore, it is also increasingly relevant to train the administration itself in open government in a digitized, digitized and platformized environment. This guide is a great first step for those who want to approach the subject.

Content prepared by Miren Gutiérrez, PhD and researcher at the University of Deusto, expert in data activism, data justice, data literacy and gender disinformation. The contents and views reflected in this publication are the sole responsibility of the author.

We are in the last days of the year, those hours that we all take advantage of to mentally review what the previous 12 months have given us. At the Aporta Initiative we are no exception and we want to take advantage of the fact that we are just over 72 hours away from eating our grapes to take stock of what we have done and what is yet to come. 

2023 has been a great year for the entire community of data publishers and users. Artificial intelligence has been in the news on multiple occasions, gaining greater prominence not only at the business level: more and more citizens are beginning to understand the challenges and opportunities that lie ahead. In this context, the quantity and quality of available data has become a pressing need, as a driver of increasingly intelligent applications that help us to make progress as a society. 

In this sense, Spain continues to do its homework and reap good results in international balances. In the last month we have known the results of two indexes that place Spain at the top of the openness and reuse of public information: the European Data Portal considers that Spain ranks fourth in terms of open data in the European Union, while the OECD ranks it fifth worldwide. To these must be added the Report on the State of the Digital Decade, whose scope is broader as it includes many other factors that influence digital transformation, and which also places Spain ahead of the average in digital infrastructures and capabilities. 

datos.gob.es consolidates its position as the meeting point for Spain's open data community   

1.700.000. That is the number of visits that datos.gob.es has received during the last year. A 21% more than in the same period of 2022. A figure that highlights the growing interest in open data in our country. This increase has also been reflected in social networks. The Twitter profile of the Aporta Initiative has consolidated its position as a channel for keeping up to date with news and trends related to data-driven innovation, attracting new users who have reported a growth of 6%, to close to 21,000 followers. Meanwhile, the growth of the community of data professionals around datos.gob.es has been reflected on LinkedIn, attracting 51% more users and reaching a total of 9,000. 

This growth is marked by the incessant activity in favor of data sharing, openness and reuse carried out by the Aporta Initiative and reflected in the datos.gob.es platform:  

•  The number of datasets in the National Data Catalog, hosted at datos.gob.es, has grown by 19%. As of today, users have at their disposal more than 76,000 datasets published by various organizations at national, regional and autonomous community level. Specifically, 77 new publishing organizations have been added. In addition, the datasets already published have been enriched, increasing by 85,000 the available distributions (i.e., the files in various formats in which the data are presented). To ensure its quality, the data.gob.es advisory team has handled more than 600 queries from 140 public institutions. In addition, audits have been carried out, as well as new surveys to promote the opening of new valuable data.   

• The platform has also continued to publish content prepared by various data experts, including aspects related to trends, regulation, success stories, best practices and technical specifications, among others. Specifically, more than 100 articles have been published, 40 examples of solutions and business models based on data (currently the catalog exceeds 500), as well as a multitude of new practical exercises, guides, reports and audiovisual content, such as infographics and videos. 

New data trends  

2024 looks set to be a very promising year in terms of data-related developments. In recent years we have seen great progress at the regulatory level, with various regulations that promote the opening and sharing of data. The most recent of these are the Data Governance Act (DGA), which became fully applicable in September, and the Data Act (DA), which was passed in November. This growing legal landscape means that during 2024 we face the challenge of achieving harmonized implementation to drive a European Digital Single Market.   

This year will also see a major focus on the drive to build data spaces and developments in high-value data. Regarding the latter, June is the deadline for making available to citizens the data sets considered of high value and detailed in the implementing regulation published a year ago, following a series of technical requirements that facilitate their reuse. In addition, the European Commission is already working on a prospection to see possible categories that could be included as high-value data in the future. 

In short, we are facing an exciting year, which will bring many new developments in the field of data, in order to promote not only the data economy but also to be the driving force behind advances that will have an impact on society as a whole. 

The regulatory approach in the European Union has taken a major turn since the first regulation on the reuse of public sector information was promoted in 2003. Specifically, as a consequence of the European Data Strategy approved in 2020, the regulatory approach is being expanded from at least two points of view:   

• on the one hand, governance models are being promoted that take into account the need to integrate, from the design and by default, respect for other legally relevant rights and interests, such as the protection of personal data, intellectual property or commercial secrecy, as has happened in particular through the Data Governance Regulation;   

• on the other hand, extending the subjective scope of the rules to go beyond the public sector, so that obligations specifically aimed at private entities are also beginning to be contemplated, as shown by the approval in November 2023 of the Regulation on harmonized rules for fair access to and use of data (known as the Data Act). 

In this new approach, data spaces take on a singular role, both in terms of the importance of the sectors they deal with (health, mobility, environment, energy...) and, above all, because of the important role they are called upon to play in facilitating the availability of large amounts of data, specifically in overcoming the technical and legal obstacles that hinder their sharing. In this regard, in Spain we already have a legal provision in this regard, which has materialized with the creation of a specific section in the Public Sector Procurement Platform.  

The Strategy itself envisages the creation of "a common European data space for public administrations, in order to improve transparency and accountability of public spending and the quality of spending, fight corruption at both national and EU level, and address compliance needs, as well as support the effective implementation of EU legislation and encourage innovative applications". At the same time, however, it is recognized that "data concerning public procurement are disseminated through various systems in the Member States, are available in different formats and are not user-friendly", concluding the need, in many cases, to "improve the quality of the data". 

Why a data space in the field of public procurement?  

Within the activity carried out by public entities, public procurement stands out, whose relevance in the economy of the EU as a whole reaches almost 14% of GDP, so it is a strategic pole to boost a more innovative, competitive and efficient economy. However, as expressly recognized in the Commission's Communication Public Procurement: A Data Space to improve public spending, boost data-driven policy making and improve access to tenders for SMEs published in March 2023, although there is a large amount of data on public procurement, however "at the moment its usefulness for taxpayers, public decision-makers and public purchasers is scarce".  

The regulation on public procurement approved in 2014 incorporated a strong commitment to the use of electronic media in the dissemination of information related to the call for tenders and the awarding of procedures, although this regulation suffers from some important limitations: 

• refers only to contracts that exceed certain minimum thresholds set at European level, which limits the measure to 20% of public procurement in the EU, so that it is up to the States themselves to promote their own transparency measures for the rest of the cases;  

• does not affect the contractual execution phase, so that it does not apply to such relevant issues as the price finally paid, the execution periods actually consumed or, among other issues, possible breaches by the contractor and, if applicable, the measures adopted by the public entities in this respect;  

• although it refers to the use of electronic media when complying with the obligation of transparency, it does not, however, contemplate the need for it to be articulated on the basis of open formats that allow the automated reuse of the information. 

Certainly, since the adoption of the 2014 regulation, significant progress has been made in facilitating the standardization of the data collection process, notably by imposing the use of electronic forms for the above-mentioned thresholds as of October 25, 2023. However, a more ambitious approach was needed to "fully leverage the power of procurement data". To this end, this new initiative envisages not only measures aimed at decisively increasing the quantity and quality of data available, but also the creation of an EU-wide platform to address the current dispersion, as well as the combination with a set of tools based on advanced technologies, notably artificial intelligence. 

The advantages of this approach are obvious from several points of view:   

• on the one hand, it could provide public entities with more accurate information for planning and decision-making;   

• on the other hand, it would also facilitate the control and supervision functions of the competent authorities and society in general;   

• and, above all, it would give a decisive boost to the effective access of companies and, in particular, of SMEs to information on current or future procedures in which they could compete. 

What are the main challenges to be faced from a legal point of view?  

The Communication on the European Public Procurement Data Space is an important initiative of great interest in that it outlines the way forward, setting out the potential benefits of its implementation, emphasizing the possibilities offered by such an ambitious approach and identifying the main conditions that would make it feasible. All this is based on the analysis of relevant use cases, the identification of the key players in this process and the establishment of a precise timetable with a time horizon up to 2025.  

The promotion of a specific European data space in the field of public procurement is undoubtedly an initiative that could potentially have an enormous impact both on the contractual activity of public entities and also on companies and, in general, on society as a whole. But for this to be possible, major challenges would also have to be addressed from a legal perspective: 

Firstly, there are currently no plans to extend the publication obligation to contracts below the thresholds set at European level, which would mean that most tenders would remain outside the scope of the area. This limitation poses an additional consequence, as it means leaving it up to the Member States to establish additional active publication obligations on the basis of which to collect and, if necessary, integrate the data, which could pose a major difficulty in ensuring the integration of multiple and heterogeneous data sources, particularly from the perspective of interoperability. In this respect, the Commission intends to create a harmonized set of data which, if they were to be mandatory for all public entities at European level, would not only allow data to be collected by electronic means, but also to be translated into a common language that facilitates their automated processing. 

Secondly, although the Communication urges States to "endeavor to collect data at both the pre-award and post-award stages", it nevertheless makes contract completion notices voluntary. If they were mandatory, it would be possible to "achieve a much more detailed understanding of the entire public procurement cycle", as well as to encourage corrective action in legally questionable situations, both as regards the legal position of the companies that were not awarded the contracts and of the authorities responsible for carrying out audit functions. 

Another of the main challenges for the optimal functioning of the European data space is the reliability of the data published, since errors can often slip in when filling in the forms or, even, this task can be perceived as a routine activity that is sometimes carried out without paying due attention to its execution, as has been demonstrated by administrative practice in relation to the CPVs. Although it must be recognized that there are currently advanced tools that could help to correct this type of dysfunction, the truth is that it is essential to go beyond the mere digitization of management processes and make a firm commitment to automated processing models that are based on data and not on documents, as is still common in many areas of the public sector. Based on these premises, it would be possible to move forward decisively from the interoperability requirements referred to above and implement the analytical tools based on emerging technologies referred to in the Communication. 

The necessary adaptation of European public procurement regulations  

Given the relevance of the objectives proposed and the enormous difficulty involved in the challenges indicated above, it seems justified that such an ambitious initiative with such a significant potential impact should be articulated on the basis of a solid regulatory foundation. It is essential to go beyond recommendations, establishing clear and precise legal obligations for the Member States and, in general, for public entities, when managing and disseminating information on their contractual activity, as has been proposed, for example, in the health data space.  

In short, almost ten years after the approval of the package of directives on public procurement, perhaps the time has come to update them with a more ambitious approach that, based on the requirements and possibilities of technological innovation, will allow us to really make the most of the huge amount of data generated in this area. Moreover, why not configure public procurement data as high-value data under the regulation on open data and reuse of public sector information? 

ASEDIE, Asociación Multisectorial de la Información, will hold its usual International Conference on the Reuse of Public Sector Information on December 13. This will be its 15th edition and the central theme is 'Learning from the past, we build the future of data'. The aim of the meeting is to address the interaction that exists between the different regulations that coexist in the data ecosystem, analyze the most significant barriers identified by the sector and discuss solutions. 

When and where will it be held?  

The event, which will be held in classroom format on December 13, 2023 at the Universidad Rey Juan Carlos, in the Salón de Grados of the Departmental Building (Paseo de los Artilleros, Madrid). The reception of attendees begins at 9:00 and the event will end at 13:10. 

What is the program?  

The focus of this edition will be on collective learning, taking as a reference the steps taken so far in the world of data and in the construction of the future of the sector.   

The session will open at 9:40 am with the inauguration of the event by the President of ASEDIE, Ignacio Jiménez and the Director of Public Governance of the Ministry of Finance and Public Function, Ms. Clara Mapelli to give way to the intervention of Carlos Romero, Advisor for Digital Transformation at the Permanent Representation of Spain to the European Union. 

The event will feature two round tables:  

•  The first table will take place at 10:00 and will deal with 'The data economy in search of a coherent regulation'. It will be attended by Santiago Graña, Deputy Director General of Planning and Governance of the Digital Administration; Joaquín Meseguer, Technical Advisor at the Madrid City Council; Lorenzo Avello, Deputy Director General of Telecommunications Management; and Mercedes Ortuño, Advisory Member of the Deputy Directorate General of Promotion and Authorizations of the Spanish Data Protection Agency (AEPD). It will be moderated by Casilda Lazcano, president of ASEDIE's Legal Commission.  

• The second round table, at 12:00h, will focus on the future under the title 'Open data in time perspective: 2024 and beyond'. Participants will include Carmen de Pablos Heredero, Professor of Business Organization at the URJC; Agustín T. de Villar Iglesias, Deputy Director of the Information Infrastructure Area of the Institute of Statistics and Cartography of Andalusia; and Fernando de Pablo Martín, Director General of the Digital Office of the Madrid City Council. 

In between, at 11:00 a.m., the ASEDIE 2023 Awards will be presented in two categories.   

• The finalists in the category Driving the Data Economy are: the Data Office, for the generation of data of proven quality from an effective and standardized government; the City Council of Zaragoza, with its project Know and explore Zaragoza; and the Generalitat de Catalunya, for the classification of datasets by the Sustainable Development Goals.  

• In the category Promoting data knowledge, the finalist initiatives are the Data Management Summit, an event that promotes data knowledge, the Spanish Federation of Municipalities and Provinces, for the standard ordinance on data governance in municipal entities, and the Geographic Institute of Aragon for ICEARAGON, with the project on spatial knowledge of the region.   

After the awards ceremony, Conchita Cornejo, Coordinator of the area of the General Secretariat for Inspection and Control of Capital Movements, and Emilio López, Director of the National Center for Geographic Information, will intervene. 

You can consult the complete program here.  

How can I register?  

Attendance is in person with limited capacity and registrations can be made on the ASEDIE website. 

The concept of High-Value data (High-Value datasets) was introduced by the European Parliament and the Council of the European Union 4 years ago, in Directive (EU) 2019/1024. In it, they were defined as a series of datasets with a high potential to generate "benefits for society, the environment and the economy". Therefore, member states were to push for their openness for free, in machine-readable formats, via APIs, in the form of bulk download and comprehensively described by metadata. 

Initially, the directive proposed in its annex six thematic categories to be considered as high value: geospatial, earth observation and environmental, meteorological, statistical, business records and transport network data. These categories were subsequently detailed in an implementing regulation published in December 2022. In addition, to facilitate their openness, a document with guidelines on how to use DCAT-AP for publication was published in June 2023. 

New categories of data to be considered of high value  

These initial categories were always open to extension. In this sense, the European Commission has just published the report "Identification of data themes for the extensions of public sector High-Value Datasets" which includes seven new categories to be considered as high-value data  

•  Climate loss: This refers to data related to approaches and actions needed to avoid, minimize and address damages associated with climate change. Examples of datasets in this category are economic and non-economic losses from extreme weather events or slow-onset changes such as sea level rise or desertification. It also includes data related to early warning systems for natural disasters, the impact of mitigation measures, or research data on the attribution of extreme events to climate change. 

• Energy: This category includes comprehensive statistics on the production, transport, trade and final consumption of primary and secondary energy sources, both renewable and non-renewable. Examples of data sets to consider are price and consumption indicators or information on energy security.   

• Finance: This is information on the situation of private companies and public administrations, which can be used to assess business performance or economic sustainability, as well as to define spending and investment strategies. It includes datasets on company registers, financial statements, mergers and acquisitions, as well as annual financial reports.  

• Government and public administration: This theme includes data that public services and companies collect to inform and improve the governance and administration of a specific territorial unit, be it a state, a region or a municipality. It includes data relating to government (e.g. minutes of meetings), citizens (census or registration in public services) and government infrastructures. These data are then reused to inform policy development, deliver public services, optimize resources and budget allocation, and provide actionable and transparent information to citizens and businesses. 

• Health: This concept identifies data sets covering the physical and mental well-being of the population, referring to both objective and subjective aspects of people's health. It also includes key indicators on the functioning of health care systems and occupational safety. Examples include data relating to Covid-19, health equity or the list of services provided by health centers.  

• Justice and legal affairs: Identifies datasets to strengthen the responsiveness, accountability and interoperability of EU justice systems, covering areas such as the application of justice, the legal system or public security, i.e. that which ensures the protection of citizens. The data sets on justice and legal matters include documentation of national or international jurisprudence, decisions of courts and prosecutors general, as well as legal acts and their content. 

• Linguistic data: Refers to written or spoken expressions that are at the basis of artificial intelligence, natural language processing and the development of related services. The Commission provides a fairly broad definition of this category of data, all of which are grouped under the term "multimodal linguistic data". They may include repositories of text collections, corpora of spoken languages, audio resources, or video recordings.  

To make this selection, the authors of the report conducted desk research as well as consultations with public administrations, data experts and private companies through a series of workshops and surveys. In addition to this assessment, the study team mapped and analyzed the regulatory ecosystem around each category, as well as policy initiatives related to their harmonization and sharing, especially in relation to the creation of European Common Data Spaces. 

Potential for SMEs and digital platforms   

In addition to defining these categories, the study also provides a high-level estimate of the impact of the new categories on small and medium-sized companies, as well as on large digital platforms. One of the conclusions of the study is that the cost-benefit ratio of data openness is similar across all new topics, with those relating to the categories "Finance" and "Government and public administration" standing out in particular. 

Based on the publicly available datasets, an estimate was also made of the current degree of maturity of the data belonging to the new categories, according to their territorial coverage and their degree of openness (taking into account whether they were open in machine-readable formats, with adequate metadata, etc.). To maximize the overall cost-benefit ratio, the study suggests selecting a different approach for each thematic category: based on their level of maturity, it is recommended to indicate a higher or lower number of mandatory criteria for publication, thus ensuring to avoid overlaps between new topics and existing high-value data.  

You can read the full study at this link. 

Since 24 September last year, the Regulation (EU) 2022/868 of the European Parliament and of the Council of 30 May 2022, on European Data Governance (Data Governance Regulation) has been applicable throughout the European Union. Since it is a Regulation, its provisions are directly effective without the need for transposing State legislation, as is the case with directives. However, with regard to the application of its regulation to Public Administrations, the Spanish legislator has considered it appropriate to make some amendments to the Law 37/2007, of 16 November 2007, on the re-use of public sector information. Specifically:

• A specific sanctioning regime has been incorporated within the scope of the General State Administration for cases of non-compliance with its provisions by re-users, as will be explained in detail below;
• Specific criteria have been established on the calculation of the fees that may be charged by public administrations and public sector entities that are not of an industrial or commercial nature;
• And finally, some singularities have been established in relation to the administrative procedure for requesting re-use, in particular a maximum period of two months is established for notifying the corresponding resolution -which may be extended to a maximum of thirty days due to the length or complexity of the request-, after which the request will be deemed to have been rejected.

What is the scope of this new regulation?

As is the case with the Directive (EU) 2019/1024 of the European Parliament and of the Council of 20 June 2019 on open data and the reuse of public sector informationthis Regulation applies to data generated in the course of the "public service remit" in order to facilitate its re-use. However, the former did not contemplate the re-use of those data protected by the concurrence of certain legal assets, such as confidentiality, trade secrets, the intellectual property or, singularly, the protection of personal data.

You can see a summary of the regulations in this infographic.

Indeed, one of the main objectives of the Regulation is to facilitate the re-use of this type of data held by administrations and other public sector entities for research, innovation and statistical purposes, by providing for enhanced safeguards for this purpose. It is therefore a matter of establishing the legal conditions that allow access to the data and their further use without affecting other rights and legal interests of third parties. Consequently, the Regulation does not establish new obligations for public bodies to allow access to and re-use of information, which remains a competence reserved for Member States. It simply incorporates a number of novel mechanisms aimed at making access to information compatible, as far as possible, with respect for the confidentiality requirements mentioned above. In fact, it is expressly warned that, in the event of a conflict with the Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (GDPR), the latter shall in any case prevail (GDPR), the latter shall in any case prevail.

Apart from the regulation referring to the public sector, to which we will refer below, the Regulation incorporates specific provisions for certain types of services which, although they could also be provided by public entities in some cases, will normally be assumed by private entities. Specifically, intermediation services and the altruistic transfer of data are regulated, establishing a specific legal regime for both cases. The Ministry of Economic Affairs and Digital Transformation will be in charge of overseeing this process in Spain

As regards, in particular, the impact of the Regulation on the public sector, its provisions do not apply to public undertakings , i.e. those in which there is a dominant influence of a public sector body, to broadcasting activities and, inter alia, to cultural and educational establishments. Nor to data which, although generated in the performance of a public service mission, are protected for reasons of public security, defence or national security.

Under what conditions can information be re-used?

In general, the conditions under which re-use is authorised must preserve the protected nature of the information. For this reason, as a general rule, access will be to data that are anonymised or, where appropriate, aggregated, modified or subject to prior processing to meet this requirement. In this respect, public bodies are authorised to charge fees which, among other criteria, are to be calculated on the basis of the costs necessary for the anonymisation of personal data or the adaptation of data subject to confidentiality.

It is also expressly foreseen that access and re-use take place in a secure environment controlled by the public body itself, be it a physical or virtual environment.  In this way, direct supervision can be carried out, which could consist not only in verifying the activity of the re-user, but also in prohibiting the results of processing operations that jeopardise the rights and interests of third parties whose integrity must be guaranteed. Precisely, the cost for the maintenance of these spaces is included among the criteria that can be taken into account when calculating the corresponding fee that can be charged by the public body.

In the case of personal data, the Regulation does not add a new legal basis to legitimise the re-use of personal data other than those already established by the general rules on re-use. Public bodies are therefore encouraged to provide assistance to re-usersin such cases to help them obtain permission from stakeholders. However, this is a support measure that can in no way place disproportionate burdens on the agencies. In this respect, the possibility to re-use pseudonymised data should be covered by some of the cases provided for in the GDPR. Furthermore, as an additional guarantee, the purpose for which the data are intended to be re-used must be compatible with the purpose for which the data were originally intended justified the processing of the data by the public body in the exercise of its main activity, and appropriate safeguards must be adopted.

A practical example of great interest concerns the re-use of health data for biomedical research purposes reuse of health data for biomedical research purposes, which the Spanish legislator which has been established by the Spanish legislator under the provisions of the latter precept. Specifically, the 17th additional provision of Organic Law 3/2018, of 5 December, on the Protection of Personal Data and the Guarantee of Digital Rightsallows the reuse of pseudonymised data in this area when certain specific guarantees are established, which could be reinforced with the use of the aforementioned secure environments in the case of the use of particularly incisive technologies, such as artificial intelligence. This is without prejudice to compliance with other obligations which must be taken into account depending on the conditions of the data processing, in particular the carrying out of impact assessments.

What instruments are foreseen to ensure effective implementation?

From an organisational perspective, States need to ensure thatinformation is easily accessible through a single point. In the case of Spain, this point is available through the platform enabled through the platform datos.gob.esplatform, although there may also be other access points for specific sectors and different territorial levels, in which case they must be linked. Re-users may contact this point in order to make enquiries and requests, which shall be forwarded to thethese will be forwarded to the competent body or entity for processing and response.

The following must also be designated and notified to the notify to the European Commission one or more specialised entities with the appropriate technical and human resources, which could be some of the existing ones, that perform the function of assisting public bodies in granting or refusing re-use. However, if foreseen by European or national regulations, these bodies could assume decision-making functions and not only mere assistance. In any case, it is foreseen that the administrations and, where appropriate, the entities of the institutional public sector, according to the ‑‑according to the terminology of article 2 of Law 27/2007‑‑who make this designation and communicate it to the Ministry of Economic Affairs and Digital Transformationwhich, for its part, will be responsible for the corresponding notification at European level.

Finally, as indicated at the beginning, the following have been classified as specific infringements for the scope of the General Administration of the State certain conducts of re-users which are punishable by fines ranging from 10,001 to 100,000 euros. Specifically, it concerns conduct that, either deliberately or negligently, involves a breach of the main guarantees provided for in European legislation: in particular, failure to comply with the conditions for access to data or to secure areas, re-identification or failure to report security problems.

In short, as pointed out in the European Data Strategyif the European Union wants to play a leading role in the data economy , it is essential, among other measures, to improve governance structures and increase repositories of quality data , which are often affected by significant legal obstacles. With the Data Governance Regulation an important step has been taken at the regulatory level, but it now remains to be seen whether public bodies are able to take a proactive stance to facilitate the implementation of its measures, which ultimately imply important challenges in the digital transformation of their document management.

Content prepared by Julián Valero, Professor at the University of Murcia and Coordinator of the "Innovation, Law and Technology" Research Group (iDerTec).

The contents and points of view reflected in this publication are the sole responsibility of the author.

On September 8, the webinar \"Geospatial Trends 2023: Opportunities for data.europa.eu\" was held, organized by the Data Europa Academy and focused on emerging trends in the geospatial field. Specifically, the online conference addressed the concept of GeoAI (Geospatial Artificial Intelligence), which involves the application of artificial intelligence (AI) combined with geospatial data.

Next, we will analyze the most cutting-edge technological developments of 2023 in this field, based on the knowledge provided by the experts participating in the aforementioned webinar.

What is GeoAI?

The term GeoAI, as defined by Kyoung-Sook Kim, co-chair of the GeoAI Working Group of the Open Geospatial Consortium (OGC), refers to \"a set of methods or automated entities that use geospatial data to perceive, construct (automate), and optimize spaces in which humans, as well as everything else, can safely and efficiently carry out their geographically referenced activities.\"

GeoAI allows us to create unprecedented opportunities, such as:

• Extracting geospatial data enriched with deep learning: Automating the extraction, classification, and detection of information from data such as images, videos, point clouds, and text.
• Conducting predictive analysis with machine learning: Facilitating the creation of more accurate prediction models, pattern detection, and automation of spatial algorithms.
• Improving the quality, uniformity, and accuracy of data: Streamlining manual data generation workflows through automation to enhance efficiency and reduce costs.
• Accelerating the time to gain situational knowledge: Assisting in responding more rapidly to environmental needs and making more proactive, data-driven decisions in real-time.
• Incorporating location intelligence into decision-making: Offering new possibilities in decision-making based on data from the current state of the area that needs governance or planning.

Although this technology gained prominence in 2023, it was already discussed in the 2022 geospatial trends report, where it was indicated that integrating artificial intelligence into spatial data represents a great opportunity in the world of open data and the geospatial sector.

Use Cases of GeoAI

During the Geospatial Trends 2023 conference, companies in the GIS sector, Con terra and 52ºNorth, shared practical examples highlighting the use of GeoAI in various geospatial applications.

Examples presented by Con terra included:

• KINoPro: A research project using GeoAI to predict the activity of the \"black arches\" moth and its impact on German forests.
• Anomaly detection in cell towers: Using a neural network to detect causes of anomalies in towers that can affect the location in emergency calls.
• Automated analysis of construction areas: Aiming to detect building areas for industrial zones using OpenData and satellite imagery.

On the other hand, 52ºNorth presented use cases such as MariData, which seeks to reduce emissions from maritime transport by using GeoAI to calculate optimal routes, considering ship position, environmental data, and maritime traffic regulations. They also presented KI:STE, which applies artificial intelligence technologies in environmental sciences for various projects, including classifying Sentinel-2 images into (un)protected areas.

These projects highlight the importance of GeoAI in various applications, from predicting environmental events to optimizing maritime transport routes. They all emphasize that this technology is a crucial tool for addressing complex problems in the geospatial community.

GeoAI not only represents a significant opportunity for the spatial sector but also tests the importance of having open data that adheres to FAIR principles (Findable, Accessible, Interoperable, Reusable). These principles are essential for GeoAI projects as they ensure transparent, efficient, and ethical access to information. By adhering to FAIR principles, datasets become more accessible to researchers and developers, fostering collaboration and continuous improvement of models. Additionally, transparency and the ability to reuse open data contribute to building trust in results obtained through GeoAI projects.

Reference

The Canary Islands Statistics Institute (ISTAC) has taken a significant step forward in the volume of geographic data thanks to the publication of a total of 4,002 new datasets (3,859 thematic maps and 143 statistical cubes) in datos.gob.es, following its federation in Canarias Datos Abiertos.     

 This type of initiative is aligned with the European Union's Data Strategy, which establishes the guidelines to achieve a single data market that benefits companies, researchers and public administrations. The automation of publication processes through common standards is key to ensure interoperability and adequate access to open data sets of public administrations.

The generation of these datasets is the culmination of an automation work that has allowed the expansion of the number of published cubes, as now combinations of granularity and year since 2004 are presented.  In early October, the ISTAC added to its catalog more than 500 semantic assets and more than 2100 statistical cubes, as we told in this post on datos.gob.es. 

 In addition, the sets published to date have undergone a renewal process to become the aforementioned 143 statistical cubes.  The increase of these datasets not only improves the directory of datos.canarias.es and datos.gob.es in quantitative terms, but also broadens the uses it offers thanks to the type of information added.    

The indicators of these cubes are represented on the cartography through choropleth maps and in multiple formats. This automation will, in turn, not only allow other datasets to be published more easily in the future, but also more frequently.  

Another of the advances of this work is that the services are generated on the fly from the Geoserver map server, and not through an upload to CKAN, as was done until now, which reduces their storage and speeds up their updating. 

How to bring demographic indicators closer to the population   

Demographic indicators are dense data cubes that offer a large amount of detailed geographic information, including total population, disaggregated by sex, residence, age and other indices up to a total of 27 different variables. 

As so much information is contained in each cube, it can be difficult to represent specific indicators on the cartography, especially if the user is not used to working with certain GIS (Geographical Information System) software. 

To bring this content to all types of users, the ISTAC has generated 3,859 new maps, representing on a choropleth map each of the indicators contained in the 143 statistical cubes. The publication of these new cartographic data is thus presented as a more efficient and simplified way of obtaining the information already represented, allowing users to easily access the specific data they need.

We could compare this transformation to flowers.  Previously, only whole bouquets were published, with 27 flowers per bouquet, which had to be managed and handled to represent the flowers that were of interest. Now, in addition to continuing to publish the bouquets, new processes have been generated to be able to publish each flower separately, automating the generation of each of these sets, which will also be updated more frequently.

This new option facilitates the use of these choropleth maps (like the one shown in the image) by people without technical GIS knowledge, since they are presented in easily downloadable formats as images (.jpg and .png) for professional, educational or personal use.

      Mapa de población de 65 o más años (% sobre total) por municipios. Año 2022

For more advanced users, ISTAC has also expanded the range of formats in which the original indicator cubes are served. The "bouquets", which previously only showed data in CSV format, now have a wide variety of distributions: KML, GML, GeoPackage, GeoJSON, WFS, WMS. Taking advantage of the benefits provided by the use of styles in the WMS format, all the styles associated with the indicators have been generated, so that, using them, it is possible to represent the same map that is downloaded in image format. These styles are calculated for each indicator-granularity-year combination, according to the method of calculating quantiles for five intervals. 

This new approach with both simple and complex geographic data enriches the catalog and allows users without specific knowledge to access and reuse them. In addition, it should be noted that this opens the door to other massive publications of data based on other statistical operations.

In short, this is an important step in the process of opening up data.  A process that improves the use and sharing of data, both for the user on the ground and for professionals in the sector. Given the growing need to share, process and compare data, it is essential to implement processes that facilitate interoperability and appropriate access to open data. In this sense, the Canary Islands Institute of Statistics is concentrating its efforts to ensure that its open data sets are accessible and in the appropriate formats for sharing. All this in order to obtain value from them.