Europa

The cross-cutting nature of open data on weather and climate data has favoured its use in areas as diverse as precision agriculture, fire prevention or the precision forestry. But the relevance of these datasets lies not only in their direct applicability across multiple industries, but also in their contribution to the challenges related to climate change and environmental sustainability challenges related to climate change and environmental sustainability, which the different action lines of the which the different action lines of the European Green Pact seek to address.

Meteorological data are considered by the European Commission, high value data in accordance with the annex to Regulation 2023/138. In this post we explain which specific datasets are considered to be of high value and the level of availability of this type of data in Spain.

The State Meteorological Agency

In Spain, it corresponds to the State Agency for Meteorology (AEMET) the mission of providing meteorological and climatological services at national level. As part of the Ministry for Ecological Transition and the Demographic Challenge. AEMET leads the related activities of observation, prediction and study of meteorological and climatic conditions, as well as research related to these fields. Its mission includes the provision and dissemination of essential information and forecasts of general interest. This information can also support relevant areas such as civil protection, air navigation, national defence and other sectors of activity.

In order to fulfil this mission, AEMET manages an open data portal that enables the reuse by natural or legal persons, for commercial or non-commercial purposes, of part of the data it generates, prepares and safeguards in the performance of its functions. This portal, known as AEMET OpenData currently offers two modalities for accessing and downloading data in reusable formats:

• General access, which consists of graphical access for the general public through human-friendly interfaces.
• AEMET OpenData API, designed for periodic or scheduled interactions in any programming language, which allows developers to include AEMET data in their own information systems and applications.

In addition, in accordance with Regulation 2023/138, it is envisaged to enable a third access route that would allow re-users to obtain packaged datasets for mass downloading where possible.

In order to access any of the datasets, an access key (API Key) which can be obtained through a simple request in which only an e-mail address is required, without any additional data from the applicant, for the sending of the access key. This is a control measure to ensure that the service is provided with adequate quality and in a non-discriminatory manner for all users.

AEMET OpenData also pioneered the availability of open meteorological data in Europe, reflecting AEMET''s commitment to the continuous improvement of meteorological services, support to the scientific and technological community, and the promotion of a more informed and resilient society in the face of climate challenges.

High-value meteorological datasets

The Annex to Regulation (EU) 2023/138 details five high-value meteorological data sets: weather station observations, validated weather data observations, weather warnings, radar data and numerical prediction model (NMP) data. For each of the sets, the regulation specifies the granularity and the main attributes to be published.

If we analyse the correspondence of the datasets that are currently available grouped in 14 categories in the portal AEMET OpenData portal, with the five datasets that will become mandatory in the coming months, we obtain the conclusions summarised in the following table:

Figure 1: Table showing the equivalence between high value datasets and AEMET OpenData datasets.

The regulation also sets out a number of requirements for publication in terms of format, licence granted, frequency of updating and timeliness, means of access and metadata provided.

In the case of metadata, AEMET publishes, in machine-readable format, the main characteristics of the downloaded file: who prepares it, how often it is prepared, what it contains and its format, as well as information on the data fields (meteorological variable, unit of measurement, etc.). The copyright and terms of use are also specified by means of the legal notice. In this regard, it is foreseen that the current licences will be reviewed to make the datasets available under a licensing scheme compliant with the Regulation, possibly following the recommendation by adopting the license CC BY-SA 4.0.

All in all, it seems that the long track record of the State Meteorological Agency (AEMET) in providing quality open data has put it in a good position to comply with the requirements of the new regulation, making some adjustments to the datasets it already offers through AEMET OpenData to align them with the new obligations. AEMET plans to include in this service the datasets required by the Regulation and which are currently not available, as it adapts its regulations on public prices, as well as the infrastructure and systems that make this possible. Additional datasets that will be available will be ten-minute observation data, hourly climatologies and some data parameters from regional radars and numerical wave and forecast models.

The European open data portal (data.europa.eu) regularly organises virtual training sessions on topical issues in the open data sector, the regulations they affect and related technologies. In this post, we review the key takeaways from the latest webinar on High Value Datasets (HVD).

Among other issues, this seminar focused on transmitting best practices, as well as explaining the experiences of two countries, Finland and the Czech Republic, which were part of the report "High-value Datasets Best Practices in Europe", published by data.europa.eu, together with Denmark, Estonia, Italy, the Netherlands and Romania. The study was conducted immediately after the publication of the HVD implementation regulation in February 2023.

Best practices linked to the provision of high-value data

After an introduction explaining what high-value data are and what requirements they have to meet, the scope of the report was explained in detail during the webinar. In particular, challenges, good practices and recommendations from member states were identified, as detailed below.

Political and legal framework

• There is a need to foster a government culture that is primarily practical and focused on achievable goals, building on cultural values embedded in government systems, such as transparency.
• A strategic approach based on a broader regulatory perspective is recommended, building on previous efforts to implement far-reaching directives such as INSPIRE or DCAT as a standard for data publication. In this respect, it is appropriate to prioritise actions that overlap with these existing initiatives.
• The use of Creative Commons (CC) licences is recommended.
• On a cross-cutting level, another challenge is to combine compliance with the requirements of high-value datasets with the provisions of the General Data Protection Regulation (GDPR), when dealing with sensitive or personal data.

Governance and processes

• Engaging in strategic partnerships and fostering collaboration at national level is encouraged. Among other issues, it is recommended to coordinate efforts between ministries, agencies responsible for different categories of HVD and other related actors, especially in Member States with decentralised governance structures. To this end, it is important to set up interdisciplinary working groups to facilitate a comprehensive data inventory and to clarify which agency is responsible for which dataset. These groups will enable knowledge sharing and foster a sense of community and shared responsibility, which contributes to the overall success of data governance efforts.
• It is recommended to engage in regular exchanges with other Member States, to share ideas and solutions to common challenges.
• There is a need to promote sustainability through the individual accountability of agencies for their respective datasets. Ensuring the sustainability of national data portals means making sure that metadata is maintained with the resources available.
• It is advisable to develop a comprehensive data governance framework by first assessing available resources, including technical expertise, data management tools and key stakeholder input. This assessment process allows for a clear understanding of the rules, processes and responsibilities necessary for an effective implementation of data governance.

Technical aspects, metadata quality and new requirements

• It is proposed to develop a comprehensive understanding of the specific requirements for HVD. This involves identifying existing datasets to determine their compliance with the standards described in the implementing regulation for HVD. There is a need to build a systemic basis for identifying, improving the quality and availability of data by enhancing the overall value of high-value datasets.
• It is recommended to improve the quality of metadata directly at the data source before publishing them in portals, following the DCAT-AP guidelines for publishing high-value datasets and the controlled vocabularies for the six HVD categories. There is also a need to improve the implementation of APIs and bulk downloads from each data source. Its implementation presents significant challenges due to the scarcity of resources and expertise, making capacity building and resourcing essential.
• It is suggested to strengthen the availability of high-value datasets through external funding or strategic planning. The regulation requires all HVD to be accessible free of charge, so some Member States diversify funding sources by seeking financial support through external channels, e.g. by tapping into European projects. In this respect, it is recommended to adapt business models progressively to offer free data.

Finally, the report highlights a suggested eight-step roadmap for compliance with the HVD implementation regulation:

Figure 1: Suggested roadmap for HVD implementation. Adapted from Figure 3 of the European Data Portal's "High-value Datasets Best Practices Report".

The example of the Czech Republic

In a second part of the webinar, the Czech Republic presented their implementation case, which they are approaching from four main tasks: motivation, regulatory implementation, responsibility of public data provider agencies and technical requirements.

• Motivation among the different actors is being articulated through the constitution of working groups.
• Regulatory implementation focuses on dataset analysis and consistency or inconsistency with INSPIRE.
• To boost the accountability of public agencies, knowledge-sharing seminars are being held on linking INSPIRE and HVD using the DCAT-AP standard as a publication pathway.
• Regarding technical requirements, DCAT-AP and INSPIRE requirements are being integrated into metadata practices adapted to their national context. The Czech Republic has developed specifications for local open data catalogues to ensure compatibility with the National Open Data Catalogue. However, its biggest challenge is a strong dependency due to a lack of technical capacities. 

The example of Finland

Finland then took the floor. Having pre-existing legislation (INSPIRE and other specific rules on open data and information management inpublic bodies), Finland required only minor adjustments to align with the national transposition of the HVD directive. The challenge is to understand and make INSPIRE and HVD coexist.

Its main strategy is based on the roadmap on information management in public bodies, which ensures harmonisation, interoperability, high quality management and security to implement the principles of open data. In addition, they have established two working groups to address the implementation of HVD:

• The first group, which is a coordinating group of data promoters, focused on practical and technical issues. As legal experts, they also provided guidance on understanding HVD regulation from a legal perspective.
• The second group is an inter-ministerial coordination group, a working group that ensures that there is no conflict or overlap between HVD regulation and national legislation. This group manages the inventory, in spreadsheet format, containing all the elements necessary for an HVD catalogue. By identifying areas where datasets do not meet these requirements, organisations can establish a roadmap to address the gaps and ensure full compliance over time.

The secretariat of the groups is provided by a geospatial data committee. Both have a wide network of stakeholders to articulate discussion and feedback on the measures taken.

Looking to the future, they highlight as a challenge the need to gain more technical and executive level experience.

End of the session

The webinar continued with the participation of Compass Gruppe (Germany), which markets, among other things, data from the Austrian commercial register. They have a portal that offers this data via APIs through a freemium business model.  

In addition, it was recalled that Member States are obliged to report to Europe every two years on progress in HVD, an activity that is expected to boost the availability of harmonised federated metadata on the European data portal. The idea is that users will be able to find all HVD in the European Union, using the filtering available on the portal or through SPARQL queries.

The combination of the report's conclusions and the experiences of the rapporteur countries give us good clues to guide the implementation of HVD, in compliance with European regulations. In summary, the implementation of HVD poses the following challenges:

• Support the necessary funding to address the opening-up process.
• Overcoming technical challenges to develop efficient access APIs.
• Achieving a proper coexistence between INSPIRE and the HVD regulation
• Consolidate working groups that function as a robust mechanism for progress and convergence.
• Monitor progress and continuously follow up the process.
• Invest in technical training of staff.
• Create and maintain strong coordination in the face of the complex diversity of data holders.
• Potential quality assurance of high value datasets.
• Agree on a standardisation that is necessary from a business point of view.

By addressing these challenges, we will successfully open up high-value data, driving its re-use for the benefit of society as a whole.

You can re-watch the recording of the session here

Spain, as part of the European Union, is committed to the implementation of the European directives on open data and re-use of public sector information. This includes the adoption of initiatives such as the Implementing Regulation (EU) 2023/138 issued by the European Commission, which defines specific guidelines for government entities with regard to the availability of High value datasets (HVD). These data are categorised into themes previously detailed in earlier discussions: Geospatial, Earth Observation and Environment, Meteorology, Statistics, Societies and Societal Properties, and Mobility. In this article we will focus on the last group mentioned.

The Mobility category encompasses data collections falling under the domain of "Transport Networks", as demarcated in Annex I of the Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE). In particular, this Directive refers to the requirement to make available to users datasets relating to road, rail, air and inland waterway networks, with their associated infrastructure, connections between different networks and the trans-European transport network, as defined by Decision No 1692/96/EC of the European Parliament and of the Council of 23 July 1996 on Community guidelines for the development of the trans-European transport network.

In addition, it includes the datasets as described in the Directive 2005/44/EC of the European Parliament and of the Council of 7 September 2005 on harmonised River Information Services (RIS) on inland waterways in the Community. The main objective of the Directive is to improve inland waterway traffic and transport, and it applies to canals, rivers, lakes and ports capable of accommodating vessels of between 1,000 and 1,500 tonnes. These datasets include:

Figure 1: Table with the high value datasets related to Directive 2005/44/EC for the creation of a trans-European river information network.

In order for all of us to make the most of the information available, the Regulation defines some basic rules on how this data is shared:

• Free and easy to use. The data must be ready to be used and shared with everyone for any purpose by acknowledging and citing the source of the data, as prescribed by the Creative Commons BY 4.0 licence.
• Easy to read and use. Data will be presented in a way that both people and computers can easily understand them and everything will be explained in public.
• Direct and easy access. There will be special ways (called APIs) that allow programs to access data automatically. In addition, the user can alternatively download a lot of information at once.
• Always up to date. It is important that data is up to date, so there will be access to the most recent version. But if the user needs to access previous data, it will also be possible to view previous versions.
• Detailed and precise. Data will be shared in as much detail as possible, to a very fine level of accuracy, so that the whole territory is covered when combined.
• Information on information. There will be "information about the information" (metadata) that will tell everything about the data. The metadata shall contain at least the elements listed in the Annex to Commission Regulation (EC) No 1205/2008 of 3 December 2008.
• Understandable and orderly: It will explain well how the data are organised and what all means, in a way that is easy for everyone to understand (structure and semantics).
• Common language. Data shall use vocabularies, code lists and categories that are recognised and accepted at European or global level.

in Spain, who is responsible for the creation and maintenance of mobility data?

In Spain, the responsibility for the creation and maintenance of mobility data generally lies with different governmental entities, depending on the type of mobility and the territorial scope:

• Level national level. The Ministry of Transport and Sustainable Mobility is the main body in charge of mobility in terms of infrastructure and transport at national level. This would include data on roads, railways, air and maritime transport.
• Regional and local level. Autonomous communities and municipalities also play an important role in urban and regional mobility. They are responsible for urban mobility, public transport and public roads, within their respective jurisdictions.
• Public business entities. There are entities such as ADIF (acronym for Administrador de Infraestructuras Ferroviarias, that is Railway Infrastructure Administrator), AENA (acronym for Aeropuertos Españoles y Navegación Aérea, that is Spanish Airports and Air Navigation), Puertos del Estado (State Ports) and others tentities hat manage specific data related to their field of action in rail, air and maritime transport, respectively.

In Spain, the Ministry of Transport and Sustainable Mobility, in collaboration with the autonomous communities, plays a key role in providing access to a wide range of mobility data. In compliance with INSPIRE and LISIGE (Law 14/2010 of 5 July 2010 on geographic information infrastructures and services in Spain, which transposes the INSPIRE Directive), it offers resources such as the Geoportal of the Spatial Data Infrastructure of Spain (IDEE in Spanish acronyms) where citizens and professionals can access geographic data and services, especially with regard to mobility.

Does Spain comply with the HVD Mobility Regulation?

To solve this question we have to go to the INSPIRE Geoportal  where official information classified as high value datasets in Europe is available. Specifically in the mobility category.

Figure 2: Screenshot of the Inspire Geoportal.

As of April 2024 Spain has published the following information in the INSPIRE Geoportal:

• Port service areas in Spain. The port service areas include the cartographic and alphanumeric information of the land service area and water areas I and II. The Spanish State-owned Port System is made up of 46 ports of general interest, managed by 28 Port Authorities.
• Spanish Transport Networks. The Transport Network of the Geographic Reference Information of the National Cartographic System of Spain is a three-dimensional network of national coverage, defined and published in accordance with the INSPIRE Directive, which contemplates five modes of transport: road, rail, inland waterways, air and cable, together with their respective intermodal connections and the infrastructures associated with each mode. This information has the linear geometry of the roads and the punctual geometry of the portals and kilometre points.
• ADIF''s Spanish Rail Transport Network. Public geographic dataset on the adaptation of the Spanish ADIF Common Traamification to the INSPIRE regulations (Transport Networks Annex I).

The publication of these high-value datasets responds positively to the question of Spain''s compliance with the HVD regulation, and is an achievement that reflects Spain''s continued commitment to transparency and access to mobility data.

The joint effort between the Ministry of Transport, Mobility and Urban Agenda, the National Cartographic System and the Autonomous Communities and Public Business Entities underlines the importance of a collaborative approach to mobility information management.

The availability of this data highlights Spain''s commitment to publishing high-value datasets and underlines the importance of continuously improving access to information to optimise inland navigation and mobility data.

Content prepared by Mayte Toscano, Senior Consultant in Data Economy Technologies. The contents and points of view reflected in this publication are the sole responsibility of its author.

The Use Case Observatory is an initiative led by data.europa.eu, the European Open Data Portal. This is a research project on the economic, governmental, social and environmental impact of open data. The project will run for three years, from 2022 to 2025, during which the European Data Portal will monitor 30 cases of open data re-use and publish findings in regular deliverables.

In 2022 it made a first report and now, in April 2024, it has presented volume 2 of the exploratory analysis on the use of open data. In this second instalment, he analyses thirteen of the initial use cases that remain under study, three of them Spanish, and draws the following conclusions:

• The paper first of all underlines the high potential of open data re-use.
• It stresses that many organisations and applications owe their very existence to open data.
• It also points to the need to unlock more broadly the potential impact of open data on the economy, society and the environment.
• To achieve the above point, it points to continued support for the reuse community as crucial to identifying opportunities for financial growth.

The Implementing Regulation (EU) 2023/138 of the European Commission sets clear guidelines for public bodies on the availability of high-value datasets within 16 months. These high-value datasets are grouped into the following themes, which were already described in this post post:

This article focuses on the geospatial category, called High-Value Geospatial Datasets (HVDG).

For all HVDGs, the following shall apply Directive 2007/2/EC of the European Parliament and of the Council of 14 March 2007 establishing an infrastructure for spatial information in the European Community (INSPIRE) with the exception of agricultural and reference parcels, for which Regulation (EU) 2021/211/EEC applies Regulation (EU) No 2021/2116 of the European Parliament and of the Council of 2 December 2021.

As reflected in the table below, the regulation provides detailed information on the requirements to be considered for these HVDGs, such as scales or granularity and attributes of each dataset. These are complementary to the attributes defined in the European Regulation (No 1089/2010), which establishes the interoperability of spatial data sets and services.

To ensure the accessibility and re-use of all these valuable datasets, it is imperative to follow certain provisions to facilitate their publication. Here are the key requirements:

• Open Licence: All datasets must be made available for re-use under a licence Creative Commons BY 4.0 or any equivalent less restrictive open licence. This encourages the freedom to share and adapt information.
• Open and Machine Readable Format: Data should be presented in an open, machine-readable format and be publicly documented. This ensures that the information is easily understandable and accessible to any person or automated system.
• Application Programming Interfaces (APIs) and Mass Download: Application programming interfaces (APIs) should be provided to facilitate programmatic access to data. In addition, direct bulk downloading of datasets should be possible, allowing flexible options for users according to their needs.
• Updated version: The availability of datasets in their most up-to-date version is essential. This ensures that users have access to the latest information, promoting the relevance and accuracy of data.
• Metadata: The description of the data will also be carefully taken care of through the use of metadata. This metadata shall, as a minimum, include the elements as defined in Commission Regulation (EC) No 1205/2008 of 3 December 2008. This regulation implements Directive 2007/2/EC of the European Parliament and of the Council and sets standards for metadata associated with datasets. The use of standardised metadata provides additional information essential for understanding, interpreting and using datasets effectively. By following these standards, interoperability and consistency in reporting is facilitated, thus promoting a more complete and accurate understanding of the available data.

These provisions not only promote transparency and openness, but also facilitate collaboration and the effective use of information in a variety of contexts.

Does Spain comply with the Geospatial HVD Regulation?

The INSPIRE (Infrastructure for Spatial Information in Europe) Directive determines the general rules for the establishment of an Infrastructure for Spatial Information in the European Community based on the Infrastructures of the Member States. Adopted by the European Parliament and Council on 14 March 2007 (Directive 2007/2/EC), entered into force on 25 April 2007.

INSPIRE makes it easier to find, share and use spatial data from different countries and in each of the regions, with HVDs available in the the Commission's new catalogue of HVDs and in each of the catalogues of the Spatial Data Infrastructures of the Autonomous Communities, as well as in the Official INSPIRE Data and Services Catalogue of Spain. The information is available through an online platform whit data from different countries.

WARNING!: They are currently working on this Geoportal carrying out the tasks of data dump, therefore, there may be some temporal inconsistency with the data provided, which correspond to the Official Catalogue of INSPIRE Data and Services (CODSI).

In Spain, we can find the Law 14/2010 of 5 July 2010 on geographic information infrastructures and services in Spain (LISIGE), which transposes Directive 2007/2/EC INSPIRE. This law frames the work to make all national spatial data available and obliges the adaptation of national spatial data to the Technical Guides or Guidelines collected by the INSPIRE Directive, thus ensuring that these data are compatible and interoperable in a Community and cross-border context.

 LISIGE applies to geographic data that meet these conditions:

• Refering to a geographical area of the national territory, the territorial sea, the contiguous zone, the continental shelf and the exclusive economic zone.
• Having been generated by or under the responsibility of public administrations.
• Being in electronic format.
• Their production and maintenance is the responsibility of a public sector administration or body.
• Being within the themes of Annexes I (Geographic Reference Information), II (Fundamental Thematic Data) or III (General Thematic Data) referred to in the aforementioned law

Furthermore, it is clarified that the geographic data and services regulated by the LISIGE will be available on the NSDI Geoportal and on the CODSI as well as in the rest of the catalogues of the Autonomous Communities. The National Geographic Institute (IGN) is responsible for its maintenance.

Thanks to the tireless efforts of the entire Spanish administration since the publication of LISIGE, Spain has achieved a remarkable milestone. It is currently available on the INSPIRE Geoportal a wide range of information classified as High Value Geospatial Data Set (HVDG) is now available on the INSPIRE Geoportal. This achievement reflects our country's continued commitment to transparency and access to high quality geospatial data.

As of January 2024 Spain has published in the INSPIRE Geoportal and in CODSI the following information related to the High Geospatial Value Datasets (HVD) 

• 31 datasets associated with their metadata
• 34 download services (WFS, ATOM Feed, OGC Api Feature)
• 28 visualisation services (WMS, WMTS)

Analysing the sets of high geospatial value we see that, according to the thematics, they have already been published:

Spain currently complies with the HVDG Regulation in all categories. Specifically, it complies with the established legislation at the level of scale or granularity, attributes, license, format, availability of the data in API or maximum download, with being the most updated version and with the metadata.

A detailed analysis of the datasets published under the HVD framework highlights several key issues:

• Comprehensive Geographic Coverage: At least one dataset covering the whole of Spain has been made available.
• Comprehensive Metadata: Metadata has been generated for all High Geospatial Value Datasets (HVDGs). These metadata are published in the Official INSPIRE Catalogue of INSPIRE Data and Services (CODSI), validated to comply with the standards of the Commission Regulation (EC) No 1205/2008.
• Viewing and Downloading Services: All HVDGs have viewing and download services. Download services can be bulk download or download APIs. Currently, they are WFS and ATOM. In the future may be OGC API Feature or API Coverage.
• Open Licences and Open Formats: All published services are licensed under Creative Commons BY 4.0, and download services use standard and open formats such as the GML format documented by the international standard ISO 19136.
• Compliance with INSPIRE Data Models: Almost all datasets comply with the INSPIRE data models, thus ensuring the consistency and quality of the attributes set out in the HVDG regulation.
• Data Updated and Maintained: Download services guarantee the availability of data in the most up-to-date version. Each public administration responsible for the data is responsible for maintaining and updating the information.

This analysis highlights the commitment and efficiency in the management of geospatial data in Spain, contributing to the transparency, accessibility and quality of the information provided to the community.

It should be noted that, in Spain, all HVDG requirements are met. Organisations such as the CNIG and the ICGC or the Government of Navarre, in addition to publishing through WFS or ATOM services, are already working on publishing these datasets with the APIs of OGC.

The INSPIRE Geoportal has become a valuable source of information, thanks to the dedication and collaboration of various governmental entities, including Spanish ones. This breakthrough not only highlights progress in the implementation of geospatial standards, but also strengthens the basis for sustainable development and informed decision-making in Spain. A significant achievement for the country in the geospatial field!

Content prepared by Mayte Toscano, Senior Consultant in Data Economy Technologies. The contents and points of view reflected in this publication are the sole responsibility of its author.

The energy transition is also a transition of raw materials. When we imagine a sustainable future, we conceive it based on a series of strategic sectors such as renewable energies or electric mobility. Similarly, we imagine a connected and digital future, where new innovations and business models related to the fourth industrial revolution allow us to solve global challenges such as food shortages or access to education. In short, we focus on technologies that help us improve our quality of life.

Why are critical minerals important?

These sectors depend on a series of key technologies, such as energy storage batteries, wind turbines, solar panels, electrolyzers, drones, robots, data transmission networks, electronic devices and space satellites. These are technologies that in recent years have undergone a great technological evolution and an enormous growth in demand worldwide. If we analyze the development forecasts to 2030, we can expect annual growth of at least double digits for many of them, as shown in Figure 1.

Figure 1: Expected growth up to 2030 of some of the key technologies for strategic sectors. Source: McKinsey (image 1, image 2, image 3)

However, as can be seen in Figure 2, many of these future technologies are highly dependent on a set of critical raw materials necessary for their development. Indium and gallium are key to the manufacture of energy-efficient LED lighting, silicon is indispensable for the manufacture of microchips and semiconductors, and the platinum group of metals (such as iridium, palladium, platinum rhodium or ruthenium) are used in catalysts for hydrogen electrolyzers.

Figure 2: Semi-quantitative representation of raw material flows to the fifteen key technologies and five strategic sectors. Source: JRC Study

So, when does a material become critical? There are several factors that allow us to determine whether a raw material is considered critical:

• Its world reserves are scarce
• There are no alternative materials that can perform their function (their properties are unique or very unique).
• They are indispensable materials for key economic sectors of the future, and/or their supply chain is high-risk.

In the words of Margrethe Vestager, Executive Vice-President of the European Commission, "without a secure and sustainable supply of critical raw materials, there will be no green (sustainable) and industrial transition".

Research into sources of critical minerals data

In order to know in detail the situation of public minerals in Europe, we need to locate quality data. A task for which we will have to look into several sources.

First of all, we go to the European open data portal. From its search engine, in a first iteration, we see that there are more than 46,000 datasets for the query "critical raw materials" (Figure 3).

After a first analysis of the available data categories, we adjusted the filters until we narrowed down the datasets of interest to 190 (Figure 4). Particular attention is drawn to the data published by the JRC (European Commission Joint Research Center) and, in particular, to the dataset entitled Critical Raw Materials (CRM), 2020 assessment.

Figure 4: Second search for critical raw materials in the European data portal.

This dataset contains a direct link to a web portal, the RMIS (Raw Material Information System), which is actually the European Commission's reference knowledge base on raw materials through which we can access very relevant data and analysis.

Figure 5: RMIS - European Commission's knowledge base for raw materials

Through the RMIS, we find a very interesting publication for any study on the subject. Although this publication is in PDF format, it allows us to access the list of strategic, critical and non-critical materials identified by the European Commission indicating their level of criticality and their use in different key technologies as shown in Figure 6.

Figure 6: Table of strategic, critical and non-critical raw materials used different key technologies contained in the PDF file. Source: Supply chain analysis and material demand forecast in strategic technologies and sectors in the EU - A foresight study, JRC 2023.

Continuing our exploration, in this case in search of data on mineral reserves in the European continent, we found the European Gelological Data Infrastructure (EDGI) platform, which has an extensive catalog with more than 5,700 datasets and geological services. In our case, after performing a search in its data catalog, we selected three datasets containing interesting information in terms of findings of critical lithium, cobalt and graphite minerals (Figure 7).

Figure 7: Searching for datasets in the EDGI catalog

From the EDGI viewer, we can view the contents of these three datasets before downloading them in GeoJSON format (Figure 8). The three datasets have been originated from the  FRAME project (Forecasting And Assessing Europe's Strategic Raw Materials Needs), in which multiple European entities participate, including the Geological and Mining Institute of Spain (IGME).

Figure 8: Querying selected datasets through EDGI visualization platform. Source: Map of cobalt occurrences in Europe, Map of graphite occurrences in Europe, Map of lithium occurrences in Europe, FRAME project.

Lastly, we went to the data portal of the International Energy Agency (IEA) (Figure 9). In this case, we found, among its more than 70 datasets, one directly related to our field of research, entitled Critical Minerals Demand Dataset, which we proceeded to download for further analysis in excel format.

Figure 9: Capture of the International Energy Agency (IEA) data portal.

After this search, we have located some interesting data that can help us to carry out different analyses.

Although this exercise has been carried out under the theme of critical minerals, European open data portals provide a large amount of information and diverse data sets on many areas of interest that can help us understand the challenges we face as a society, from the energy transition to the fight against poverty or food waste. Data that will allow us to carry out analyses aimed at making better decisions to move towards a more prosperous and sustainable future.

Content elaborated by Juan Benavente, industrial engineer and expert in technologies linked to the data economy. The contents and points of view reflected in this publication are the sole responsibility of the author.

In order to comply with Directive (EU) 2019/1024 and its subsequent implementing regulation, EU member states are working on making available so-called high-value datasets (HVD). The aim is to enable citizens and businesses to access such data under technical requirements that favour its re-use and its positive impact on society, the economy and the environment.

Opening up these datasets is a major challenge for public administrations in all EU countries. Although much of this data is already available tousers, countries need to identify it in order to be able to report on it and resolve the high heterogeneity in formats, structures and semantics. In particular, from February 2025, Member States will have to report to the Commission every two years on available high-value datasets, including links to licence conditions and APIs.

To assist in this task, the European Data Portal has published the report "Report on Data Homogenisation for High-value Datasets" where it proposes a methodological approach to facilitate the identification and homogenisation of HVD. Among other issues, the report provides examples of standards that help to achieve greater interoperability not only between data, but also between the applications that use them.

A method for identification and homogenisation

The report describes a methodological approach based on three steps:

1. The identification of HVDs in existing data portals. Although there are some guidelines for HVD publication,  like these for applying DCAT-AP, the naming of already published datasets is not uniform, which makes it difficult to find them. The report proposes a protocol that consists of defining keywords, based on the datasets and their associated attributes, contained in Annex I of the Implementing Regulation. The idea is to use these keywords to search the various existing data portals. The report explains how the identification protocol has been tested with datasets from the categories of business registers, statistical data and transport network data, including tables with the keywords used.
2. Localisation or development of data models, ontologies, controlled vocabularies and/or common APIs. In this section, the report describes some useful resources, which are summarised in the following table:

Source: Report on Data Homogenisation for High-value Datasets

The report also mentions some models to be used in the field of smart cities, such as  Smart Data Models and the Spanish Open Cities.

1. The application of such models. The last step is the actual harmonisation of the data. Once the models to be used have been selected, it is time to apply them. In this phase, the necessary conversion processes will be carried out to provide the data in the appropriate formats and with unified quality metadata. The way in which these transformations are applied will vary depending on the intended end result. For example, it may consist of transforming tabular data (comma-separated values or CSVs, Excel, relational databases, etc.) into other data sources that are also tabular but follow the structure provided in common data models. You can also go further and transform them into tree-based representations (such as JSON) or RDF according to the ontologies and controlled vocabularies you select.

Conclusions of the report

The report ends with a series of conclusions and recommendations. There are still challenges around the identification of HVDs and the implementation of the Implementing Regulation in all European countries, especially in raising awareness and disseminating information about their importance. In HVD categories where there are large data harmonisation initiatives, such as Inspire on geospatial data or Eurostat on statistical HVD, we can find a larger amount of data available in an interoperable and harmonised way. In contrast, in categories where there is no majority initiative, such as companies and company ownership, there is still some way to go to implement the regulation.

The recommendations set out in the European Data Portal report help to shape a roadmap for publishing high-value datasets in each of the categories defined by the European Commission. A challenge that administrations will have to address during 2024 and that will facilitate the re-use of public information.

Building Europe's digital infrastructure of tomorrow

As a global technology race unfolds, Europe is deploying the regulatory framework and investments needed to foster innovation and technological leadership in areas such as online platforms, artificial intelligence, data, cloud, quantum technologies and virtual worlds. In today's rapidly changing economic context, a state-of-the-art telecommunications infrastructure is a key pillar for growth, innovation and job creation.

For this technological revolution to succeed, says EC Internal Market Commissioner Thierry Breton, it must be ensured that European networks are up to the task in terms of transmission speed, storage capacity, computing power and interoperability. To this end, it will seek to promote a Digital Networks Act that will serve to redefine the DNA of Europe's telecommunications sector.

Exploratory consultation on the future of the electronic communications sector and its infrastructure

From 23 February to 19 May 2023, the European Commission conducted an exploratory consultation on the future of the electronic communications sector and its infrastructure. The aim was to gather views from different stakeholders, in particular on the technological and market changes affecting the sector, as well as the types of infrastructure and investments that Europe is expected to need to lead the digital transformation in the coming years. The consultation was divided into four areas: (i) technological and market developments, (ii) fairness for consumers, (iii) barriers to the single market and (iv) fair contribution of all digital actors.

The Commission received close to 500 responses to the consultation from different interest groups such as companies (including network providers as well as large traffic generators), business associations, citizens, non-governmental organisations, academic institutions, consumer organisations and trade unions, as well as comments from public authorities. Most of the responses came from the EU, although there were also participants from other invited countries such as the United States and the United Kingdom.

From the exploratory consultation on the future of the electronic communications sector and its infrastructure, the following conclusions can be drawn:

• The need for innovation and efficient investment in technologies such as network virtualisation, artificial intelligence, open networks and perimeter cloud (in that order of importance), recognising that these will have a substantial impact on the sector in the coming years by driving cost reductions. Network virtualisation is expected to be the technology with the greatest impact by enabling greater flexibility and improved network efficiency, offering a great opportunity to develop new business models. In terms of investment, most respondents expect that a significant part of their annual revenues in the coming years (up to 50% of revenues) will have to be spent on investments in connectivity infrastructure and replacement of devices from providers considered high risk. Public funding is seen as crucial, but questions remain as to whether it will be enough and how to attract more private investment.
• The second conclusion, relating to consumer equity, is that the majority of respondents indicate that overall broadband access prices will decrease in the coming years, although there is more discrepancy when considering high transmission speeds. There is also no consensus among respondents on the effectiveness/efficiency of the Universal Service Obligation rules to protect consumers with special needs, and there is also disagreement on whether it should continue to be financed by the public budget or by network providers.
• It also points to the importance of harnessing the single market to drive investment and innovation, cooperating on key technology developments, and standardising technologies and platform building, so as to support the deployment of initiatives based on federated, interoperable and open source models. The majority of responses indicate that streamlining and simplifying regulation by harmonising best practices at EUlevel would serve to reduce administrative burdens, supply chain and/or regulatory costs, thereby increasing efficiency and speed of infrastructure deployment.
• The fourth conclusion focuses on the need to protect EU networks. In an interconnected world with growing geopolitical tensions, security is critical. Despite advances in the security of 5G networks, gaps remain in the protection of network infrastructure. A more coordinated European approach, especially with regard to further integration of radio spectrum, and with a better aligned auction model and licensing conditions between regions, could improve coverage in border areas and strengthen the EU against harmful external interference.
• Finally, as regards the contribution of digital players to network roll-out, several telco providers anticipate a negative outlook for the next 5 years, driven by the continued fall in unit prices (in terms of EUR/Mbps), which offset the potential revenues from increased data traffic and, therefore, to the detriment of the investments needed to support such traffic. More than half of the respondents answered in the affirmative on the question of whether large digital players should contribute in a fair and proportionate way to the costs of public goods, services and infrastructure, and on the potential introduction of a mandatory mechanism for direct payments from content application providers.

The role of communications networks in the development of data spaces

The data spaces are ecosystems from which to realise the voluntary sharing of data among their participants, based on the creation of an environment of sovereignty, trust and cybersecurity. In contrast to traditional monolithic models, data spaces are virtual, federated environmentsand are therefore established through integrated governance, organisational, regulatory and technical mechanisms.

Data spaces ensure that a large amount of data and algorithms are available for use in the economy and society, while the companies, organisations and individuals that generate these resources retain control over them. As such, these data sets and algorithms will aspire to maintain their residence in the computer systems of their respective owners, connecting with others on an ad hoc basis according to precise needs, which is why data spaces require a renewed infrastructure of communication networks. Based on 5G (or even 6G) technology, data transmission with lower latency and higher capacity is enabled, and also drives the development of edge computing solutions (edge computing), which allow added flexibility for the emerging European Data Economy.

Likewise, operators, through initiatives such as Open Gateway, will also be able to transform their telecommunication networks into value-added platforms, making their capabilities more flexible and available through standardised APIs, with which to develop new applications and digital solutions of greater complexity and scope. Such developments may encourage the participation, collaboration and interoperability of the different actors in the data spaces, with telecommunications operators also playing an important role as facilitators, not only in the development of use cases, but also in the implementation and operation of these use cases.

The European Data Portal, data.europa.eu, has just published its Data Maturity Index, an index that assesses the level of maturity of European countries in terms of open data. For its elaboration, an evaluation survey has been carried out and has been completed by 35 countries, including the 27 Member States of the European Union, three countries of the European Free Trade Association (Iceland, Norway and Switzerland) and five candidate countries (Bosnia and Herzegovina, Montenegro, Albania, Serbia and Serbia and Ukraine).  

In this year's edition, Spain obtained a score of 95% out of 100%. This places it in fifth place overall and in fourth place if only European Union (EU27) member countries are taken into account. This figure represents an improvement of three percentage points over last year's score and places Spain 12 points above the EU27 average (83%).  

The top positions in the ranking are occupied by France, Poland, Ukraine and Estonia. 

Above the EU27 average in all dimensions analyzed 

The index is accompanied by a report containing the analysis carried out and an overview of the good practices applied in Europe. In the case of Spain, it is above the EU-27 average in the four indicators analyzed:  

• Policy, focused on the open data policies of the different countries. It analyzes the existence of national governance models for open data management and the measures that have been applied to implement existing strategies. This is the indicator in which Spain obtains a higher score, with 99% compared to 89% in the EU27. The report highlights how the country's national open data strategy helps promote the openness of public information through innovative and structured actions in collaboration with public and private partners. Among other issues, the strategy includes the objective of identifying business models and business success stories to share successful practices. The report also highlights the existence of various digital strategies that complement specific policies on open data, such as the national artificial intelligence strategy, which includes provisions related to the availability of open data for the operation and training of artificial intelligence systems. 

• Impact, which analyzes the activities undertaken to monitor and measure both the reuse of open data and the impact created by such reuse. Traditionally, this has been the least mature dimension across Europe. Nevertheless, Spain scores 96% compared to 77% in the EU27. The best results are achieved in measuring the impact of open data use cases in the environmental, economic and political sectors.  

• Portal, focused on evaluating the functionalities of the national platform that allow users to access open data and help drive interaction within the community. With 96% compared to 85% in the EU27, Spain stands out in the use of analytical tools to understand user behavior, and in the implementation of strategies to ensure the sustainability of the portal and increase its visibility, including presence in social networks. It also highlights the existence of a private area that allows editors to act according to the feedback received. 

• Quality, which examines the mechanisms for ensuring the quality of (meta)data. Here Spain scores 88% compared to 82% in the EU27. Spain's score is driven by compliance with the DCAT-AP standard (providing educational materials for publishers), the existence of a systematic approach to ensure that metadata is up to date, and the wide range of data offered, both historical and current. 

The report also measures how EU27 countries are progressing in the implementation of the implementing regulation on high-value datasets. In this section, Spain ranks ninth, with 68% implementation. In general, Member States are making more progress on geospatial and statistical datasets. Progress is also being made on the underlying technical and legal requirements.  

Overall recommendations 

The report includes a number of recommendations for Spain, among other countries, including encouraging the development of initiatives at the local and regional level, fostering better coordination between teams, and activating the network of open data officers to implement monitoring activities within their organizations. Emphasis is also placed on the need to promote existing open data courses and promote new training materials, paying special attention to developing strategic awareness of the reuse and impact of open data.

Overall, the report shows good progress in open data across Europe. Although there are areas for improvement, the European open data landscape is consolidating, with Spain at the top of the table.  

In 2024, new waves of implementation of the European data strategy will present national teams with new challenges. On the one hand, they will have to redouble their efforts to inform citizens of the new data sources arising from initiatives such as the Data Governance Act and the data spaces. In this sense, coordination will be necessary between the new figures arising from these legislative developments and the traditional world of open data, enhancing the obvious synergies between the two to boost the data economy and collectivize the value generated. 

The European Commission has published the first Report of the state of the Digital Decade. It takes stock of the progress of the European Union on its way to the success of the digital transformation, as set out in the Digital Decade Policy Agenda 2030. The document builds on the agreements and recommendations adopted at European level on digitization towards 2030, including the Council Recommendation (EU) 2021/2122 of 26 November 2021 on a Pact for Research and Innovation in Europe.  

Various studies and statistical sources, such as the Eurobarometer on the Digital Decade, the Strategic Foresight Report for 2021 and 2022, as well as different statistical sets from Eurostat on the state of technology, have been taken into account in compiling the report. It also includes the monitoring of the European Declaration on Digital Rights and Principles for the Digital Decade, which translates the EU's vision on digital transformation into principles and commitments. 

According to the report, the success of the Digital Decade will be crucial for the future prosperity of the EU. Achieving the agenda could unlock more than €2.8 trillion in economic value, equivalent to 21% of the EU's current economy. To achieve this end, the text highlights the need to incentivize policies and investments in digital technologies, skills and infrastructure.   

On this basis, the report includes concrete recommendations to Member States prior to the adoption of their national strategic roadmaps and for their future adjustments. 

Spain, a major player  

The Commission has prepared a report for each member country in which it exhaustively analyzes its strengths and future duties. About Spain it points out that the country is "in an ambitious digital transformation of the economy" and high rates in connectivity: Spain is, according to the report, one of the EU states with the best results in digital infrastructure.  

The three key aspects analyzed in the report are: 

1. Connectivity and Public Administration  

In this regard, the report points out that Spain is "well above the EU average" in terms of very high capacity fixed networks, with 93% availability compared to 73% of the EU average. In fiber coverage "far exceeds" the EU average, with 91% connectivity compared to 56% of the EU average.   

In terms of other technologies such as 5G, Spain is at the European average, with 82% improved broadband access. In addition, the document reveals that 98% of all pioneering 5G bands have already been allocated in the country and that "appropriate measures are being implemented to achieve the objectives of the Digital Decade" such as the Telecommunications Law of 2022. 

The report also points out that Spain "is at the forefront of e-Government and digital public services in the EU". It also highlights the effort to update services and infrastructures to adapt them to the "rapid technological evolution" and the needs of companies. In fact, Spain is "well above the EU average" in indicators that measure the number of Internet users who use electronic administration services, with 84% compared to 74% in the EU or in digital public services for citizens, with 86% access, and companies, with 91%. In this sense, it highlights the work that has been done in the interoperability of digital public services at national, regional and local level.  

The report includes some recommendations on healthcare digitization and notes that the country should continue to implement its public service digitization policies to "connect more types of healthcare providers to medical records" and implement measures "to ensure regionally comparable quality of service and completeness of healthcare data." 

2. Economic development and companies  

 In terms of economic and technological development, the report praises the country's progress with PERTE Chip, whose objective is the development of semiconductors and cutting-edge technologies to strengthen their design and production in Spanish industry in order to "promote national and EU strategic sovereignty". In this field, it also highlights Spain's participation in the Important Project of Common European Interest (IPCEI) on Microelectronics and Communication Technologies with 11 direct participants active in various fields (material, open source design, equipment, packaging, connectivity, photonics) and that "could help to achieve the first European computer with quantum acceleration by 2025". 

As for SMEs, the level of digital development is at the EU level (68%) and highlights the efforts "to improve the digitization of companies" that can be seen in figures: 12.3% of Spanish companies already use Artificial Intelligence and 14.3% use Big Data for analysis and internal use. In addition, it joins the Kit Digital initiative for the digital acceleration of SMEs and the Agents of Change program to hire experts in digital transformation in the field of small and medium-sized enterprises, a type of company with a high number of companies in the country and a "significant participation in the Spanish economy", which makes them a target for reforms and investments that "will have an indirect multiplier impact". The report encourages the country to continue "implementing these policies" in the area of business digitalization, particularly in the case of SMEs. 

3. Citizen digitization  

In addition, at the citizen level, the report analyzes the digitalization of the Spanish population. In this sense, the document points out that Spain obtains good results in terms of digital skills with 64% and 38% of the population benefiting from these skills respectively. In both cases, the Spanish average is above the EU.  

Within the efforts being made when it comes to boosting ICT profiles both in the educational area and in the labor market, the report highlights measures to increase the number of ICT specialists, in particular a law to modernize the vocational education and training (VET) system, approved in March 2022, and a new VET specialization course on AI and big data that allows training in the new digital environment. 

The report, which analyzes the state of play in 2022, notes the importance of that year, which stands out as having been the period "of a further acceleration of key trends affecting the digital transformation of the EU digital EU" with increasingly rapid technological advances, such as AI, climate change and associated social and economic concerns, a growing demand for high-speed connectivity or the multipolarization of the global scene in the face of an ever-increasing technological race.   

The report concludes that the potential impact of technological change is significant and will require the EU to be agile and swift in its transformation.