The European data portal, data.europa.eu, has published the Open Data Maturity Index 2024, an annual report that assesses the level of open data maturity of European countries.

The 34 participating countries, including the 27 EU Member States, four candidate countries (Bosnia and Herzegovina, Albania, Serbia and Ukraine) and three European Free Trade Association countries (Iceland, Norway and Switzerland) were surveyed.

In this year's edition, Spain obtains an overall rating of 95% out of 100%. This places it in sixth place overall. As reflected in the following image, for yet another year, Spain is in the group of so-called trendsetter countries (trendsetter) , which are those with the best scores in the ranking, and which also include France, Poland, Ukraine, Slovakia, Ireland, Lithuania, Czech Republic, Italy, Estonia and Cyprus.

Figure 1: Groups of participating countries according to their overall open data maturity score.

Above the EU27 average in all four dimensions analysed

The Policy Dimension, focusing on open data policies in different countries, analyses the existence of national governance models for open data management and the measures that have been put in place to implement existing strategies.  In these aspects, Spain scored 96% compared to the European average of 91%. The most positive aspects identified are:

• Alignment with European policies: The report highlights that Spain is fully aligned with the European Open Data Directive, among other recent data-related regulations that have come into force.
• Well-defined action plans: It highlights the strategies deployed in different public administrations focused on incentivising the publication and re-use of data generated in real time and data from citizens.
• Strengthening competences: It focuses on how Spain has developed training programmes to improve the skills of civil servants in managing and publishing open data, ensuring quality standards and fostering a data culture in public administration.

The Impact Dimension analyses the activities carried out to monitor and measure both the re-use of open data, and the impact created as a result of this re-use.  Year after year, this has been the least mature dimension in Europe. Thus, compared to an EU average of 80%, Spain obtains a score of 100% for the development of numerous actions, among which the following stand out:

• Multi-sectoral collaboration: The report highlights how our country is presented as an example of interaction between public administrations, private companies and civil society, materialized in examples such as the close ties between the public sector and the Multisectoral Association of Information (ASEDIE), which produces year after year the ASEDIE report on the reuse of public sector information.
• Examples of re-use in key sectors: It shows how Spain has promoted numerous cases of open data reuse in strategic areas such as the environment, mobility and energy.
• Innovation in communication: The document highlights the effort invested in innovative communication strategies to raise public awareness of the value of data, and especially young audiences. Also noteworthy is the production of podcasts featuring interviews with open data experts, accompanied by short promotional videos.

The Portal Dimension focuses on analysing the functionalities of the national platform to enableusers to access open data and interact within the community. With 96% compared to 82% in the EU27, Spain is positioned as one of the European benchmarks in improving user experience and optimising national portals. The highlights of the report are:  

• Sustainability and continuous improvement: According to the report, Spain has demonstrated a strong commitment to the sustainability of the national open data platform (datos.gob.es) and its adaptation to new technological demands.

• Interaction with users: One of the great strengths is the active promotion from the platform of the datasets available and of the channels through which users can request data that are not available in the National Catalogue.

Finally, the Quality Dimension examines the mechanisms for ensuring the quality of (meta)data. Here Spain scores 88% compared to 79% in the EU. Spain continues to stand out with initiatives that ensure the reliability, accessibility and standardisation of open data. Some of the strengths highlighted in the report are:  

• Metadata automation: It highlights the use of advanced techniques for automatic metadata collection, reducing reliance on manual processes and improving accuracy and real-time updating. 
• Guidelines for data and metadata quality: Spain provides many practical guidelines to improve the publication and quality of open data, including anonymisation techniques, publication in tabular formats (CSV) and the use of APIs.

Continuing to innovate to maintain Spain's advanced position in open data maturity

While Spain continues to stand out in the EU thanks to its open data ecosystem, efforts must continue. To this end, the same report identifies lines of work for countries, such as Spain, that seek to maintain their advanced position in open data maturity and to continue innovating. Among others, the following recommendations are made:

1. Consolidating open data ecosystems: Strengthen thematic communities of providers and re-users by prioritising High Value Datasets (HVDs) in their development and promotion.
2. Promoting coordination: Align the national strategy with the needs of agencies and local authorities.
3. Develop country-specific impact metrics: Collaborate with universities, research institutions and others to develop impact assessment frameworks.
4. Measure and disseminate the impact of open data: Conduct regular (annual or biannual) assessments of the economic, environmental and social impact of open data, promoting the results to generate political support.
5. Facilitate the participation of the open data community: Ensure that providers improve the publication of data based on user feedback and ratings.
6. Increase the quality of data and metadata: Use automated tools and validations to improve publication standards, including adopting artificial intelligence technology to optimise metadata quality.
7. Promote successful reuse cases: Publish and promote success stories in the use of open data, interact with providers and users to identify innovative needs and applications.

Overall, the report shows good progress on open data across Europe. Although there are areas for improvement, the European open data landscape is consolidating, with Spain at the top of the table. Read here the complete Open Data Maturity Index 2024.

The transformative potential of open data initiatives is now widely recognised as they offer opportunities for fostering innovation, greater transparency and improved efficiency in many processes. However, reliable measurement of the real impact of these initiatives is difficult to obtain.

From this same space we have also raised on more than one occasion the recurring question of what would be the best way to measure the impact of open data, we have reviewed different methods and best practices to try to quantify it, as well as to analyse it through detailed use cases or the specific impact on specific topics and sectors such as employment, geographic data, transport or sustainable development objectives. Now, thanks to the report "Indicators for Open Data Impact Assessment" by the data.europa.eu team, we have a new resource to not only understand but also be able to amplify the impact of open data initiatives by designing the right indicators. This publication will provide a quick analysis of the importance of these indicators and also briefly explain how they can be used to maximise the potential of open data.

Understanding open data and its value chain

Open data refers to the practice of making data available to the public in a way that makes it freely accessible and usable. Beyond ensuring simple availability, the real value of open data lies in its use in various domains, fostering economic growth, improving public sector transparency and driving social innovation. However, quantifying the real impact of data openness poses significant challenges due to the multiple ways in which data is used and the wide-ranging implications it can have for society. 

To understand the impact of open data, we must first understand its value chain, which will provide us with a structured and appropriate framework for transforming raw data into actionable insights. This chain includes four main stages that form a continuous process from the initial production to the final use of the data: 

Figure 1: Open Data Value Chain

1. Collection: this consists of identifying existing data and establishing the necessary procedures for their cataloguing.
2. Publication: making data available in an accessible form and easy to locate.
3. Uptake:will come sooner when data is easy to use and accompanied by the right incentives to use it.
4. Application:either through direct consumption of the data or through some transformation that adds new value to the initial data.

Each of these steps will play a critical role in contributing to the overall effectiveness and value derived from open data. The indicators developed to assess the impact of open data will also be closely linked to each of these stages, providing a holistic view of how data is transformed from simple information into a powerful tool for development.

Indicators for impact assessment

The report introduces a set of robust indicators that are specifically designed to monitor open data initiatives through their outputs, outcomes and impact as a result of their value chain. These indicators should not be seen as simple metrics, but as tools to help us understand the effectiveness of open data initiatives and make strategic improvements. Let us look at these indicators in a little more detail:

• Output indicators: are those that focus on measuring the immediate results that come from making open data available. Examples would be the number of datasets released, the frequency of dataset updates , the number of visitors to the data catalogue, the accessibility of the data across various platforms, or even the efforts made to promote the data and increase its visibility. Output indicators help us to assess the efficiency of data publication and dissemination processes and are generally easy to measure, although they will only give a fairly superficial measure of impact.

• Indicators of effect: Outcome indicators measure the short- and medium-term consequences of open data. These indicators are crucial to understand how open data influences decision-makingprocesses, leads to the development of new applications or improves government transparency. Thus, improved public transport planning based on usage data, increased citizen participation in the development of public policies to tackle climate change brought about by the increased availability of data and information, or improved productivity of public services through the use of data can be considered as significant examples of outcome indicators.

• Impact indicators: This is the deepest level of measurement, as impact indicators assess the broader, long-term effects of open data. These indicators may include economic benefits such as job creation or GDP growth, social impacts such as improved trust in public entities, or even environmental impacts such as the effective reduction of greenhouse gases. At this level, indicators are much more complex and specific, and should therefore be defined for each specific domain to be analysed and also according to the objectives set within that domain.

Figure 2: Indicators for impact assessmen

Implementing these indicators in practice will require the implementation of a robust methodological framework that can capture and analyse data from a variety of sources. It is recommended to combine automated and survey-based data collection methods to collect more comprehensive data. This type of dual approach allows for capturing quantitative data through automated systems while also incorporating qualitative insights through user feedback and more in-depth use case analysis.

Looking to the future

The future of open data impact assessment looks towards refining the indicators used for measurement and consolidating them through the use of interactive monitoring tools. Such tools would enable the possibility of a more continuous assessment that would provide real-time information on how open data is being used and its effects in different sectors. In addition, the development of standardised metrics for these indicators would facilitate comparative analysis across regions and over time, further improving the overall understanding of the impact of open data.

Another important factor to take into account are possible privacy and ethical considerations applicable to the selected indicators. As in any other data-centric initiative, privacy and data protection considerations will be paramount and mandatory for the indicators developed. However, once we get into its use by users, it could lead to more delicate situations. Generally, this should not be a particularly problematic issue when monitoring data. However, once we get into its use by users, it could lead to more delicate situations. Ensuring anonymity in indicators and secure practices in their management is also crucial to maintaining trust and integrity in open data processes.

In conclusion, the development and implementation of specific detailed indicators following the recommendations of the report"Indicators for an Open Data Impact Assessment" would be a significant step forward in terms of how we measure and understand the impact of open data.Continuous refinement and adaptation of these indicators will also be crucial as they evolve in tandem with the open data strategies they accompany and their growing sphere of influence. In the medium term the European Commission will further develop its analysis in this area of work through the data.europa.eu project with the ultimate goal of being able to formulate a common methodology for the assessment of the impact of the re-use of public data and to develop an interactive monitoring tool for its implementation.

Content prepared by Carlos Iglesias, Open data Researcher and consultant, World Wide Web Foundation. The contents and views expressed in this publication are the sole responsibility of the author.

Geographic data allow us to learn about the world around us. From locating optimal travel routes to monitoring natural ecosystems, from urban planning and development to emergency management, geographic data has great potential to drive development and efficiency in multiple economic and social areas. They are therefore considered high-value datasets by the European Commission, and have a specific obligations to make their publication accessible and interoperable.

In order to understand the real impact of this type of data, several reports and studies have been carried out. The following are several of them, which address the challenge of measuring the impact of geographic information.

Geospatial data in the Ministry of Transport and Sustainable Mobility. Impact of information co-produced by IGN and CNIG (2024)  

• You can read the full report here.  

This report, produced by ASEDIE and CNIG, aims to draw conclusions about the use and perception of the services co-produced by National Geographic Institute (IGN) and National Centre for Geographic Information (CNIG) in order to understand the benefit they bring to the daily activity of the companies that use them and to society. For this purpose, a survey was carried out among companies using geospatial data, thanks to which a classification of companies reusing geographic data could be drawn up.

Of the 170 companies considered, 70.0% are self-employed and micro-enterprises (less than 10 employees). These companies are mainly located in the Community of Madrid (25.6%), Catalonia (16.3%), Andalusia (14%), Valencia (11.6%) and Castilla y León (11.6%).  53.3% claim to reuse data from Spatial Data Infrastructures (SDI) and 51% open data from INE, among others. The most used products are orthophotos and satellite images (74%), followed by vector maps and cartographic and topographic databases (63%), and LiDAR (58%).

In terms of economic impact, the report estimates an average impact of 35.7% on the sales of the companies surveyed. Specifically, open geographic information from the IGN and CNIG account for an impact of 12.4% of sales.

The report also includes the analysis of collected use cases, as well as in-depth interviews with companies in the sector as examples of best practices and, on the other hand, updates the information from Asedie's annual report on the Data Economy in its 2023 infomediary scope with respect to the economic data of the geographic subsector.

Economic benefits of the SDI central node by CNIG and University of Leuven (2021) 

This document develops and tests a methodology for estimating the economic benefits generated by the Spanish Spatial Data Infrastructure (IDEE), which establishes the publication and accessibility of spatial data through free geographic web services for viewing and downloading produced by cartographic, environmental, cadastral and land observation organisations at national, regional and local level since 2004, in accordance with norms, standards and recommendations that guarantee their interoperability.

The study was to answer the question of what would happen if the NSDI were to disappear. For the study, only the central node of the NSDIE was considered, understanding as such the geographic services and data co-produced among the partners of the National Cartographic System, and focusing on web map services (WMS) and map tiles (WMTS). The nodes of ministries, autonomous communities and local entities were not part of its scope.

Two investigation paths were used to carry out the calculation:

The European Data Portal, data.europa.eu, has just published its Data Maturity Index, an index that assesses the level of maturity of European countries in terms of open data. For its elaboration, an evaluation survey has been carried out and has been completed by 35 countries, including the 27 Member States of the European Union, three countries of the European Free Trade Association (Iceland, Norway and Switzerland) and five candidate countries (Bosnia and Herzegovina, Montenegro, Albania, Serbia and Serbia and Ukraine).  

In this year's edition, Spain obtained a score of 95% out of 100%. This places it in fifth place overall and in fourth place if only European Union (EU27) member countries are taken into account. This figure represents an improvement of three percentage points over last year's score and places Spain 12 points above the EU27 average (83%).  

The top positions in the ranking are occupied by France, Poland, Ukraine and Estonia. 

Above the EU27 average in all dimensions analyzed 

The index is accompanied by a report containing the analysis carried out and an overview of the good practices applied in Europe. In the case of Spain, it is above the EU-27 average in the four indicators analyzed:  

• Policy, focused on the open data policies of the different countries. It analyzes the existence of national governance models for open data management and the measures that have been applied to implement existing strategies. This is the indicator in which Spain obtains a higher score, with 99% compared to 89% in the EU27. The report highlights how the country's national open data strategy helps promote the openness of public information through innovative and structured actions in collaboration with public and private partners. Among other issues, the strategy includes the objective of identifying business models and business success stories to share successful practices. The report also highlights the existence of various digital strategies that complement specific policies on open data, such as the national artificial intelligence strategy, which includes provisions related to the availability of open data for the operation and training of artificial intelligence systems. 

• Impact, which analyzes the activities undertaken to monitor and measure both the reuse of open data and the impact created by such reuse. Traditionally, this has been the least mature dimension across Europe. Nevertheless, Spain scores 96% compared to 77% in the EU27. The best results are achieved in measuring the impact of open data use cases in the environmental, economic and political sectors.  

• Portal, focused on evaluating the functionalities of the national platform that allow users to access open data and help drive interaction within the community. With 96% compared to 85% in the EU27, Spain stands out in the use of analytical tools to understand user behavior, and in the implementation of strategies to ensure the sustainability of the portal and increase its visibility, including presence in social networks. It also highlights the existence of a private area that allows editors to act according to the feedback received. 

• Quality, which examines the mechanisms for ensuring the quality of (meta)data. Here Spain scores 88% compared to 82% in the EU27. Spain's score is driven by compliance with the DCAT-AP standard (providing educational materials for publishers), the existence of a systematic approach to ensure that metadata is up to date, and the wide range of data offered, both historical and current. 

The report also measures how EU27 countries are progressing in the implementation of the implementing regulation on high-value datasets. In this section, Spain ranks ninth, with 68% implementation. In general, Member States are making more progress on geospatial and statistical datasets. Progress is also being made on the underlying technical and legal requirements.  

Overall recommendations 

The report includes a number of recommendations for Spain, among other countries, including encouraging the development of initiatives at the local and regional level, fostering better coordination between teams, and activating the network of open data officers to implement monitoring activities within their organizations. Emphasis is also placed on the need to promote existing open data courses and promote new training materials, paying special attention to developing strategic awareness of the reuse and impact of open data.

Overall, the report shows good progress in open data across Europe. Although there are areas for improvement, the European open data landscape is consolidating, with Spain at the top of the table.  

In 2024, new waves of implementation of the European data strategy will present national teams with new challenges. On the one hand, they will have to redouble their efforts to inform citizens of the new data sources arising from initiatives such as the Data Governance Act and the data spaces. In this sense, coordination will be necessary between the new figures arising from these legislative developments and the traditional world of open data, enhancing the obvious synergies between the two to boost the data economy and collectivize the value generated. 

There is a recurring question that has been around since the beginning of the open data movement, and as efforts and investments in data collection and publication have increased, it has resonated more and more strongly: What is the value of a dataset?

This is an extremely difficult question to answer given not only the inherent complexity of the data itself, which grows exponentially as we begin to combine it, but also the different points of view from which the question of value can be approached.

• If we know that the value will not be immediate, how can we foresee and quantify the potential benefits at some point in the future?
•  Could the value of data become negative in some cases, if we can also cause some kind of 'harm' with it?
• Can the value of data degrade over time?

In this space we have recurrently analysed the value of open data for the administration from different approaches: high-value data and its identification, the perspective of suppliers, the keys to the value of data, how to generate value through data or what is the value of data in real time. However, analysis and research work in this area continues to grow unstoppably. In this regard, we would like to highlight a recently published paper from the University of Cambridge. It is a study in which some of the most common methods for data valuation are described.

Based on their previous analysis of the characteristics of the data and its associated value, a review of the methods that currently exist has been carried out. They concluded that these methods can be divided into several categories, the characteristics of which are detailed below.

Methods based on cost análisis

This approach is based on the traditional statistical principle of "sum of costs". It takes into account the costs of generating, collecting, storing and replacing datasets, as well as the costs to the organisation in case the data results in some kind of loss. These methods have the advantage that they are relatively easy to calculate, but on the other hand, they have the difficulty of having to differentiate between costs directly attributable to the data and other indirect costs related to, for example, the variety of professional work involved or the different software elements used.

An example of the application of this method is the case of Statistics Canada with its analysis of the valuation of the costs associated with investment in data, databases and data science in Canada.

Methods based on revenue análisis

In this case, revenue stream expectations are used, taking as a reference the existing potential market for the exploitation of the data. This may take into account, for example, usage fees, trademarks or patents. The main limitations of these methods are generally that they require the application of somewhat more subjective criteria and the complexity of estimating the value when the data are not exploited directly but indirectly, e.g. through analytics.

These methods are used in the OECD study on the prospects for the value of data. It calculates the reported revenues related to the collection and sale of data through the US enterprise survey.

Methods based on market análisis

Generally, these are the preferred methods to use when all the elements necessary to make the calculations are available. However, today there is still a large amount of data in organisations for internal use only, which makes it difficult to use these methods, as their behaviour is not visible to the market. Furthermore, these methods cannot fully incorporate the social value of the data.

An example of this method is the analysis made in the study carried out by the Economic Commission for Latin America and the Caribbean (ECLAC) on the data marts launched by the European Union and the Government of Colombia, respectively.

Experiments and surveys

This approach to the value of data consists of assessing market sentiment in relation to the data by directly asking about the willingness to pay for certain data or to do without it. It is generally used when the public market value is not known or in cases where social value is important, for example in the environmental area. A limitation of these methods is that, when respondents are not specialists, it can be quite difficult for them to assess the possible uses of the data and thus its full value.

The study carried out by the UK Office for National Statistics is a clear example of such valuation methods.

Impact-based methods

In this case the assessment is carried out through experiments or case studies that analyse the causal effect on certain outcomes attributable to the data. This option is particularly useful for evidence-led policy makers, as it allows a cause-effect relationship to be established, making it easier to understand the benefits and to develop a narrative in favour of the use of the data. However, if the experiments are not well designed or are not well adjusted to the specific context we want to analyse, we run the risk of obtaining an excessively subjective assessment.

The decision-based evaluation framework proposed by the Internet of Water Coalition is a good example of how to apply impact-focused methods to a particular case.

Actor-chain methods

The aim of these methods is to use a more comprehensive view to assess the data from different points of view. This means that evaluations can also be more complex by involving different definitions of what constitutes the value of the data. However, it also makes it the most appropriate method when one wants to assess a data ecosystem as a whole. Moreover, it is a growing method for organisations considering socially responsible investment.

An example of how these methods can be applied in practice is the case study carried out with Highways England.

Methods based on real options analysis

The main advantage of these methods is that they can be applied even when not all possible use cases for the data are yet defined. Their aim is to get an estimate of the value of the data in certain possible future scenarios - usually through computer simulation - so that if such a scenario is reached, exploitation of the data could be justified. Thus, certain data-related decisions and investments could be postponed until the ideal scenario that maximises the value of the data is reached, thereby minimising the associated costs and risks until that point.

The UK case study on the transport sector provides an example of how these methods could be applied using financial models.

And what is the method I should use in my particular case?

Unfortunately, there is no golden rule for selecting a particular method. However, there are a number of questions that the authors of the study suggest we ask ourselves in order to find the most appropriate method for each case:

• What exactly we are assessing: Data goes through several stages in its life cycle - from raw data to processed data, analysis or generated knowledge. Depending on which phase we want to focus our analysis on, some methods may be more appropriate than others.
• From which point of view the valuation is carried out: value can have different definitions depending on the point of view of who is carrying out or commissioning a valuation. In some cases, for example, cost containment due to budgetary constraints may be the priority, while in others one might choose to try to maximise social value.
• When the assessment process takes place: basically it should be considered whether the assessment will be carried out in a predictive way before all the elements assessed are available or whether it will be carried out a posteriori, once all the variables are already known.
• What is the purpose of the assessment: several of the available methods omit or minimise certain aspects of the data by focusing on other features of the assessment process. Therefore, it will be necessary to be clear about the priorities of our evaluation when selecting the most appropriate method: are we interested in social impact, improving productivity or maximising the cost-effectiveness of data?

We should therefore first analyse our needs and our own definition of value, asking ourselves what exactly we want to evaluate and how best to carry out that evaluation, and then develop our own valuation framework using the most appropriate methods from the wide variety available.

Content prepared by Carlos Iglesias, Open data Researcher and consultant, World Wide Web Foundation.

The contents and views expressed in this publication are the sole responsibility of the author.

What are the needs of European public institutions that reuse open data? This is the question posed by the European Commission, through the European open data initiative data.europa.eu, and which is the starting point of the report "Measuring data demand within the public sector", recently published by the initiative.

The report is part of a series of actions that data.europa.eu is undertaking to encourage the re-use of data by the public sector. It is a year-long campaign that will include a series of articles and a webinar. The campaign will culminate in a second report on the findings.

This first report lays the groundwork for the issues to be addressed, setting out 3 objectives:

• Clarify the importance of public institutions as data re-users.
• Identify methods and good practices for assessing the demand from public institutions.
• Stimulate debate on the most appropriate way to foster the re-use of open data by public institutions.

Ultimately, the aim is to foster a data-driven public sector that recognises data as an integral asset for policy formulation, service delivery, management and public innovation.

The benefits of open data re-use by public institutions

Traditionally, in the open data ecosystem, an approach has been followed where the roles were divided: the public sector was the provider of data and the private sector was the re-user. However, this is changing and more and more institutions are realising the benefits of harnessing the potential of open data.

The report highlights how the OECD has identified three areas where data re-use can improve the effectiveness of public institutions:

1. Anticipating governance. Open data helps to predict trends and patterns in order to mitigate emerging risks and respond to developing crises. One example is the interactive dashboard developed by Eurostat with statistical, monthly and quarterly indicators. This dashboard is used by different countries to monitor the economic and social recovery linked to the COVID-19 pandemic.

2. Policy and service design and delivery. Open data helps to understand issues, engage citizens and drive evidence-based policy making. The report gives the example of the Baltic Sea regions, where cross-border use of open government data is being applied to improve social services.

3. Performance management. Open data can also have an impact on increased public sector productivity, more efficient use of resources and better policy evaluation. In this respect, the European Commission's agri-food data portal, which integrates data from multiple European institutions, facilitates the calculation of key indicators for the evaluation of agricultural policies in all countries.

Open data therefore helps to make public services and policies more efficient, but also more sustainable, inclusive and trustworthy, benefiting citizens and businesses. However, despite these advantages, we find that much data in public institutions still does not flow freely, but is siloed. There is a lack of incentive to share, but also a fear of loss of control, among other factors.

The report suggests that the solution lies in taking into account and measuring the demand for data from public institutions. Knowing the benefits of opening up certain data encourages other institutions to open up and reuse it. In this regard, the report highlights, among others, Spain's efforts in engaging with user communities and monitoring the re-use of public data through the Aporta Initiative.

Existing approaches to assessing public institutions' demand for data

After this first part focusing on the benefits, the report goes on to analyse the approaches and indicators that currently exist in the European landscape for assessing the needs of public institutions as data re-users, in order to determine which methods are the most appropriate.

The literature review and the analysis of international measurement frameworks show that demand measurement is not common. The approaches and indicators developed by EU institutions and Member States are few and far between, and focus on fostering demand for data that is already available, without determining the extent of actual re-use of publicly available data and the impact achieved with it. Moreover, they usually focus on re-users in general and on groups such as businesses, journalists, civil society, etc., while neglecting the public sector.

Among the most advanced countries in this area, the report mentions Spain, where a proactive strategy is adopted through challenges and hackathons, co-creation events, information sessions and regular training.

When it comes to measuring impact, a combination of quantitative and qualitative methods is recommended, for example:

• Analyse IP addresses and web statistics.
• Implement web crawling techniques (e.g. search APIs) to identify mentions of open data reuse.
• Quantitative analysis of Tweets mentioning open data.
• Conduct online questionnaires and interviews with re-users.
• Use of contact forms so that users can share use cases or rating systems so that they can rate datasets.

In Spain, quite a few initiatives have already implemented these mechanisms, along with additional ones.

The report also reviews some international indices and reports in search of indicators that measure demand. Many, such as the Global Open Data Index, the Open Data Barometer or the Open Data Inventory (ODIN) focus on data provision and do not include any indicators to calculate the demand for open data by or within public institutions. However, here too the situation is changing and we find other reports such as the European Open Data Maturity Assessment or the Open Data Readiness Assessment (ODRA) that do include this type of metric. Also the forthcoming Global Data Barometer, produced by the Latin American open data initiative ILDA and the Data for Development Network, will include demand indicators as part of the study.

In this sense, the report includes in the annex two tables, one with the overview of the frameworks examined and another one that groups the indicators included in these frameworks that can be used to analyse the demand for open data from public institutions.

Next steps

The paper concludes with a list of key questions emerging from the research, which will be used to trigger a debate among stakeholders on appropriate methods and indicators to measure the demand for data from public institutions, in the context of the data.europa.eu portal. Examples of such questions are: what are the appropriate activities to attract re-users in the public sector or how can automated metrics be leveraged to measure the demand for data from public institutions.

A webinar will take place on Tuesday 29 March to discuss this report. Speakers from different agencies will explain how they engage with data re-users in public institutions, measure their demand and incorporate data into their open data policies. You can register at this link.

Today, no one can deny that open data holds great economic power. The European Commission itself estimates that the turnover of open data in the EU27 could reach 334.2 billion in 2025, driven by its use in areas linked to disruptive technologies such as artificial intelligence, machine learning or language technologies.

But in addition to its economic impact, open data also has an important value for society: it provides information that makes social reality visible, driving informed decision-making for the common good.

There are thousands of areas where open data is essential, from refugee crises to the inclusion of people with disabilities, but in this article we will focus on the scourge of gender violence.

Where can I obtain data on the subject?

Globally, agencies such as the UN, the World Health Organization and the World Bank offer resources and statistics related to violence against women.

In our country, local, autonomous and state agencies publish related datasets. To facilitate unified access to them, the Government Delegation against Gender Violence has a statistical portal that includes in a single space data from various sources such as the Ministry of Finance and Public Administration, the General Council of the Judiciary or the Public Employment Service of the Ministry of Employment and Social Security. The user can cross-reference variables and create tables and graphs to facilitate the visualization of the information, as well as export the data sets in CSV or Excel format. 

Projects to raise awareness and visibility

But data alone can be complicated to understand. Data need a context that gives them meaning and transforms them into information and knowledge. This is where different projects arise that seek to bring data to the public in a simple way.

There are many associations and organizations that take advantage of published data to create visualizations and stories with data that help to raise awareness about gender violence. As an example, the Barcelona Open Data Initiative is developing the "DatosXViolenciaXMujeres" project. It is a visual and interactive tour on the impact of gender violence in Spain and by Autonomous Communities during the period 2008-2020, although it is updated periodically. Using data storytelling techniques, it shows the evolution of gender violence within the couple, the judicial response (orders issued and final convictions), the public resources allocated, the impact of COVID-19 in this area and crimes of sexual violence. Each graph includes links to the original source and to places where the data can be downloaded so that they can be reused in other projects.

Another example is "Datos contra el ruido” (Data against noise), developed within the framework of GenderDataLab, a collaborative platform for the digital common good that has the support of various associations, such as Pyladies or Canodron, and the Barcelona City Council, among others. This association promotes the inclusion of the gender perspective in the collection of open data through various projects such as the aforementioned "Dotos contra el ruido", which makes visible and understandable the information published by the judicial system and the police on gender violence. Through data and visualizations, it provides information on the types of crimes or their geographical distribution throughout our country, among other issues. As with "DatosXViolenciaXMujeres", a link to the original source of the data and download spaces are included.

Tools and solutions to support victims

But in addition to providing visibility, open data can also give us information on the resources dedicated to helping victims, as we saw in some of the previous projects. Making this information available to victims in a quick and easy way is essential. Maps showing the location of help centers are of great help, such as this one from the SOL.NET project, with information on organizations that offer support and care services for victims of gender-based violence in Spain. Or this one with the centers and social services of the Valencian Community aimed at disadvantaged groups, including victims of gender violence, prepared by the public institution itself.

This information is also incorporated in applications aimed at victims, such as Anticípate. This app not only provides information and resources to women in vulnerable situations, but also has an emergency call button and allows access to legal, psychological or even self-defense advice, facilitating access to a social criminologist.

In short, we are facing a particularly sensitive issue, which we must continue to raise awareness and fight to put an end to. A task to which open data can make a significant contribution.

If you know of any other example that shows the power of open data in this field, we encourage you to share it in the comments section or send us an email to dinamizacion@datos.gob.es.

Content prepared by the datos.gob.es team.

Measuring the impact of open data is not always easy. As we saw a few weeks ago, there are several theoretical models that are not easy to implement, so we have to look for different approaches. In the Aporta Initiative we use a mixed approach, as explained here: a quantitative analysis through indicators on data publication and its characteristics, and a qualitative one through the collection of cases of data use.

This approach is also used by various local, regional and state initiatives in our country. In today's article, we will focus on concrete examples of mechanisms implemented by Spanish open data initiatives to monitor and measure the impact of the use of their data.

Quantitative analysis

One of the first steps in monitoring impact is to know quantitatively if users are accessing the published data. To do this we can use different tools.

Dashboards

Thanks to the incorporation of web analytics tools in open data platforms, such as Google Analytics or Motomo (which until 2018 was called PIWIK), a series of indicators can be set around data consumption variables, such as how many users visit the web, what is their origin, which data sets are most in demand or in what format they are downloaded. All of this information is of great value when it comes to making decisions that imply improvements to continue promoting the reuse of public information.

With this data, dashboards can be created so that users can also know this information. This is the case of the Castellón Provincial Council, the Madrid City Council, the Catalan Government, Renfe, the Basque Government or the datos.gob.es itself.

Conducting surveys and periodic studies

In addition, it is advisable to carry out frequent public consultations and studies that allow us to know directly the impact of our data. The ONTSI periodically carries out a characterization study of the sector, and an analysis of the Public Administrations as reusers of their own data and that of third parties. Another example is the report on the Infomediary Sector of ASEDIE, now in its 8th edition. This report measures the products and services based on open data that have been generated. Both reports use a stable methodology that allows comparisons between different years. 

Qualitative analysis

It consists of the identification of use cases through different mechanisms, such as

Application and enterprise tracking

Thanks to the mapping of open data use cases, we can know what the impact of a certain data set is. In many open data platforms, whether local, regional or state, we can find a section of applications or companies with examples and reuse success stories that in turn serve as reference and inspiration for the creation of new value services. In the case of datos.gob.es, we have a form for companies or applications that wish to register their information, but we also carry out a proactive search, through contact with the main actors in the ecosystem and media alerts.

Other examples of portals that have applications sections are Andalusia, Castilla y Leon, Navarra, Barcelona, Santander, Malaga, Zaragoza, Valencia, Vitoria or Bilbao, although there are many more.

Implementation of data communities

In order to be aware of new developments in the field of reuse and to exchange knowledge and experiences to align the data publication strategy with the needs of reusers, some initiatives have opted for the implementation of communities. This is the case of the Basque Country, which has created a space to centralize everything that happens around the reuse of its data. This community has been especially useful to collect and measure the work that has been developed on COVID-19 using open data. Under its umbrella there are also activities and competitions that encourage reuse.

For its part, the Castellón Provincial Council has created a Provincial Council of Reusers, a mixed public-private body made up of technicians from the provincial institution itself, and people with recognized professional backgrounds in different economic sectors. These professionals meet once a quarter to hold a conversation to monitor use cases and which favours constant feedback and the enrichment of the Provincial Council's open data strategy. 

The National Library of Spain is working along the same lines and has launched a collaborative work platform so that those citizens who wish to do so can participate in specific projects to enrich the Library's data, making it more accessible and easier to reuse.

In short, all these activities allow monitoring the activity of an open data initiative and its impact on society. They help us to know what challenges we are solving in fields as important for humanity as the environment, health or education. In this way, we will be able to know its evolution over time and easily detect trends and possible areas of improvement, which will lead us to distribute the efforts and resources available in a more effective way.  

Content elaborated by datos.gob.es team.

The difficulties in adequately measuring the impact of open data initiatives are widely known, as it is a global debate that has been with us practically since the beginning of the first open data initiatives, more than ten years ago. The problem is that it is relatively easy to find isolated examples of how various benefits and improvements have been achieved through open data, but in general - and unless we are talking about very specific areas such as economic impact - it is not common to find complete and structured measurements of the impact that has been achieved through open data initiatives in order to demonstrate their full value.

A review of the various initiatives and methods that exist to measure the impact of open data also shows that there is no clear consensus among researchers on how best to capture the results and impact of these initiatives. 

In view of this scenario, we share here the approach to impact measurement that we have been following from the Spanish Government's open data and public information reuse initiative and that is inspired by the recommendations provided by several guides that we consider to be reference in this aspect: the UNE 178301:2015 standard, the framework of common methods for the evaluation of open data, the guide for the empirical analysis of open government data initiatives and the taxonomy of the impact of open data.

Within our personal approach to the problem we must first clarify what we understand “impact” as – “any positive effect or benefit obtained directly or indirectly for individuals, communities or society as a whole, which occurs over a certain period of time and which results from the development of different activities in a given field characterized by the use of open data as a means to an end”. 

Thus, the method we use to measure such impact is based on two main components: a quantitative analysis through indicators on data publication and its characteristics, and a qualitative one through the collection of data use cases.

Quantitative analysis

The objective of quantitative analysis is to offer a series of indicators in a quick and simple way that provide us with an overview of the data publication activity. In this way, we can track its evolution over time and easily detect trends and possible areas for improvement. This will help us to distribute available efforts and resources more effectively. 

The datos.gob.es initiative has its own public control panel that provides quantitative indicators on the following aspects:

Qualitative analysis

Through the qualitative approach, use cases are identified from different sources that can include personalized interviews, content available online, information provided by different media or academic publications. 

These use cases help us to understand the extent to which open data can be considered to have led to positive changes in three main areas described below:

Government level

Including the possible impact on the transparency and accountability of governments, as well as on improving their efficiency and effectiveness. Some of the issues that can be raised in this area are:

• How does openness of data help to improve government efficiency?
• How is data used to examine government resource use and improve existing public services?
• How is open data used to create new public services?
• How does open data contribute to increased transparency and accountability through public scrutiny?

Examples of measurements that are also useful in measuring the impact of open data actions in this area are:

•     Increased collaboration between different government departments and agencies.
•     Creation of platforms and applications that allow citizens to report on their experience in government services.
•     Greater participation of citizens in the formulation of public policies.
•     Improved public resource planning.
•     Reduction in citizens' perception of corruption.

Social area

Including the environmental aspects and the improvements obtained in the inclusion of minority groups in society. Some of the issues that can be raised in this area are:

•     What are the social benefits obtained through the use of open data?
•     How is open data used to improve equality and direct public resources to the citizens who need them most at any given time?
•     What are the benefits of open data in the area of the environment, climate change, pollution, or sustainability?

Examples of indicators that are also useful when measuring the impact of open data actions in this area are:

•     Evidence of greater equality in terms of age, gender, race, social class, disability, geographical location or poverty level.
•     Evidence of improvement in social policies.
•     Number and effectiveness of sustainability programs implemented as a result of the open data
•     Increased attention to environmental factors in project planning.
•     Awareness and sensitization of citizens about their own environmental impacts.

Economic area

Through the influence of open data in supporting existing businesses or creating new business models. Some of the issues we address in this area are:

•     What is the impact of open data on economic growth and innovation?
•     How is open data used to reduce costs in companies and help them become more efficient?
•     What new business models are being developed around open data?

Examples of indicators that are also useful when measuring the impact of data openness actions in this area are:

•     Companies and/or jobs created from the new economy of data openness.
•     New products or services using the open data.
•     Contribution of open data to the growth of the economy in terms of better macroeconomic planning.

 Future areas of improvement

While the method applied may not be perfect - as it may have limitations in terms of the ability to adequately capture desired changes -, in the absence of other reference methodologies, it allows us to explore the emerging impact of ongoing open data initiatives and provide some substantial evidence regarding their impact, at least.

Thus, it would also be desirable to explore new methods in the future that allow for evidence of impact obtained in a more systematic way through the measurement of specific goals and objectives with respect to the initial starting point, and that could also be replicated among the different initiatives, also considering the entire data value chain in the measurements carried out. Some interesting options to be able to put these objectives into practice could be the methods of outcome mapping or social return on investment (SROI) -both already proposed previously in the field of open data but with very limited or no acceptance so far, due most probably to the higher added cost that their implementation would imply.

Content prepared by Carlos Iglesias, Open data Researcher and consultan, World Wide Web Foundation.

Contents and points of view expressed in this publication are the exclusive responsibility of its author.