The impact of open data along its value chain: Indicators and future directions

The transformative potential of open data initiatives is now widely recognised as they offer opportunities for fostering innovation, greater transparency and improved efficiency in many processes. However, reliable measurement of the real impact of these initiatives is difficult to obtain.

From this same space we have also raised on more than one occasion the recurring question of what would be the best way to measure the impact of open data, we have reviewed different methods and best practices to try to quantify it, as well as to analyse it through detailed use cases or the specific impact on specific topics and sectors such as employment, geographic data, transport or sustainable development objectives. Now, thanks to the report "Indicators for Open Data Impact Assessment" by the data.europa.eu team, we have a new resource to not only understand but also be able to amplify the impact of open data initiatives by designing the right indicators. This publication will provide a quick analysis of the importance of these indicators and also briefly explain how they can be used to maximise the potential of open data.

Understanding open data and its value chain

Open data refers to the practice of making data available to the public in a way that makes it freely accessible and usable. Beyond ensuring simple availability, the real value of open data lies in its use in various domains, fostering economic growth, improving public sector transparency and driving social innovation. However, quantifying the real impact of data openness poses significant challenges due to the multiple ways in which data is used and the wide-ranging implications it can have for society. 

To understand the impact of open data, we must first understand its value chain, which will provide us with a structured and appropriate framework for transforming raw data into actionable insights. This chain includes four main stages that form a continuous process from the initial production to the final use of the data: 

Figure 1: Open Data Value Chain

1. Collection: this consists of identifying existing data and establishing the necessary procedures for their cataloguing.
2. Publication: making data available in an accessible form and easy to locate.
3. Uptake:will come sooner when data is easy to use and accompanied by the right incentives to use it.
4. Application:either through direct consumption of the data or through some transformation that adds new value to the initial data.

Each of these steps will play a critical role in contributing to the overall effectiveness and value derived from open data. The indicators developed to assess the impact of open data will also be closely linked to each of these stages, providing a holistic view of how data is transformed from simple information into a powerful tool for development.

Indicators for impact assessment

The report introduces a set of robust indicators that are specifically designed to monitor open data initiatives through their outputs, outcomes and impact as a result of their value chain. These indicators should not be seen as simple metrics, but as tools to help us understand the effectiveness of open data initiatives and make strategic improvements. Let us look at these indicators in a little more detail:

• Output indicators: are those that focus on measuring the immediate results that come from making open data available. Examples would be the number of datasets released, the frequency of dataset updates , the number of visitors to the data catalogue, the accessibility of the data across various platforms, or even the efforts made to promote the data and increase its visibility. Output indicators help us to assess the efficiency of data publication and dissemination processes and are generally easy to measure, although they will only give a fairly superficial measure of impact.

• Indicators of effect: Outcome indicators measure the short- and medium-term consequences of open data. These indicators are crucial to understand how open data influences decision-makingprocesses, leads to the development of new applications or improves government transparency. Thus, improved public transport planning based on usage data, increased citizen participation in the development of public policies to tackle climate change brought about by the increased availability of data and information, or improved productivity of public services through the use of data can be considered as significant examples of outcome indicators.

• Impact indicators: This is the deepest level of measurement, as impact indicators assess the broader, long-term effects of open data. These indicators may include economic benefits such as job creation or GDP growth, social impacts such as improved trust in public entities, or even environmental impacts such as the effective reduction of greenhouse gases. At this level, indicators are much more complex and specific, and should therefore be defined for each specific domain to be analysed and also according to the objectives set within that domain.

Figure 2: Indicators for impact assessmen

Implementing these indicators in practice will require the implementation of a robust methodological framework that can capture and analyse data from a variety of sources. It is recommended to combine automated and survey-based data collection methods to collect more comprehensive data. This type of dual approach allows for capturing quantitative data through automated systems while also incorporating qualitative insights through user feedback and more in-depth use case analysis.

Looking to the future

The future of open data impact assessment looks towards refining the indicators used for measurement and consolidating them through the use of interactive monitoring tools. Such tools would enable the possibility of a more continuous assessment that would provide real-time information on how open data is being used and its effects in different sectors. In addition, the development of standardised metrics for these indicators would facilitate comparative analysis across regions and over time, further improving the overall understanding of the impact of open data.

Another important factor to take into account are possible privacy and ethical considerations applicable to the selected indicators. As in any other data-centric initiative, privacy and data protection considerations will be paramount and mandatory for the indicators developed. However, once we get into its use by users, it could lead to more delicate situations. Generally, this should not be a particularly problematic issue when monitoring data. However, once we get into its use by users, it could lead to more delicate situations. Ensuring anonymity in indicators and secure practices in their management is also crucial to maintaining trust and integrity in open data processes.

In conclusion, the development and implementation of specific detailed indicators following the recommendations of the report"Indicators for an Open Data Impact Assessment" would be a significant step forward in terms of how we measure and understand the impact of open data.Continuous refinement and adaptation of these indicators will also be crucial as they evolve in tandem with the open data strategies they accompany and their growing sphere of influence. In the medium term the European Commission will further develop its analysis in this area of work through the data.europa.eu project with the ultimate goal of being able to formulate a common methodology for the assessment of the impact of the re-use of public data and to develop an interactive monitoring tool for its implementation.

Content prepared by Carlos Iglesias, Open data Researcher and consultant, World Wide Web Foundation. The contents and views expressed in this publication are the sole responsibility of the author.