Multiple ways to assess and quantify open data

Free and open access to data through the Web provides us the transformative potential to enable citizens to be fully involved in the democratic process, making governments and other development agents more efficient and foster the creation of new services that improve our lives. However, in order to make these potential benefits become a reality, first we should be able to implement an effective open data model.

It is on this point where it is essential to measure the different open data models and their progress, in order to understand which ones work best in different contexts. Our aim is not only to be able to measure the performance of the different initiatives, but improve our understanding of the theory and practice in open data, and its value chain.

Nevertheless, measuring the openness of data is not always an easy task. The evaluation of open data initiatives has been identified as one of the main challenges on the global agenda of open data. There are many ways to measure open data models from rankings to use cases through quantitative or qualitative metrics of the impact or performance of a given initiative. But the first question that arises is: what do we exactly want or have to measure?

Fortunately, the common framework for assessing open data, led by the international organization World Wide Web Foundation and the American institution The GovLab, defines a number of dimensions and components that should be present in such assessment, including:

• Context: including all the circumstances of the environment in which the data is published, including legal, organizational, political, technical, social and economic aspects.
• Data: analyzing the variety, quantity and quality of available data, including all general principles of open data, such as availability, update, usability, licensing, etc.
• Use: deepening on aspects such as who is using the data today? Why are they being used? Who may be being excluded? What goals have we set? What sectors are still the most successful?
• Impact: trying to measure the profit obtained by opening up data, including social, political or economic benefits in general or other more specific impacts in areas such as environment, innovation and inclusion.

On the basis of this framework multiple studies and analyzes have been built over time to assess the different dimensions of open data initiatives existing nowadays, so it will not be difficult to find one that fits our interests or needs.

Some of these studies, such as the Open Data Barometer conducted by the Web Foundation, cover with their analysis all dimensions indicated in the reference framework. Others focus on a specific subset, such as the OURdata Index of the OECD which only analyzes the data context and availability. In other cases, the analysis focuses on a specific dimension, such as the availability of data in the Open Data Index, the use of data in the Open Data Impact Map, or the impact of these initiatives in the Open Data's Impact. There are also more specialized studies in specific sectors, such as the Open Data Inventory which focuses on statistical data, or those within a single component that focus on certain aspects of the analysis, such as the Open Data Usability Index or the Open Data Accessibility Framework.

Finally there are also more comprehensive studies, especially in the field of Open Government, which also include in one way or another some aspects of open data in their analysis. Such is the case for example of the Open Government Index or the popular eGovernment survey of the United Nations.

The working group on measurement of open data initiatives of the International Open Data Charter continues to work on the definition of new methods to assess the implementation of its principles among countries adhered to the Charter. The mid-term aim will be to update the common framework for assessment of open data, resulting in a new, more complete version and fully aligned with these principles. This new framework will serve as reference for the assessment of all open data initiatives which uses the principles of the Charter in implementing their policies.