Data are fundamental to solving the social and economic challenges we face today. It allows us to understand the causes behind a given situation and the factors that influence it, to estimate how it is evolving and to make informed decisions for change.

The problem is that solving such challenges often requires a combination of data from different sources:

•     Data provided by the public sector
•     Data from multiple private sector companies
•     Citizen-generated data

But how can such collaboration be achieved? This is the question posed in the report "How to facilitate data collaboration to solve societal problems", written by Jose Luis Marín in the framework of the Aporta Initiative.

The report begins with a brief introduction outlining the current state of data openness in the public and private sectors, as well as in the citizenry. Then, it discusses the main barriers to data sharing by companies and citizens, the individual and collective benefits that can motivate these practices in the public interest, and the policies that can be put in place by public administrations to compensate and encourage collaboration.

Once the context is clear, it is time to look at some of the existing mechanisms for collaboration in data collection, sharing or processing to address a societal challenge. Although there are no systematised analyses of ideal forms of governance, four formulas have been identified for this report:

In order to illustrate and better understand each formula, the report includes multiple examples of international success stories, such as the Accelerating Medicines Partnership (AMP), which focuses on transforming the current model of developing new diagnostics and treatments, or the Open Apparel Registry (OAR), which aims to contribute to improving human rights and environmental conditions around factories.

The report concludes with a series of resources to help organisations collaborate successfully and reduce barriers, including collaboration networks, collaboration frameworks, courses and trainings.

Below, you can download the report, as well as access the complementary materials (executive summary, video-interview and summary presentation). The report is translated into English, but additional materials are available in Spanish version only.

Local statistical data helps us better understand our environment and identify variations between regions. This is essential to be able to formulate local policies tailored to the specific needs of the local population, something that has even been highlighted by the UN in one of its reports. In this sense, in the datos.gob.es catalog you can find statistical information on different localities and regions, such as the population and housing census, administrative records or even economic indicators.

One of the latest additions to our catalog is the Canary Islands Statistics Institute (Istac), as part of the datos.canarias.es initiative. The institute is the new Canary Islands Open Data Portal that is positioned as the only point of access to open data on the island in collaboration with the rest of the regional public administrations.

At the end of January, datos.canarias.es federated with datos.gob.es, incorporating 7,460 new datasets from the Istac and other organizations on the Island.

What kinds of data sets are available?

The federated data are categorized according to the recommendations of the Application Guide of the Technical Standard for Interoperability for the Reuse of Information Resources and address a wide range of topics related to the territory, the environment, demographics, the economy, and living conditions or the public sector. Shared data is divided into the following categories:

Istac's commitment to open data

The Decree approving the Statistical Plan of the Canary Islands 2018-2022 (PEC-22), establishes that during its execution the reuse of statistical data will be promoted in accordance with the Law on reuse of public sector information. At the same time, it indicates that the Statistical Data and Metadata Infrastructure (eDatos) will be the support for the open and interoperable dissemination of the data published by the statistical activities of the PEC-22, becoming the only channel for the decentralized dissemination of statistics in the corporate websites of the Government of the Canary Islands.

To comply with the aforementioned guidelines, the Istac has made the open data portal of public statistics in the Canary Islands available to the public, which, under the principles of public statistics and data reuse, distributes the data generated in a manner free, in open formats and with licenses that allow its reuse for commercial and non-commercial purposes.

The portal integrates data and metadata based on standardized semantic assets, geographic information and services to promote their use; and it has programmable application interfaces (API) that facilitate the access and download of the information by third parties. In addition to these APIs, it also provides a series of query tools that allow both downloading the data (for example an Extension for QGIS or an R Package), and taking it to another web or application, such as Widgets, Tableau Public or Google Public Data Explorer.

The website also has a statistical indicator viewer. The user can select the information they want to view from a large number of categories, for example, births and deaths, or the workforce. You can also choose the geographic space (the community as a whole or any specific island or municipality), the type of data (annual, interperiodic variation, etc.) and the temporal range. With this information, the tool will generate the graph with the Istac data.

All these tools show the interest of the Istac and the Government of the Canary Islands not only to facilitate access to their data, but also to promote its reuse by developers who want to create value-added products. With its integration in datos.canarias.es, the visibility of the local statistical data of the Canary Islands is promoted, at the same time that access to the data of interest of the entire Autonomous Community is homogenized.

Infomediary activity is not classified as such in the CNAE. Therefore, the census of companies must be updated through indirect and ad hoc actions. Periodically, the National Observatory of Telecommunications and the Information Society (ONTSI) analyzes the state of this sector and collects the results in a report, which this year reaches its fifth edition.

Under the title "From Infomediary Sector to Economy Data. Characterization of the Infomediate Sector", this year's report starts from a new approach, more aligned with the European Union's vision. The document highlights that the infomediate sector is evolving towards the data economy, and takes as a reference the words of the European Commission, which defines "data companies" as "organizations whose main activity is to produce products, services and technologies related to data".

A sustainable sector in continuous growth

This year's report shows us that we are facing a sector that does not stop growing in a sustainable way. 708 companies have been identified, 32% more than in 2016. It is worth noting that the businesses that are created around this activity last over time, since 63.7% of the companies analyzed are more than 10 years old. The sector is mainly concentrated in Madrid, where more than 50% of the companies are located. It is followed by Catalonia, with 18.4%.

In recent years there has been an evolution towards businesses centred on digital technologies: the most recent companies are mainly in the field of programming and IT consultancy and information services.

73.5% of the companies carry out more than one infomediaries activity, although the one that attracts more business in data analysis and/or visualization.

Almost 50% of these companies are micro-enterprises (with less than 10 employees) and only 6% have more than 250 employees. The turnover of the sector, however, is not negligible, estimated to reach 1,987 million euros in 2018, 15.4% more than in 2015. Of this volume, 718 million come from the reuse of information.

With regard to employment, in recent years there has been significant growth. The sector employs between 14,000 and 16,000 people, 14.3% more than in 2016. If we focus only on employees linked to data reuse, the growth is even more spectacular, 61.5%, reaching 7,700-8,400 workers. 

Traditionally, the distribution of employment by gender in the infomediate sector has been equal (49.9% women and 50.1% men). However, the progressive introduction of digital technologies is breaking this parity in favor of men. Newer (and more technological) companies have many more men than women on their staff.

1 in 5 companies uses data from public sources

More than 80% of the companies analyzed use data from public sources to develop these services, which are combined with private data in 69.9% of the cases. The most demanded public information belongs to the fields of trade, environment, economy and demography. They mainly access this data in non-proprietary structured formats (csv, xml), although it is worth noting that there are still many suppliers that provide their data without structure (pdf, jpg), making it difficult to process automatically.

When asked about the data they most miss, the companies surveyed indicated that they would like to have more data on weather, real-time transport, tourism, demographics (with information disaggregated at the level of census sections), justice at the regional and local level, the public sector, the economy and the treasury.

These are just some of the report's conclusions, but there are many more. You can download the full report and its presentation below.

From its origins, the open data movement has been focused on opening up public sector data, and even the original statement of open data principles itself refers only to data from public administrations (open government data - OGD). However, this perception has evolved and today it is more common to speak of open data in a more general scope that includes all sectors, as demonstrated for example by the declaration of open data principles promoted more recently by the Open Data Charter, the wording of which is perfectly applicable to any sector.

On the other hand, while we now have a more than respectable and valuable amount of data available in open format from government, there are also some limitations with regard to governance and sustainability models for public data which, together with the remaining challenges of digital transformation in the sector, are to some extent limiting the ultimate availability of data. Data lifecycle management, technical and legal issues, adapting the skills of public employees or growing concerns about potential privacy issues are some of the barriers that public administrations have to face when trying to take their open data projects to the next level.

In this context, other data-producing sectors are also emerging as complementary sources to the hundreds of open data catalogues that governments have been publishing in recent years. These new data come from other publishers outside the public sector - such as the private sector, academia or the so-called third sector - all of them driven by different motivations and by the potential benefits associated with social reputation, innovation or attracting talent, among others. Even citizens themselves have now become a broad source of data, albeit sometimes unwittingly.

Thanks to this greater variety of data producers, the possibilities of enriching information by combining the different sources now available are increasing, but, at the same time, there is also a new need to improve coordination and integration between all parties in order to enable more efficient production and reuse of data and thus obtain the expected improvements in the social, economic and scientific spheres. The current challenge is therefore to find new models and collaborative relationships between the different parties that provide an adequate response to the existing challenges in achieving better management and availability of data, ultimately benefiting all parties and facilitating its reuse to a greater extent.

In order to get a more complete picture of the situation and to understand in more detail who these new data-producing sectors are, what their motivations are and what new models of collaboration are emerging between them, this report is divided into two parts:

In the first part of the report we will take a look at the different data producing sectors - leaving the public sector aside for a moment - and the alternative data sources they offer us, showing their motivations, their particularities and several examples.

In the second part of the report we will focus on analysing what models of collaboration can emerge between the public sector and other sectors to create joint actions to improve the governance, availability and ultimate re-use of data, showing again the differences and similarities between each of them and different examples of how they have been put into practice so far.

From the experimental vision of contests and competitions, to the purely entrepreneurial objective linked to the launch of incubators. From the models related to the legal responsibility point of view of data trusts, to the participative models of data collaboratives. From the actions with a more formative perspective of data fellowships, to the proposals aimed at empowering data cooperatives. All of them are possible models, some of them in a preliminary phase of consolidation but others more tested and refined, that can be more easily adopted. Let's discover them in this report and see how they can be put into practice in different contexts.

Spain is the second country with the highest life expectancy in the world, only behind Japan. Spaniards live 83 years on average. This positive data is stained by a negative one: the low birth rate, which is leading us towards a population aging. This situation means that we need a more efficient health system to continue providing quality health services to citizens.

As in other sectors, the improvement of efficiency goes through the necessary digital transformation, in which data in general - and open data in particular - have a leading role. Open data can help us better understand the situation of patients and, together with technologies such as big data or artificial intelligence systems, facilitate early detection of diseases. In short, they can help improve both the management and the provision of services.

But in an area where patient privacy is essential, we have a series of doubts: What types of data can be opened? What does the legislation say about it?

The report "Open data and health: technological context, stakeholders and legal framework", prepared by Julián Valero, tries to shed some light on this situation. For this, the following objectives are set:

1. Knowing the conditions, limitations and restrictions imposed by current legal regulations.
2. Posing how the guarantees offered by the Law should be adapted to a new reality based on technological innovation.

The report begins by showing the current situation of the Spanish health system, gathering the challenges to be faced, but also the opportunities that come hand in hand with new technological trends, such as Internet of Things or the aforementioned Artificial Intelligence.

Once the context has been explained, the report focuses on the different stakeholders involved in the provision of health services, both public and private, and the main laws and regulations that affect each group. The novelties of the General Data Protection Regulation (GDPR) and its impact on the opening of health data are also addressed.

The report ends with a series of conclusions and recommendations to promote public policies in the field of health that drive improvements in the provision of health services.

You can download the full report below.

We invite all companies that have successfully carried out projects with reusable data generated by Spanish public administrations to submit their application for the Aporta Awards. 

The Aporta Awards 2017, an initiative promoted by the State Secretariat for the Information Society and Digital Agenda, the Public Enterprise Body Red.es and the General Secretariat for Digital Government, have been created with the aim of disseminating and recognizing success stories that may serve to help other public sector professionals to innovate and focus on reuse of open data as an engine for digital transformation and innovation.

Companies wishing to participate have until September 12 to present projects carried out in the last two years in which data reuse has contributed to the generation of social value, new businesses and/or improvements to society.

Applications, which will refer to initiatives that have made use of open public data generated by the Public Administration or data provided by private entities, must be submitted using the form available in the electronic head office of Red.es.

Projects will be evaluated during the month of September by representatives of the Aporta Initiative. The originality, usefulness and impact of the initiative will be taken into account in terms of the beneficiaries thereof.

The best two initiatives will receive recognition in the Aporta Conference to be held in late October 2017.

We hope that our dissemination and recognition of success stories in the reuse of public data will help many other professionals in the private sector to innovate and commit to the use of data generated by Public Administrations as a source of competitive advantage.

All information in datos.gob.es and Competition Rules of the 2017 Aporta Awards.

The first phase of the Aporta Challenge 2017: The value of data for the Administration has come to an end. The competition "How to reuse open data to improve efficiency in public administration", promoted by the State Secretariat for the Information Society and Digital Agenda, the Public Body Red.es and the General Secretariat for Digital Administration, received a total of 45 ideas from working groups in the academic and business sectors as well as individuals. Ten of these have been selected and move on to the second stage. The jury, which evaluated the ideas presented taking into account their relevance, quality and usefulness, stressed the high quality of all the proposals received.

The ten proposals selected as finalists in this first phase are the following:

• QME -Aporta: NL guidance in datos.gob.es consultations
  Search engine based on natural language guidance that seeks to facilitate the discovery of the data provided by the public administrations and reuse thereof between them.
• Licitalio: Your tool for comparison of public contracts
  A web tool which, using datamining and clustering techniques, aims to improve access to public sector contracts in order to facilitate selection or and reuse of the best bids according to the needs of the administration concerned.
• Informal analysis of licenses
  Mechanism to characterize licenses, using natural language processing techniques, aiming to detect swiftly what administration resources are duly licensed for reuse and which of them have conditions of use needing to be readjusted.
• Anfitrión (Host)
  Client server system intended to act as an interpreter for the transformation of sets of data in different formats to JSON, easily reusable by businesses and other administrations.
• Light
  Mobile application that, by introducing gamification mechanisms, proposes to encourage people to contribute together with the administrations to enable a more sustainable environment, by rewarding habits and actions such as using the public transport service.
• PLAN Platform
  Project aimed at publication of announcements prior to the launch of national tenders to which all economic agents could subscribe, encouraging competition and transparency.
• Analysis of content for public communication
  Initiative to streamline the response of administrations to the general public by means of analysis of the channels of communication citizens-city council, and identification or the interest prior to the request for information.
• Open News
  System designed to recommend open public datasets related to content generated by digital means, in order to encourage active citizen participation and transparency in public administrations.
• Funding Analytics For Innovation Projects
  Proposed development of an analytical web tool based on open data R&D+i projects aimed at analyzing and visualizing subsidies granted and detecting trends in research that help public administrations to evaluate and create new business lines for companies.
• My turn
  Mobile application that, while showing the number of people waiting their turn at the offices associated with a service offered by the administration, would analyze and manage waiting times for the public and therefore improve the administration’s efficiency.

Now the second phase of the Aporta Challenge is beginning, in which those responsible for the winning ideas have a period of two and half months to develop their prototypes, which they will present to the jurors next October 23.

Prizes of 3,000, 2,000 and 1,000 euros will be presented to the top three prototypes, which will be announced during the Aporta 2017 Meeting on 24 October.

We encourage the open data community to attend this meeting (attendance free) which this year reaches its seventh edition.

The recent entry into force of Law 9/2017, of November 8, on Public Sector Contracts last March, has introduced a novelty of great relevance in the Spanish legal system regarding to the reuse of Public sector information. In effect, the aforementioned legal regulation has consolidated the role of the contractor's profile on the Internet as an essential instrument to facilitate access to information regarding the contractual activity of public authorities. Apart from other provisions aimed at reinforcing the content of publications and the accessibility of the information collected there, article 63 of the aforementioned Law requires that all information included in the contractor profile be published "in open and reusable formats". In addition, this content must necessarily be spread through the Public Sector Contracting Platform in the terms of article 347 - the management of this electronic instrument correspond to the General State Administration-.

This is a new advance in the process of transformation of the contractual management carried out by public authorities to be aligned with Open Government parameters. This approach has already started with Law 19/2013, of December 9, on transparency, access to public information and good governance, whose Article 8 requires the dissemination of "all contracts, indicating the purpose, duration, amount of the bidding and the awarding, the procedure used for its conclusion, the instruments to publicize it - if it takes place-, the number of bidders participating in the procedure and the identity of the successful bidder, as well as the amendments to the contract ", in addition to “the decisions of withdrawal and waiver of contracts”. However, this publication refers to a later phase of the contractual process, in such a way that, ultimately, its objective is the accountability related to their decisions and the circumstances in which it takes place.

As is well known, the reform in the area of ​​eGovernment in 2015, was an important step forward, since the use of electronic means in management can ultimately be a boost for increasing the efficiency of public information availability in accordance with the standards that facilitate its reuse.

In this regard, from October 2nd, 2016, all administrative acts must be recorded electronically, unless, by its nature, another more appropriate form could be required, which will undoubtedly be exceptional. This requirement is established not only for functions, but also for any document issued by the Public Administrations, which must also incorporate the minimum required metadata. In this sense, the necessary respect for the National Interoperability Framework already allowed us to affirm that, based on a systematic interpretation of all the aforementioned standards, the document management of Public Administrations should not only be carried out electronically, but also, in accordance with standards that allow the reuse of information.

What, then, is the scope of the new regulation on public procurement in this matter? On the one hand, it has been firmly established -and without, therefore, any interpretation required- that the dissemination of the information to be published in the contractor's profile must necessarily be in open and reusable formats. Consequently, this represents an advance with respect to 2 previous laws:

• The legislation on transparency, which only considers it as a mere preference.
• The legislation on the reuse of public sector information, which simply requires public authorities to ensure access to documents so that they can be reused.

On the other hand, the object of publication in the new contract legislation does not refer only to finalized procedures, as is the case of transparency regulation, but also to those that are being processed at that time or will be initiated in the future. It is, therefore, a measure that aims:

• Promote effective competition in the selection process of companies that aspire to be selected, so the obligation of dissemination in accordance with these requirements becomes an inexcusable obligation.
• Facilitate the audit and, in general, the control of the contracts carried out by the public sector, which undoubtedly is decisively promoted in the contractual field with initiatives and projects based on open and reusable data.

In short, after the presentation of the main novelties of the new legislation on public procurement, a question remains to be answered, what would undoubtedly be the main issue to be raised from the point of view of its practical application: What would happen if a call for bids was published in the contractor's profile without complying with the legal requirements regarding open and reusable data?

Certainly, it could not be said that the legal requirement specifically contemplated in the legislation on public procurement has been ignored (Article 39.1.c of the Law on Public Sector Contracts), since there has been publicized. However, we could find a violation of the general regulation on common administrative procedure (Article 48.2 of Law 39/2015), because the announcement of bidding would lack the essential formal requirements to achieve its purpose; even if there are undeniable publicizing through the legally required instrument, that is, the profile of the contracting party.

In any case, we will have to wait to know the courts’ interpretation of contractual remedies on the scope of the new legal regulation.

Content prepared by Julián Valero, professor at the University of Murcia and Coordinator of the Research Group "Innovation, Law and Technology" (iDerTec).

Contents and points of view expressed in this publication are the exclusive responsibility of its author.

The public sector is not only a large supplier of open data but also one of the largest users and beneficiaries of the opening of government data. The report ‘The value of open data for the Government’ that we published on Datos.gob.es outlines a large number of examples of how the government could see benefits in various areas and rely on the data for feedback and to reflect, cooperate, understand, optimise, and learn.

The creation of an open database in Japan to improve support for victims of the devastating earthquake in 2011, the early warning system to detect trends in diseases or epidemics launched by the Korean government, the portal for the quality of teaching in public schools in Brazil that drew on data from the Ministry of Education, the buildings in France in order to optimise supply, and the large global database of companies presented by Open Corporates are excellent examples of how to make public administration more agile and efficient by making good use of data.

The report highlights how the use of open data improves the efficiency and effectiveness of the government itself through better planning of available resources, collaboration between different departmental areas, scrutiny of the correct use of resources, improvements in the interoperability of data and processes, as well as the adoption of standards that facilitate the sharing and storage of data.

It also emphasises that the use of such open data drives public innovation by contributing to the identification of patterns that help decision-making, to more active development and implementation of public policies, and the improvement of data quality thanks to public feedback.

The document compiles concrete examples, such as the improvement in the management of public health services in Malawi due to the study of the in the various districts, the reduction in levels of infant mortality in Mexico due to the greater availability of information, and the significant improvement in quality seen in the database of addresses in Denmark due to the openness of the information.

Another advantage and benefit of open data for the government itself, as detailed in this report, is the promotion of transparency and innovation.  ‘Monitoring of government's decisions and actions can be better performed, facilitating understanding of public policies and helping citizens to rebuild trust in their governments,’ notes the report.

Examples of the potential of open data in government include the Philippines’ open and participatory budgeting, the scrutiny of contracts in Slovakia due to their publication online, the project to publish all the information regarding public spending by local governments in Austria, and the transparency portal in Brazil.

The attached Report can be downloaded in PDF, Word, and ODT format.

Annually the Multisectorial Information Association (ASEDIE) publishes a report analyzing in detail the national infomediary market. This fifth edition evaluates 636 Spanish companies whose activity is based on the re-use of public and/or private information to develop value-added products for third parties or citizens in general in the following sub-sectors: culture, directory services, economic and financial, publishing, market research, geographic information, meteorological, or tourism. The list has grown this year adding two new sectors: Infomediation technology and technical consulting.

According to the data provided in this last study, as in previous years most of the infomediary companies are located in Madrid (38%) and Catalonia (19%) In addition, the geographic information (24%), market research (23%), and the economic and financial (22%) industries are the most representative areas.

Regarding the total revenues for the year 2015, they increased to €1,705 billion; which means an average turnover of 2.68 million euros per company. An increase in 2015 close to 2% in comparison with the previous year.

In relation to the jobs created, according to the ASEDIE report, the infomediary sector employed 19,362 workers in 2015, a positive variation of 4.3% compared to the previous edition; with the largest growth was recorded in the economic and financial sector, with 219 employees more than in 2014, a similar number to the geographic information industry, with new 215 more employees.

Finally, at the closing of 2016 the total subscribed capital was 366 million euros, 20.7% lower than the previous year, after three consecutive years of growth.

Through this report, ASEDIE provides a detailed view of a growing sector at national level, which, thanks to its horizontal nature, is especially relevant in improving business management and the impact of the opportunities generated, both economically as politically and socially.