Public-Private Partnership for Big Data

13 October 2014
by eub2 -- last modified 13 October 2014

The European Commission and Europe's data industry have committed to invest EUR 2.5 billion in a public-private partnership (PPP) that aims to strengthen the data sector and put Europe at the forefront of the global data race. Mastering big data could mean: up to 30% of the global data market for European suppliers; 100,000 new data-related jobs in Europe by 2020; 10% lower energy consumption, better health-care outcomes and more productive industrial machinery.

What is big data?

"Big data" is large amounts of data produced very quickly by many different sources such as people, machines or sensors. This could be climate information, satellite imagery, digital pictures and videos, transaction records or GPS signals. Every single minute, the world generates 1.7 million billion bytes of data, equal to 360,000 DVDs. This works out at over 6 megabytes of data for each person every day.

As a result, the data sector is growing by 40% per year, 7 times quicker than the overall information and communications market. Businesses that build their decision-making processes on knowledge generated from data see a 5-6% increase in productivity. Big data is already helping us speed up the diagnosis of brain injuries, or forecast crop yields in developing countries. Global big data technology and services will create hundreds of thousands of new jobs in the coming years

While big data presents great opportunities, it is also challenging. Today's datasets are so huge and complex to process that they require new ideas, tools and infrastructures. It also requires the right legal framework, systems and technical solutions to ensure privacy and security.

Where can big data help?

Using big data can help doctors make the right choices more quickly, on the basis of information collected by other medical staff. Patients can benefit from more timely and appropriate treatment and be better informed about health care providers. A smart use of big data can better manage traffic flows, making our cities smarter. Citizens and companies can save time by using of route planning support systems. Big data enables the timely and appropriate delivery of products for consumers and more efficient processes, with cost savings, for business. With big data retail companies better know the needs and interests of costumers and, as a result, offer more personalised products (see What big data can do for you).

Why do we need a PPP on data?

Big Data is one of Europe's key economic assets. Harnessing its potential could give European industry a huge competitive advantage. However, today only 2 out of the top 20 companies changing lives and making money out of big data are European.

For this situation to be reversed, Europe needs to invest to strengthen all parts of the "data value chain", the people and organisations involved in data whether producing, analysing, using or creating value from it.

What is the PPP about?

The Public Private Partnership links up European industry (large players and SMEs), researchers, academia and the European Commission to cooperate in data research and innovation. The Commission will respond to the main research challenges and needs identified by industry and academia in a Strategic Research and Innovation Agenda (SRIA) in future Horizon 2020 work programmes and calls for proposals.

How much money is involved?

Investments are expected to reach around €2.5 billion over 2016-2020. The Commission has earmarked a budget of around €500 million in Horizon 2020. Each euro of that EU investment is expected to trigger €4 of private sector investment (around €2 billion).

Who is involved?

The Data PPP is a partnership between the European Commission and the Big Data Value Association, a non-profit, industry-led organisation whose members include ATC, IT Innovation, IBM, SINTEF, University of Bologna (CINI), Polytechnical University of Madrid, NOKIA Solutions and Networks, THALES, University of Duisburg Essen, Siemens, SAP, Engineering, TIE Kintetx, ANSWARE, Software AG, Orange, Atos, INDRA, ITI, VTT, Fraunhofer, DERI, and the Technical University of Berlin. The association is open for additional companies and research organisations to join.

The public and private side will meet regularly in a Partnership Board to discuss and decide joint strategies. The PPP will be led by an executive board consisting of the industry and research centres, as well as the Commission. An advisory board with Members States and EU representatives will provide political support and coordination across national programmes.

How will the PPP benefit traditional or large-scale industries?

The PPP will help focus investment and work on datasets that are strategic for European industry and where Europe can take a business or technological lead (e.g. manufacturing or agriculture). The PPP works across sectors and across borders, enabling companies from different industrial sectors to collaborate and benefit.

The PPP will include leaders from many European industrial sectors and will help identify high-volume and high-value industrial processes that today are inefficient due to a lack of agreement on data formats and processes.

For example, many companies in a given industrial sector collect data of a similar nature, but the absence of common standards creates a barrier to third parties to carry out data analytics on those data. Proposals to eliminate inefficiencies by means of industry-wide development of data standards will be funded under the PPP.

How will the big data PPP help SMEs?

As with all Horizon 2020-funded research, the PPP aims for at least 20% of the organisations participating in its projects to be SMEs. The initiatives will link up large and small companies and start-ups, by bringing together data owners and data analytics providers to develop new business ideas and new services. For example, an insurance company could offer particular insurance models and services for farmers based on weather data collected by national authorities.

The PPP will foster new business opportunities for different players, for example by making it easier for SMEs to access large data sets of big businesses and organisations, which can be a win-win for all.

The EU-funded Optique project has already shown that companies like Siemens who produce extremely large datasets from complex machinery with extremely expensive down-time, could save tens of millions of Euro a year if specialised SMEs running predictive maintenance analytics on their datasets, for example enabling them to replace a machine part before it breaks. The SMEs will in turn be able to re-apply these technologies at the service of other companies and industries with similar needs.

What about the "innovation spaces"?

The PPP will make use of "Innovation Spaces (i-Spaces)" that will offer secure environments for experimenting with both commercial and public data. These are cross-organisational and cross-sector environments that will allow experiments to happen in an interdisciplinary way. The PPP wants to build on existing national and European initiatives such as Germany's SDIL (smart data innovation lab), France's Teralab or the open data institute in the UK. i-Spaces will also act as business incubators and hubs for the development of skills and best practices.

What will the Big Data PPP do for jobs?

Big data means hundreds and thousands of new jobs across Europe in the coming years. For example, according to a study, in the UK the number of specialist big data staff working in larger firms is forecast to increase by 243% over the next 5 years to approximately 69,000 people. As a growing industry, the data sector relies on thousands of skilled data specialists, from clerical workers, to data analysts, to data scientists. However, data science is a relatively new discipline and the data industry, like many other parts of the ICT sector is facing a skills shortage.

The projects running under the PPP will give students at universities and research institutions access to large and realistic datasets and to user-friendly open source data analytics tools for which realistic data is needed. Projects will also promote the education of data workers on-the-job training, special courses to train different levels of data skills and offering new qualifications for engineers or other available human resources. More people will be trained for the new jobs.

What about security and privacy?

Better privacy and trust in big data is essential for more data to become available and for the data-driven economy to grow. The industry's Strategic Research and Innovation Agenda identifies data protection and pseudonymisation mechanisms among the technological priorities to be addressed in order to strengthen data privacy. The Commission will echo these industry priorities in Horizon 2020 work programmes. In parallel, the Commission will fund further research into technical solutions to integrate privacy- and security- enhancing features 'by design'.

The Commission will work with Member States and stakeholders to ensure that businesses receive guidance on how to make data anonymous and use pseudonyms to perform personal data risk analyses, and on tools and initiatives available to enhance consumer awareness. The EU data protection reform package is also the regulatory backbone for the data-driven economy. The Commission is working with the Council to ensure quick adoption of the data protection rules.

How will the PPP tackle data-related legal barriers and concerns?

PPP-related projects funded through Horizon 2020 will also aim to identify and find technical solutions for bottlenecks and legal barriers to a data-driven economy by:

• looking at the tension between privacy/data protection and access rights to data and databases;
• identifying the barriers related to ownership of data and intellectual property rights such as copyright and access rights, addressing the succession of data rights and the legacy of stored data (including the data of new, merged or bankrupt companies) and discussing legal aspects of new technical developments such as rights of automatically generated data;
• measuring the value of data (e.g. its pricing and the proper accounting of data resources as assets of businesses);
• identifying obstacles to transportability and free movement of data in the EU (e.g. enabling citizens to take over their complete set of telecom data when changing their telecom operator).

 

How will the PPP help reduce energy consumption?

Processing data requires a lot of energy – because of the vast amount of information being processed but also sometimes due to a lack of appropriate algorithms for the job at hand. In some cases, data needs to be processed down to very fine details (e.g. precise time or price including multiple decimals), in other cases, approximate answers are sufficient to meet the user's needs. Determining when such approximate analytics are sufficient is important because they are less costly, less timely and less energy consuming. The PPP will strive for solutions, new tools and algorithms that require fewer resources and less time to provide the same quality or right level of analysis. A concrete aim is to reduce the energy consumption for analytics by 10%.

What else is the EU doing about big data?

The October 2013 European Council concluded that: "EU action should provide the right framework conditions for a single market for big data".

In response, July 2014, the Commission outlined a new strategy on Big Data, supporting and accelerating the transition towards a data-driven economy in Europe. As part of this the EU will:

• support "lighthouse" data initiatives capable of improving competitiveness, quality of public services and citizen's life;
• develop enabling technologies, underlying infrastructures and skills, particularly to the benefit of SMEs;
• extensively share, use and develop its public data resources and research data infrastructures;
• focus public R&I on technological, legal and other bottlenecks;
• make sure that the relevant legal framework and the policies are data-friendly;
• accelerate the digitisation of public administration and services to increase their efficiency;
• use public procurement to bring the results of data technologies to the market;
• further consult with stakeholders to draw up a detailed, multi-layered and evidence-based action plan for advancing towards the data-driven economy and addressing Europe's future societal challenges.

To help EU citizens and businesses reap the full potential of data, the Commission is working with Parliament and Council on the successful completion of the reform of the EU's data protection rules and final adoption of the Directive on network and information security to ensure the high level of trust fundamental to a thriving data-driven economy.