Using private sector data to create public value
As the potential for using private sector data to tackle societal challenges grows, the need for clear rules becomes more urgent
Debates over data and its governance usually revolve around how the increased collection and use of data can result in breached privacy, concentrated economic power, growing inequities, more divided societies and eroded political systems. Far less attention has been devoted, however, to another failure: our inability to re-use private sector data for the public good.[1]
This oversight is relevant for at least two reasons. One, the ability to use data is fundamental for tackling societal challenges (including, among these challenges, the governance of data itself). Two, most of the data that could be leveraged to tackle public problems is under the control of the private sector. Just think about the mobile phone data collected by telecommunication companies, data on credit-card transactions registered by financial corporations, data on all our online activity gathered by digital platforms, data on energy and water consumption managed by utilities, data on traffic, air pollution and the state of public infrastructure controlled by companies owning the sensors embedded in the built-in fabric of our cities or the data on mobility patterns collected by ride-sharing companies. And this is just the tip of that much deeper iceberg called datafication.
Recognizing the importance of this issue, the European Commission requested a report from a High-Level Expert Group on Business-to-Government (B2G) Data Sharing that released its findings in early 2020. The report identified several B2G data sharing cases, but also stated that most of these practices are experimental and not sustainable for a number of reasons. In a recent paper, Bertin Martens and Néstor Duch-Brown (2020) identified some of the main barriers to increased data sharing from both the supply (Business) as well as the demand (Government) side of the equation.
On the supply side, there are several reasons why private sector companies do not want to share their data with the public sector. First, they fear that governments may use the data to regulate their activities. Second, they think that sharing the data they control can result in breaches in their obligation to protect the privacy of data subjects. Third, they may be worried about potential losses in their competitive advantage, or in some cases a direct hit in their business model, for example when selling data or data-related products is the core product of their business. Fourth, they are unable to monetize the positive externalities of the use of data by the public sector.
On the demand side, one of the main barriers is that often public bodies lack the capacity to access and use data from the private sector to create public value. Another challenge is the lack of awareness by public organizations about available datasets or suitable data-sharing partners. This is aggravated by the fact that it is often difficult to assess ex-ante whether the data to be shared by the private sector is of the required quality to tackle the issue it was acquired for. Acquiring data can also be costly for public organizations, particularly when the data provider holds a monopolistic position and can therefore charge high prices for the data.
There are other problems that are cross-cutting. The absence of common standards and interoperable systems may become an important technical barrier to data sharing. There is also a general lack of transparency and citizen engagement in current data sharing, which may generate general concerns and resistance, particularly if some of these exchanges do not address important ethical considerations. That is why data sharing needs to be conducted making sure that biases are recognized and addressed, and that the rights of individuals and businesses are protected.
Over the last years experiments with models to access data generated by the private sector have mushroomed. Some of these models include the mandatory “sharing” of data, such as regulations establishing data sharing obligations or data sovereignty clauses introduced in public contracts. On the opposite side we can find completely voluntary data exchanges carried out through public procurement of data under market conditions. In between these there are different models that, although in general still voluntary, can vary greatly in the type of data shared (i.e., the level of granularity and rawness of the data), the level of access, the level of collaboration required and the type of compensation paid for the data.[2]
Cities are among the governments that are experimenting with some of these models to access data from the private sector. This is hardly surprising, given the amount of novel data being generated that could be useful to tackle the many societal issues that city governments need to deal with. In a recent paper, Marina Micheli presented the findings of a qualitative research that examined such practices across 12 European municipalities. The four models that featured more prominently in her research were: data donorship, public procurement of data, data partnerships and pools, and data sharing obligations. An important finding of the paper is how, given the lack of broader regulatory and policy frameworks, certain municipalities (particularly bigger ones) can negotiate beneficial terms with companies, while smaller municipalities lack any leverage to enter in similar agreements. This can widen the inequality in the access to data among municipalities and shows the limitations to the city-by-city approach, but also the “potential of collective efforts to increase local administrations’ negotiating-power and address some of the asymmetries of current data ecosystems.”
The current state of the art suggests that while there is large room for experimentation and for businesses and governments to try different models based on their goals, incentives, and interests, there are aspects that will need to be regulated. For example, it will be important to determine what are the reasons that justify the mandatory sharing of private sector data with public bodies, and under what conditions (type of data, access level, compensation, etc.) that access needs to happen.[3] In these cases, the general benefits of the access to the data will need to be weighted against the costs incurred by private actors as well as the potential negative impact on companies’ incentives to innovate if they cannot reap the benefits of their investments (but without forgetting the public investment that often enabled this private innovation in the first place, nor the need to continue funding such public infrastructure).
There may be other situations where access and sharing will need to be completely voluntary, and the terms of the exchange freely established by the parties. Even in these cases, however, a broader regulatory framework may be required to ensure interoperability, ethical standards, anti-monopolistic pricing, and the protection of rights (both of citizens and companies) in the data economy. What seems clear is that, as the potential for using private sector data to tackle societal challenges increases, the need for more clear rules and refined models will become even more urgent. The lessons from the many cities who are already experimenting in this area will provide invaluable input into this process and should be therefore closely followed.
[1] Stefaan G. Verhulst, Andrew Young, Michelle Winowatan and Andrew J. Zahuranec, 2019. “Leveraging private data for public good. A descriptive analysis and typology of existing practices,” GovLab, at https://datacollaboratives.org/static/files/existing-practices-report.pdf,
[2] The Report by the High-Level Expert Group distinguishes between four types of compensation: free of charge, marginal costs for dissemination, marginal costs for dissemination plus fair return on investment and market price. The report also contains a list of data sharing models, and another typology can be found in Verhulst et al. (2019) see footnote [1]
[3] See, for example, the regulation of the access to private data by public bodies in exceptional situations established by the Data Act published by the European Commission on 23 February 2022.
I have exciting news to share: You can now read Datapoli’s Newsletter in the new Substack app for iPhone.
With the app, you’ll have a dedicated Inbox for my Substack and any others you subscribe to. New posts will never get lost in your email filters, or stuck in spam. Longer posts will never cut-off by your email app. Comments and rich media will all work seamlessly. Overall, it’s a big upgrade to the reading experience.
The Substack app is currently available for iOS. If you don’t have an Apple device, you can join the Android waitlist here.