Data as a force for public good

Public intent data—data collected with the intent of serving the public good by informing the design, execution, monitoring, and evaluation of public programs and policies—are a prerequisite for many government functions. Such data include administrative, census and survey data produced by government agencies, citizen-generated data produced by individuals, and machine-generated data produced without human interactions.

Public intent data can bring value to development through several pathways. Three of these pathways include (1) holding governments accountable and empowering individuals, (2) improving service delivery, and (3) prioritizing scarce resources.

One example of how public intent data brings value to development comes from Pakistan. Prior to 2008, Pakistan’s Punjab province suffered from poor government service delivery due to inefficiencies, corruption, and other factors. As one example, when visiting a land services official, 75% of households reported paying a bribe. Lack of digitized service delivery processes made it impossible to track service delivery and monitor performance and satisfaction with services.

In an attempt to take on these challenges, in 2008, officials in one district of Punjab put in place a pilot Citizen Feedback Monitoring Program. Through this program, data on service provision were crowd-sourced from citizens using simple text messaging and other ICT applications. This program gave voice to the people who received these messages, empowering them to directly report on gaps and petty corruption in service delivery. Analytical reports were sent to government officials enabling them to identify patterns and take evidence-based corrective measures. In 2012, the initiative was scaled up to 36 districts of the province and across 25 different public services. The initiative brought value to development through the three channels mentioned above:

Gaps in public intent data

Despite the potential for public intent data to improve programs and policies, is the value from the data fully reaped? Or can governments do better? One tool that can help us answer these questions is the Statistical Performance Indicators (SPI) created by the World Bank. The SPI tells us where there are gaps in data services, data products, and data sources, amongst other areas. Data services look at the interaction between data supply and demand such as the openness of data and quality of data releases. Data products review whether countries report on important SDG indicators. Data sources assess whether censuses, surveys, and other data sources are available.

Each of these domains can be summarized in one score from 0 to 100, where a value of 100 indicates a perfect score. Most countries are far from receiving a perfect score in any of the domains. In other words, they have a long way to go reap the full value from public intent data.

In Libya, for example, online access to data is limited and metadata are often unavailable, giving a low score on data services.

At the same time, key recent data sources such as surveys, census and administrative data are missing in Libya, giving the country a low score on data sources as well.

In Mexico, on the other hand, statistical performance is quite high. Mexico has recent estimates of many key SDG indicators, giving it a high score in data products. It is one of few countries with recent data on indicators on gender equality.

So how can countries do better? How can they get closer to the statistical performance of Mexico? Chapter 2 of the WDR highlights four areas through which countries might improve their statistical performance and the value derived from public intent data. Below, we will dive into two of them, financing and governance.

Deficiencies in financing of data

One way to improve statistical performance is to increase financing for data. Underinvestment in public intent data systems is widespread. Only half of countries (among those with information available) had a national statistical plan that was fully funded in 2019. Lack of national funding for statistics is especially a struggle for fragile and conflict-affected countries, countries in Sub-Saharan Africa, and low-income countries. Whereas 93% of high-income countries have a fully funded national statistical plan, not a single low-income country has one.

Deficiencies in governance of data

Another way to improve statistical performance relates to data governance. The legal framework governing data production and data exchanges is a common barrier. Outdated statistical laws can make it difficult for data-producing agencies to operate and collaborate effectively in light of recent changes in the data landscape, such as the proliferation of new data types, sources, and producers.

In general, the older the national statistical law, the lower is statistical performance as measured by the statistical performance index (which averages the three pillars plotted above as well information on data use and data infrastructure).

The institutional independence of the National Statistics Office (NSO) is another important element of a successful public intent data system. An indicator capturing the independence of NSOs in all African nations is included in the Ibrahim Index of African Governance. The indicator measures the institutional autonomy and financial independence of an NSO. A perfect score indicates that an NSO is able to publish data without clearance from another government branch and has sufficient funding to do so.

A higher score on the NSO independence indicator is highly correlated with higher statistical performance as captured by the statistical performance index.

Realizing the potential of public intent data

How can countries improve their data financing, governance, technical skills, and demand? Often, this necessitates a political understanding and appreciation of the value of data for policy making.

Civil society has an important role to play. A free and empowered press is a critical check on government power in general and on government interference with statistical independence and data transparency in particular. Greater press freedom, as measured in the World Press Freedom Index compiled by Reporters Without Borders is correlated with statistical independence and performance.

As such, a free press is one way of addressing a data governance issue. A free press also empowers people through transparent sharing of information. Chapter 2 lays out other specific ways through which civil society, academia, and government agencies can address the deficiencies in data financing, governance, technical skills, and demand.