Quantifying the challenges facing the Data Revolution in Africa: A first attempt

A blog by Bill Anderson for the Africa Open Data Conference 2015 in Dar es Salaam, Tanzania

The job of statisticians is to make the best sense they can of the data available to them. By and large they do a good job. National Statistics Offices play a critical role in the government and development of all countries. In Africa, however, they do so against steep odds as very little actual data is available.

In many countries the maternal mortality rate, for example, is calculated from an algorithm that includes the GDP (calculated from an algorithm based on a range of estimates), the ‘Fertility Rate’ (calculated from an algorithm based on births and age spans) and a statistic known as ‘skilled attendance at birth’ (calculated from survey estimates). The reason for this is that very few countries have mature, functioning Civil Registration Systems and Health Management Information Systems that can produce accurate vital statistics on the causes of death. Would this kind of methodology, happily accepted as part of the Millennium Development Goals, be acceptable to the governments and citizens of developed countries?

The bread and butter of statisticians in developing countries is a 10-yearly population census and a variety of surveys based, generally, on relatively small samples. (Uganda has one of the continent’s best national statistics systems, but its main survey covers only 1 in 1,000 households.) This kind of data is useful for global databases, is generally funded and supervised by global institutions interested in globally comparable datasets and is currently critical for budgeting and planning departments in developing countries in the absence of anything better.

But if we are going to take the global rhetoric of the Data Revolution seriously – “A World That Counts”, “Leave No One Behind”, and “No One should be Invisible” – this isn’t good enough. National and local decision makers need information based on data that is disaggregated down to the lowest level of administration. Small sample surveys won’t tell you where poor people actually live. Small sample surveys don’t allow for a health service to know where to best allocate its resources. Services are delivered through local authorities who need intelligence on their local communities to know how best to serve the people. Counting people to make people count is what the Data Revolution is about.

This can only be achieved through investments in people and infrastructures to collect and produce registry and administrative data that delivers a continuous information service to governments and citizens alike. Surveys will always play an important role – to fill gaps and provide sense-checking quality control – but they cannot deliver a sustainable solution on their own. You can’t heal a broken leg with sticking plasters.

What then are the specific challenges facing African countries? There are currently two sources of comparative information that provide a good starting point. PARIS21 has collected data from a variety of sources to produce an extensive Metabase containing statistical capacity profiles for all developing countries. The World Bank Statistical Capacity Indicator provides a 10-year timeline of 25 scores assessing methodology, data sources, periodicity and timeliness.

At Development Initiatives we believe there is a need for a more granular and timely resource that reflects both best practice and challenges across all the data sources required to meet the needs of statisticians and the goals of the Data Revolution. This is our first attempt based on a preliminary desk study, which we hope to build upon through more in-depth country-based research. We have built a table for all African countries mapping nine indicators against a simple traffic light system. A table of references (for each cell in the table) is also provided. For comparison we have reproduced the World Bank’s Statistical Capacity Source Data and Summary Scores into a similar format.

It needs to be stressed that the exercise that we have conducted is work in progress. The notes that follow highlight the problems collating the metadata, as much as they point to the actual data challenges we face.

Censuses

Basic data on population censuses is readily available and over 80% of countries have conducted a census in the last 10 years. More challenging is an assessment of the coverage and quality of the census and the availability of the data (particularly anonymised microdata that is disaggregated down to the lowest level of administration).

Surveys

As most surveys are driven by global institutions it is perhaps not surprising that they are well documented and the data provided. Performance, however, is not great. Only 45% of countries have conducted one or another form of household survey in the last 3 years. According to the World Bank only 33% have conducted more than 2 poverty-related surveys in the past 10 years. And only 20% have conducted an agricultural survey in the past 3 years.

Civil registration

The only credible way to count people on an ongoing basis is through civil registration – not only by documenting the events but by producing usable vital statistics. This is undoubtedly the biggest challenge facing the Data Revolution. The excellent World Bank report on scaling up investment in civil registration and vital statistics highlights the crisis. At the same time however substantial efforts are being made across Africa to deal with this. Data on the current state of progress is surprisingly difficult to obtain. The UN Statistics Division collects metadata on the status of systems through its Vital Statistics Questionnaire but doesn’t publish it. Assessments of progress from Africa often make it difficult to distinguish between planned and actual achievements and the World Bank indicator is too binary and provides simplistic overview. So while the World Bank reports that only four African countries have ‘complete’ systems. Our tentative research indicates that 50% of countries have made progress including data from the registration of births in vital statistics, and 11 countries have at least partially functioning systems for registering deaths.

Administrative systems

All ministries and government agencies collect data for planning, budgeting, operations and monitoring. In most instances these datasets are non-standard and inconsistent. Turning the efforts that go into these unstructured practices into sustainable, usable statistics is the second big challenge faced by the Data Revolution. Data on the status of such systems or datasets is extremely difficult to collate, a reflection of the disregard currently shown to these systems. In our first attempt we have looked at Health and Education Management Information Systems (which typically record staff, logistics and performance in all health and education facilities). We attempted to investigate Land and Agricultural Management but failed to find sufficient data.

Other data

Our work highlights some of the key data sources underpinning development statistics. It is far from exhaustive. Satellite data, mobile phone metadata, feedback loops, academic, private sector and citizen data should all be included in a comprehensive assessment.

Methodologies

As important as the data are the methodologies employed to convert source data into meaningful statistics. Of particular importance is an understanding of how methodologies change and statistics are rebased as and when better data from new sources becomes available.

The start of a journey

This work is being carried out in the belief that unless we continue to map out the specific challenges facing the Data Revolution (and the successes) a lot of financial and human resources are going to go to waste. We trust that our second attempt will be more accurate and purposeful.

Download our table for all African countries as an Excel file.

Research by Manon Simeon, Rosanna Collins and Guto Ifan.