Development Initiatives

Reflections on the 3rd OECD World Forum on 'Statistics, Knowledge and Policy' Oct 2009 - Dan Coppard

Q. How many statisticians does it take to change a light bulb?
A. That depends, it’s really a matter of power.

The measure of societal progress was the light bulb in question at last week’s OECD World Forum, and having the power to change it emerged as one of its key unofficial themes. Held in the thriving city of Busan, South Korea, this was the third biennial forum tasked with rethinking and challenging what progress really means, and what measurements best define societal goals, best inform the policies to get there, and best monitor the extent of their success.

Countering the supremacy of income-based indicators such as GDP, building new measurement frameworks that incorporate social and environmental aspects of wider wellbeing and using data to hold policy makers accountable and rebuild state-society trust were common themes running throughout the week. Harnessing political support for change and the opportunity to influence and advance the international agenda was the real name of the game.

If conference size alone was the primary indicator of progress, then this event had it by the bucket load: over 1900 participants, 200 speakers, 59 countries, making it the largest conference ever hosted by the OECD. Grand statements were aplenty. This was OECD’s most important project; ‘few projects had greater potential to improve the wellbeing of people’s lives’ claimed more than one discussant. And while sombre-suited statisticians and economists inevitably formed the largest constituency, ministerial presentations of country initiatives (Australia, South Africa, Dominican Republic, South Korea) heads of state keynote addresses (Slovenia, South Korea) and senior representation of international consortia (International Association of Economic and Social Councils, INTOSAI) and international organisations (World Bank, UNDP) leant a degree of gravitas.

Angel Gurría was keen to emphasise progress made over last two years, from academic and civil society initiatives to examine wellbeing, to the OECD’s own Global Project network and the World Economic Forum’s Global Council on Benchmarking Progress. Recent increased political demand for alternative indicators, including the Sarkozy-mandated Stiglitz Commission, the EC Communication on ‘GDP and Beyond’, and the G20 Pittsburgh Communiqué calling for development of better wellbeing and sustainability indicators provided tangible relevance for the project. Stirred up by the financial crisis, there was a real sense of being on cusp of a paradigm shift – although a shift towards what was not always clear.

A keynote presentation by Stiglitz set an early high and reference point for the Forum. The central message: what you measure affects what you do. Measuring the wrong thing will promote poor policies, distort economic behaviour, and propagate wrong inferences of good policy. Rather than rehearsing the findings of his Commission report, he detailed why criticisms of GDP - never intended as a measure of wellbeing and an increasingly poor measure of market activity - have become mainstream.

The recession has highlighted its limitations: healthy GDP growth 2005-07 was fictitious, based largely on evaporating financial sector profits and the real estate bubble. A poor measure of output, GDP also fails to capture financial sustainability, while other, better measures, such as household and national debt, do exist. The inability to consider environmental sustainability is even more damaging, exemplified by the distorted behaviour created by a failure to account for the social cost of carbon. In this case, prices will inevitably adjust, and our choice of indicators will have to be ready for that. GDP misinterprets the growing share of government in the economy by measuring inputs (e.g. health expenditure) rather than outputs (e.g. state of health). That the US’s health expenditure (17% of GDP) is higher than France’s (11%) does not capture the fact that health in France is better, nor that much US expenditure is ‘defensive’ to counter unhealthy lifestyles perpetuated by economic growth. Similarly, high prison expenditures might well contribute to GDP (government $s to prisons outcompete universities in the US), but others may well view this more as an indicator of a dysfunctional society than a healthy economy. Crucially, processes of commodification – the transition of goods from non-market to market, or from government to private sector, should not be allowed to artificially inflate GDP simply because it becomes easier to put a price on them.

Revising the metrics we use to measure progress, he argued, will remove a number of ‘false choices’ faced by public policy. Incorporating the costs of environmental degradation into GDP, for example (the ‘green GDP’) will overcome economic growth ¬vs. environmental protection debates. However, the implications of using the wrong metrics go far beyond policy choices. Stiglitz emphasised the growing divergence between what indicators told society about itself and what people actually felt. This is generating a ‘dangerous mistrust of government’. Increased inequality has meant that GDP growth has been shared by a small minority while most people saw no benefit. GDP in 2008 was higher than 2000 despite the crisis, leading to claims of psychological distortion – “people were better off but just didn’t know it”. Other indicators such as median household incomes, however, showed that people were indeed worse off. GDP can therefore mask inequality with averages, as well as other forms of ‘illbeing’ not directly linked to income, but central to ideas of multidimensional poverty.

This threat to the citizen-state ‘social contract’ was a concern repeated by statisticians throughout the Forum, resonating with calls for a greater need for public participation in the identification of wellbeing indicators, and wider recognition of the role of civil society in the use of information and formulation of policy. Need for more transparent, timely, disaggregated information to better hold governments and their policies to account was highlighted more than once, and Pier Carlo Padoan (OECD deputy secretary general) neatly encapsulated the ‘virtuous circle’ public accountability could provide in generating momentum for better measures, policies and outcomes. Vikram Nehru (World Bank Chief Economist for East Asia and the Pacific) went even further: a healthy statistical infrastructure is not just a measure of good governance, it’s the creator of good governance; and its absence, particularly in fragile and low-income states.

So far so good, but an event of such scale can’t be without its tensions. The debates, not always explicit, seemed to focus on three areas.

1.      Complexity: One or many? Hundreds of indicators were described during the numerous parallel sessions, with calls for hundreds more to fill crucial information gaps. Stiglitz concluded, quite reasonably, that no one single composite indicator can capture wellbeing. Combining indicators, at best, yield meaningless indicators (apples and pears), and at worst, distort realities with a whitewash of averages. His Commission proposed a range of indicators, allowing different people, institutions and governments to combine and prioritise in ways to measure what is most important to them. However, there are potentially diminishing returns for each new indicator. One participant complained of a ‘fog of confusion’ created by a plethora of indicators in Africa, while Duncan Green highlighted the inherent conflict between a ‘dashboard’ of indicators, the weak statistical infrastructure of many low income countries, and the need for high frequency, real time data in poor countries when crises strike. Goeff Mulgan perhaps showed a middle path, calling for the selection of 3 or 5 standardised indicators, each a constellation of data that could be disaggregated to identify relationships between different components.

2.      Subjective or objective While open debate was minimal, there was a clear tension between discussions in parallel sessions elaborating numerous subjective measures of wellbeing, and plenaries that emphasised objective measures that could replace, or complement GDP. Only the former can assert the claim of fostering a true paradigm shift. Quoting Einstein, one participant argued, “you can’t solve problems with the same thinking that created them.” Subjective measures commonly identified common sets of ‘domains’ of wellbeing (e.g. health, security, empowerment, aspiration) through participatory methods – asking people what was important to them and how it could be measured. These are often grounded in ideas of multidimensional aspects poverty (e.g. the Oxford Poverty and Human Development Initiative, Oxfam Hong Kong / Bath Centre for Wellbeing in development, Legatum’s Prosperity Index) that are rarely captured by standard indicators. Richard Layard and Frances Stewart called for the primacy of self-reported life satisfaction (happiness), with Layard claiming this would replace GDP within 25 years.

By contrast, government initiatives remained firmly grounded in objective numbers. Tae-shin Kwon, Korea’s minister to the Prime Minister’s office praised Oxfam Hong Kong’s participatory ‘conversations on wellbeing’ programme, before launching his own 101 objective indicators of social progress. Stiglitz, and, one suspects from various comments, the OECD, favour such objective indicators. Here, the sub-debate is over toppling GDP entirely, or complementing it with a range of other measures. “Let’s have more indicators, but please keep GDP” called Antonio Marzano in his plenary address, to muted applause. Rebecca Blank, US Secretary of Commerce, argued that in policy circles, subjective data would always be subsidiary. New data would add, not replace GDP. Much data is already available, but it’s disparate, unused and inconsistent. Considerable benefit could come from better use, organisation and coordination of existing information.

3.      Politics of change. Blank further injected a caffeine shot mid-way through the forum with her ‘get real’ address, accusing the conference room of political naivety. All these indicators may be interesting, but statisticians and economists are small fry in the big wide world. Better data, she argued, wouldn’t have prevented the last economic cycle, nor will it the next. Policy makers look for ‘good news data’ and so will continue to make the same mistakes. Rather than better statistics, education of civil society and government officials on how to use them is where energies must be directed. This political economy of evidence-based policy was taken up specifically in a parallel session, and the message was clear: better data does not automatically equal better policy.

This concern was passionately applied by Pronab Sen (chief Statistician of India) in the concluding discussion, calling for a shift away from the technocratic to a focus on an ethical model that would distil the Forum’s ideas into a coherent message for policy makers. The Forum was developed by statisticians, he argued, but it mustn’t become dogged by debates over statistical methods. Rather, it must now “create the conditions where statisticians can identify strategic partners, based on ethical positions, in respective governments.” Simultaneously announcing India’s hosting of the next Forum, his call was met with grand applause.

The last word goes to Pali Lehohla (Statistician-General, South Africa): We become what we measure. The means of living – what we measure – has become the object of life. Do we like what we have become? Who said statisticians were dull?