Nobel Laureate Paul Romer had a brief, but tempestuous, tenure at the World Bank as its Chief Economist. He reportedly had a tense and adversarial relationship with Bank staff on multiple issues, but the proverbial straw that broke the camel’s back turned out to be Romer’s critique of the World Bank’s celebrated Doing Business reports (henceforth DBR). He made an abrupt exit in January 2018 after this critique became public. Even worse, media reports attributed to him the sensational claim that Bank staff fabricated the data to create an unfavourable impression of a particular country (Chile) because it was being ruled by a left-of-centre government and, therefore, presumed not to be business-friendly.
Romer subsequently recanted his position in his personal blog pointing out that he never intended to cast aspersions on the intellectual integrity of World Bank staff but stood by his methodological critique. It is ironic that, after his departure from the World Bank, Romer went on to win the Noble prize in economics, a singular honour that he shared with William Nordhaus. The World Bank was thus deprived of the opportunity to claim that it had a Nobel Laureate as its Chief Economist.
When Romer took aim at the DBR, he was taking on a ‘flagship publication’ of the World Bank. It grew out of a ‘project’ conceived in 2002. As the Bank states:
The Doing Business project provides objective measures of business regulations and their enforcement across 190 economies and selected cities at the subnational and regional level.
The Doing Business project, launched in 2002, looks at domestic small and medium-size companies and measures the regulations applying to them through their life cycle.
By gathering and analyzing comprehensive quantitative data to compare business regulation environments across economies and over time, Doing Business encourages economies to compete towards more efficient regulation; offers measurable benchmarks for reform; and serves as a resource for academics, journalists, private sector researchers and others interested in the business climate of each economy.
Countries are scored (100=best) and ranked (1=best) for ‘ease of doing business’. The progenitors of DBR did not anticipate that the ranking exercise would become so influential. Timothy Besley calls it ‘…one of the most prominent knowledge products produced by the World Bank.’
Governments, especially in the developing world, eagerly consulted the annual publication of DBR to find out how well or how poorly they were ranked. When significant ‘improvements’ in the business environment took place, they led to the World Bank President (David Malpass) to join in the national celebrations, as in the case of Pakistan. He proclaimed: ‘Your country jumps to 108th place from 136th last year…I congratulate you all, prime minister, chief minister, your team for this achievement.’
Some, such as India, set a policy goal in terms of reaching a particular rank (‘top 50’). Others, such as Indonesia, took an identical approach, aiming to be ranked 53rd in 2020 and 40th in about four years.
In retrospect, this scramble to win the rankings game by many countries was a misguided endeavour that the World Bank made little or no attempt to rectify. Now, it seems, it has no choice. An official statement issued by the World Bank on August 27 acknowledges ‘data irregularities’ that were considered serious enough for the DBR to be ‘paused’. Here is an extensive extract from the August 27 statement.
A number of irregularities have been reported regarding changes to the data in the Doing Business 2018 and Doing Business 2020 reports, published in October 2017 and 2019…
The integrity and impartiality of our data and analysis is paramount and so we are immediately taking the following actions:
- We are conducting a systematic review and assessment of data changes that occurred subsequent to the institutional data review process for the last five Doing Business reports.
- We have asked the World Bank Group’s independent Internal Audit function to perform an audit of the processes for data collection and review for Doing Business and the controls to safeguard data integrity.
We will act based on the findings and will retrospectively correct the data of countries that were most affected by the irregularities.
The Board of Executive Directors of the World Bank has been briefed on the situation as have the authorities of the countries that were most affected by the data irregularities.
The publication of the Doing Business report will be paused as we conduct our assessment.
This embarrassing development was duly reported by leading representatives of the global media – see, for example, The Economist, Financial Times, Bloomberg Quint and Reuters, UK . All of them referred to Paul Romer’s concerns about DBR.
One wonders what Romer is thinking now. He also wanted to review past DBRs because he felt that changes in methods of measurement were too frequent and led to arbitrary changes in national rankings. He highlighted the case of Chile whose rank fell by more than twenty places in a short span of time.
Two analysts, Justin Sandefur and Divyanshi Wadhwa, from the Center for Global Development illustrated spurious improvements in the ranking for India due to changes in methodology and even changes in sample size. An external audit of DBR commissioned by the World Bank in June 2018 exonerated World Bank staff from data manipulation but noted the following: ‘… both the survey questionnaires and how the indicators are calculated have changed frequently… (F)requent methodology changes reduce the value of the indicators to researchers, policy makers and the media.’
It appears, however, that the worst fears of Romer have turned out to be true. It was not just a case of frequent changes in methodology, but ‘data irregularities’ that has caused the World Bank to temporarily cease the publication of DBR. Who committed these irregularities is unclear. The Bank faces the formidable challenge of salvaging the reputation of a flagship publication. Will it be able to persuade member states to take the DB rankings seriously following this debacle? Only time will tell.
Reflections on development and governance 