BY ANAND VENKATANARAYANAN ON 03/10/2017 •
The World Bank’s estimate that Aadhaar has the potential to save $11 billion in subsidies every year has repeatedly been used by the Centre to justify the programme. But does this figure hold up under close scrutiny?
The organisation has admitted that its referencing was incomplete but says that its $11-billion figure is based on an internal extrapolation of two other studies. Credit: PTI, Reuters
Evidence-based policy-making usually proceeds by outlining the problem at hand, discussing various solutions, conducting studies to understand the effectiveness of these solutions and then prescribing a course of action based on accumulated evidence.
It’s safe to say that the media and policy narrative surrounding India’s biometric authentication programme hasn’t followed this process at best and has been skewed in favour of speedy implementation at worst.
For instance, it is clear that both the previous UPA-II and current NDA government painted a highly-exaggerated picture of savings from direct benefit transfers for LPG – a narrative that was largely accepted by an unquestioning mainstream media.
Another figure that has floated around over the last two years is that the Aadhaar project has the potential to save $11 billion in subsidies every year.
There is enough evidence to show that this figure – which originates from the World Bank – doesn’t hold up against close scrutiny.
Most recently, this $11 billion statistic was used as part of the Centre’s argument before the Supreme Court, as part of its reply to a writ petition filed by Shantha Sinha challenging the government’s notifications that made Aadhaar compulsory for various welfare schemes.
A screenshot of the affidavit submitted by the Centre to the Supreme Court. Credit: The Wire
The affidavit by the government of India (shown above) also attaches the relevant portions of a 2016 World Bank report on digital dividends (page 195, shown below).
A clipping of the World Bank’s 2016 report. Credit: The Wire, World Bank
The World Bank study specifically talks about how India’s digital ID programme can potentially save “over US$11 billion per year in government expenditures through reduce leakage and efficiency gains”.
The reference for this sentence, as can be seen, is provided in a footnote (number 4) which refers to a separate study. Crucially, this other study referenced by the World Bank was not attached in the government’s affidavit before the Supreme Court.
What is this study that backs up the $11 billion savings figure? On page 197 of the World Bank report, the footnote 4 corresponds to a study carried out by Shweta Banerjee in 2015.
The term ‘CGAP’ referred to here is the Consultative Group to Assist the Poor – a global partnership of research organisations that is housed out of the World Bank.
On closer examination, the CGAP study mentioned in the link above shows a very different story than the one put forth by the Indian government in its Supreme Court affidavit.
A clipping from the 2015 Banerjee study. Credit: The Wire
The last sentence of the brief (shown above) is enlightening. The CGAP report does not talk about savings as a result of adopting a direct cash transfer model – but about the quantity or total value of the money that has been transferred.
If this is taken at face value, it shows that World Bank was quite clearly mistaken. Is it possible that the government was not aware of this mistake? Very unlikely, since this potential goof-up was extensively discussed and laid out by IIT Delhi professor Reetika Khera in July 2016.
Since then, the claim, however, has been repeated by IT minister R.S. Prasad (September 8, 2016) and by UIDAI’s CEO (May 13, 2017). Capping it off, of course, was reference to the $11-billion figure in the SC affidavit on April 27, 2017.
So what gives? Did the World Bank make a mistake – or did it produce a different study and forget to cite it?
World Bank response
This author, and others, reached out to the World Bank in July 2017. The first email sent asked about the issue of the value of transfers versus value of savings. Specifically, the organisation was asked how it referenced Banerjee (2015) as the source for the $11-billion figure when that study made no such claim.
The World Bank replied with the following response:
“The potential savings of over $11 billion in government welfare programs is an extrapolation based on two rigorous papers that estimated the reduction in leakages in the National Rural Employment Guarantee Scheme (NREGS) and the government’s domestic fuel subsidy program respectively due to Aadhaar. Muralidharan et al (2014) estimate that biometric registration, authentication, and payments in NREGs led to a 10.8 percentage point reduction in the leakage of funds (the difference between the wage payment outlays reported by government officials and those reported by households). Barnwal (2015) estimated the UID-based transfer policy reduced fuel purchases in the domestic fuel sector by 11-14%, suggesting a reduction in subsidy diversion. Extrapolating these leakage reduction rates to all Government of India welfare programs – amounting to roughly $70-100 billion in government expenditures—yields savings in the range of $8-14 billion. Instead of reporting this range, we reported the midpoint of $11 billion as potential savings.”
The words extrapolation, estimate and potential are important in this response. The World Bank response basically implies that there are two other inputs (research papers by Barnwal and Muralidharan), and these two inputs were extrapolated using a calculation to generate the final potential savings figure.
A follow-up question was consequently sent to the World Bank, asking it if the organisation’s extrapolation calculations could be shared and also as to why Banerjee (2015) was quoted as a reference in the 2016 report if the institution had actually performed separate research to support its claim.
The World Bank replied as below:
“It turns out we have shared with you all information we could. There is one small caveat to add – the footnote 4 is incomplete and should also have included a reference to Muralidharan et al (2014) and Barnwal (2015), the latter two papers being cited elsewhere in the report.”
Let’s recap the Aadhaar savings narrative so far. The government has officially claimed $11 billion in potential savings based on a World Bank report. The actual report contains a reference to a study that does not talk about $11 billion in savings, but $11 billion in transfers.
When asked, the World Bank admitted that the referencing in its report was incomplete and pointed to two papers (one by Barnwal which is on LPG and the other by Muralidharan which looks at smart cards in NREGA).
Barnwal (2015)
Barnwal (2015) was first brought to the public’s attention when chief economic adviser Arvind Subramanian referred to in a New York Times article and used it as a basis to state that Aadhaar could potentially save $2 billion. The Barnwal study has since then been debunked by the Comptroller and Auditor General of India (CAG), the International Institute for Sustainable Development and various articles in Economic and Political Weekly.
The government itself has admitted that LPG savings were due to other factors in Parliament: “Lower subsidy during the successive years is due to various factors, including introduction of direct transfer of subsidies into the accounts of consumers (PAHAL Scheme), fall in international crude oil prices and “GiveItUp” campaign.”
Rahul Lahoti, who is associated with Germany’s University of Goettingen and has written extensively on the issue of Aadhaar savings, has pointed out that extrapolation of LPG savings to other schemes is “not advisable”.
“The current analysis of LPG savings are based on very big assumptions, which might not hold. If these assumptions about exclusion/identification of duplicates are violated the actual savings might vanish. But additionally there are several other important caveats. In LPG – it is not clear how much of the savings are due to Aadhaar vs. the direct bank transfer. It might be the case that DBT without Aadhaar itself was effective in eliminating a majority of the leakages claimed by Aadhaar. So Aadhaar might not be necessary in its implementation,” he said, in response to a questionnaire sent by The Wire.
Lahoti went to add:
“LPG subsidy as is well-known and I show in my previous EPW piece is mostly a benefit enjoyed by urban rich/middle class. This group of beneficiaries is different than ones using PDS/mid-day meal. PDS is enjoyed by relatively more rural and poor population where infrastructure is less developed.
In LPG subsidy there is only a one-time linkage between Aadhaar number and the bank account, whereas use of Aadhaar in PDS requires biometric authentication each time the benefit is availed and that process can be error-prone/dependent on infrastructure (electricity/internet connection). “
Muralidharan (2014)
The second paper the World Bank used as part of its extrapolation is by Muralidharan (2014) which studies not Aadhaar but the usage of biometric smart cards in the Centre’s NREGA programme.
The salient features of the Muralidharan paper are:
- It measured the impact of introduction on biometric smart cards for NREGA and Social pensions and found that the introduction did not change the fiscal outlays (Section 4.2, Page 16).
- It found a miniscule number of “ghosts” in the NREGA program after biometric smart cards were introduced and only 1.1% of ghosts in the Social pension scheme (Table 5, Page 36), which is entirely consistent with other studies.
The first point requires more explanation. What really is savings? There are two distinct categories of savings:
- Fiscal savings implies that if ₹100 is spent on a particular program and if introducing biometric smart cards removed duplicates which reduced expenditure to ₹90, then savings is 10%.
- Efficiency implies that If ₹100 is spent on a particular program and only ₹80 was reaching the beneficiaries, it means program efficiency is at 80%. If introducing biometric smart cards increases efficiency to 90% when ₹100 is still spent, it is efficiency improvement but does not affect fiscal outlays.
Muralidharan (2014) explicitly said that there are no fiscal savings because of introduction of biometric smart cards and there are only efficiency improvements. Hence using this paper as an input to forecast/extrapolate fiscal savings would be inaccurate.
On this point, Lahoti in his emailed response further clarified on the question of using Barnwal and Muralidharan as basis for calculating Aadhaar savings on other social welfare schemes as “unclear”.
“The problem with using Barnwal’s findings to extend Aadhaar to other spheres is that LPG subsidies are different as argued before. Maybe both in NREGA and LPG, having just a direct benefits transfer to the bank might reduce leakages and it’s not clear if biometric authentication (smart cards or Aadhaar) adds any value,” Lahoti said.
World Bank and Aadhaar savings
Taking a step back, there is no publicly available information that shows how an extrapolation of these two papers can be used to derive potential Aadhaar savings in the range of $8 to $14-billion.
The World Bank has refused to share these calculations – and indeed glosses over the fact that the Banerjee article that was actually referenced talks about value of “transfers” and not “subsidy savings”. Further attempts made by The Wire to reach out to World Bank officials and World Bank chief economist Deepak Mishra went unanswered.
In the absence of a specific study or proof-of-work, the simplest explanation is that the World Bank mistakenly equated value of “transfers” with “savings in subsidy” in its 2016 study.
In the right to privacy case, the attorney general made two specific references to the World Bank report to argue against right to privacy, implying that it would impact the state from pursuing its economic and welfare goals of lifting it’s citizens from poverty.
Further Justice Chandrachud while ruling with other judges in the same case that privacy is indeed a fundamental right observed the following:
Data mining with the object of ensuring that resources are properly deployed to legitimate beneficiaries is a valid ground for the state to insist on the collection of authentic data. But, the data which the state has collected has to be utilised for legitimate purposes of the state and ought not to be utilised unauthorizedly for extraneous purposes.
The raison d’être for the Aadhaar project in welfare is that it allows the government to target the subsidies to legitimate beneficiaries.The study by the World Bank that Aadhaar can potentially save $11 billion every year is one of the most important intellectual scaffoldings on which such a claim stands as it was repeatedly cited by the government in multiple public forums and also in the Supreme court.
That the stakes have risen so high over a potentially murky claim is troubling and raises the following questions: If the World Bank had conducted independent research, why did it not make that clear in its 2016 report?
Secondly, if the research report’s referencing was incomplete, as it admits, why was it not corrected or an explanation put out after pointed out by Khera? Thirdly, if it erred in not making it clear in the first place, why does it refuse to make the study and its calculations public now?
Lastly, and perhaps more importantly, if the origin of the data is murky, why does the government cite it so often?
Anand Venkatanarayanan is a Senior Engineer at Netapp. Views expressed here are personal and do not reflect the views of his employer.
The author would like to thank Reetika Khera for her inputs on the World Bank savings claim, twitter user databaazi for this thread which formed the basis for this report and Sam Jawed for the original follow up questions to the World Bank.
Liked the story? We’re a non-profit. Make a donation and help pay for our journalism.
