by Rossa O’Keeffe-O’Donovan, Economics PhD Candidate, University of Pennsylvania.
Which factors predict the functionality of hand pumps? Do communities free ride on their neighbors’ water sources? Are there positive spillover effects in the maintenance of nearby pumps? And what does this all mean for practitioners? This post gives an overview of my ongoing Economics PhD research, which tries to answer these questions.
Note: this research is still in progress, and I am seeking survey responses to complement my quantitative work, and help understand and interpret my results. If you have knowledge of how decisions are made in the installation and/or maintenance of hand pumps, please take this 8 minute survey here: bit.ly/PumpSurvey
Economics and rural water supply
My first job after graduating was conducting Monitoring and Evaluation for a medium-sized international water NGO in Malawi, where I saw first hand the large number of non-functional hand pumps. I started thinking about the economic factors affecting pump functionality, and how I might use economic tools to better understand breakdown.
Where did the data come from?
I started out with a statistical exercise, looking at which community or pump characteristics predict whether a pump is functional or not. Step one was finding data to analyze – Joseph Pearce (then at WaterAid, now at IRC), was very helpful in this regard, as was Brian Banks at GETF, and the Water Point Data Exchange (WPDx) has been incredibly useful in helping to locate data as I have updated and expanded the research.
Although I was most familiar with the Malawian context, I started using Tanzanian data, as it had good detail on pump and community characteristics. The data can be seen in the map below, where different colors represent different types of water source. My analysis focuses on 10,747 hand pumps in the data, 63% of which were recorded as functional (producing water) at the time of data collection.
Map downloaded from mWater on February 6, 2016
Predictors of pump functionality
The results of this statistical exercise build on the excellent research by Tim Foster which uses data from Liberia, Sierra Leone and Uganda, a great summary of which can be found here. I find that the following factors are significant predictors of pump functionality (note that this does not mean that there is a causal relationship):
- Older pumps are less likely to be functional (as in Foster 2013)
- Pumps of a less common technology (i.e. those that are not an Afridev, India Mark II or III, SWN 80 or Nira/Tanira) were less likely to be functional (as in Foster 2013)
- Whether a pump is installed on a hand dug or machine drilled well does not predict its functionality (slightly different to Foster 2013)
- Pumps further from the capital city are less likely to be functional (similar to Foster 2013), though distance to other cities did not predict functionality (different to Foster 2013)
- Pumps with user fees are more likely to be functional, though it does not matter whether the fees are paid annually, monthly, per bucket or upon failure (as in Foster 2013)
- Pumps where the water is (subjectively) perceived to be of good quality are more likely to be functional (as in Foster 2013)
- Pumps that are managed by a parastatal organization were more likely to be functional, though those that were privately managed were not (similar to Foster 2013)
- Pumps in wards with high population density are more likely to be functional, as are pumps with more users (slightly different finding to Foster 2013)
- Pumps in wards with greater diversity of nationalities are less likely to be functional, though there was no effect of religious diversity (results similar to Miguel and Gugerty, (2005) in Kenya and other economics literature)
- Of the 16 largest installing organizations (accounting for 60% of the pumps in my data), three had functionality rates significantly lower than those installed by other organizations.
- The following factors are not significant predictors of pump functionality: the organization that funded the installation (different to Foster 2013), whether a community had a public meeting upon installation of the pump (similar to Foster 2013), the height above sea level, the geographic size of a ward, the population of a ward, the gender ratio of a ward, the month in which data was collected
Do neighbors’ pumps matter?
I also conducted some spatial analysis: I looked at whether the number, type and technology of nearby water sources help predict whether a pump will be functional or not and found some interesting results:
- Pumps are more likely to be functional if the distance to an alternative working water source is smaller, and even more likely to be functional if this water source is another pump of the same technology (e.g. both are India Mark II pumps)
- The number of water sources within a certain distance also predicts pump functionality. A pump is more likely to work if there are more pumps of the same technology nearby, but less likely to work if there are more pumps of a different technology nearby. The number of non-pump water sources does not predict pump functionality.
There are a number of potential explanations for these spatial correlations in pump functionality, and I explore each of these in my research. I think the most convincing explanation is that there are positive spillovers in the maintenance of very similar water sources – i.e. it is easier to maintain a pump if there are many similar pumps nearby. These spillovers might be a result of: increased availability of spare parts and pump mechanics familiar with the technology, explicit cost sharing between communities with similar pumps, the development of skills, or sharing of information between communities. Such spillovers would explain why having pumps of the same technology nearby increases the probability that a pump will be functional, but having pumps of a different technology, or non-pump water sources does not.
There is also some (weaker) evidence for free riding, which occurs if a community is less willing to pay for their pump’s maintenance when there are other working water sources nearby, because if their pump breaks down there is an alternative source available. The main evidence for this is the fact that a pump is less likely to be functional if there are more pumps of a different technology nearby.
Using an economic model to explain functionality
Positive spillovers and free riding effects are opposing forces: having other pumps nearby can increase both, and we only observe the ‘net effect’ in the data. Therefore, to further understand these effects, I developed an economic ‘network’ model in which communities’ decisions depend on the decisions of their neighbors. This allows me to disentangle the two effects, measure their magnitude, and test the extent to which they depend on different community and pump characteristics. For example, my initial results suggest that the strength of positive spillovers depends on the distance between pumps, and whether they are of the same technology; free riding seems to depend more on whether an alternative pump charges user fees than the distance a community has to travel to use it.
What would happen if we standardized pump technology?
Estimating this model also allows me to perform ‘counterfactual analysis’ – i.e. estimate how outcomes would change if we changed some of the model inputs. Focusing on a subset of 4116 pumps with the richest data, I estimate that if pump technology was standardized to fully exploit positive spillovers in their maintenance, the functionality rate would increase from 67% to 74%. I caution against interpreting this preliminary as a policy prescription, as there may be benefits to having different technologies: for example, perhaps some technologies are easier to install in different terrains, or are resistant to different types of physical shocks. However, I think this research gives a good estimate of the cost of a lack of coordination between different installing organizations, and the resulting fragmentation of pump technologies.
Future work: health, education, dependency and other countries
I am currently using the model to estimate the effect of non-functionality of pumps on health and education outcomes, using Tanzanian census data. In future work I hope to explore the dynamics of pump maintenance decisions: whether communities’ maintenance decisions depend on their past decisions, their neighbors’ past decisions, or whether they respond to the installation of new pumps. If communities respond to new installations, and form expectations about when a new pump will be installed in the future, this might induce ‘dependency’, with communities preferring to wait for a new pump to be installed rather than repairing their older pump. A dynamic analysis will allow me to explore such effects. Finally, I can apply this approach to data from other countries.
Your input is very valuable!
This is ongoing work, and I am still refining my model so my estimates are subject to change – you can find the latest draft of my paper at my website. I am also working to better understand my results, which is where drawing on the knowledge and experience of a wide variety of stakeholders is incredibly valuable. If you have experience in the sector, please take my survey and share it with others who might have useful insights! And please get in touch if you have any comments, suggestions or feedback on this research – hearing from practitioners is extremely valuable, and I hope that this research can be useful for decision-making in the sector!
Hi Rossa, this is very interesting research you are doing, and you did a good job in making it easy to understand you preliminary conclusion. Not knowing the details of your work and just based on your contribution, it seems suprising to me that even if we fully standardized pumps in an area, functionality would “only” go up from 67% to 74% – so there seems to be a lot of other factors which make pumps a technology which overall does not function very well. Did you look into other factors as well?
Best regards,
Matthias
Hi Matthias,
Thanks for the feedback, and it’s interesting to hear your intuition about whether an increase from 67% to 74% functionality is large. My prior is that this is actually quite a large effect, though it depends on the framing of the stats: it’s essentially estimating that 7 out of every 33 broken pumps (21%) would be working if we standardized technologies. This is only a preliminary finding though, as you note, and I might get a different estimate when I finalize this work.
Your other point is important: there are indeed many other factors that contribute towards pump breakdown. Some of these are observable in the data I use (see the list of predictors of pump breakdown in the blog, though these should not be interpreted as causal relationships), whereas others do not appear in the data (e.g. how good the maintenance training was, other community characteristics (wealth, education etc)). I am currently working on merging the pumps data with World Bank surveys to see if community characteristics in the World Bank surveys predict functionality, but this will likely greatly reduce my sample size, so I’m not sure I’ll find much that is statistically significant.
One interpretation of my findings so far might be something along the lines of: there is evidence that ‘fragmentation’ of pump technologies results in a lower functionality rate, but this only explains about 20% of the breakdowns. Even if we standardized technologies, pumps would still break down because of lack of community resources, poor installations, droughts, poor community engagement etc.
Best,
Rossa
Hi Matthias,
Thanks for the feedback, and it’s interesting to hear your intuition about whether an increase from 67% to 74% functionality is large. My prior is that this is actually quite a large effect, though it depends on the framing of the stats: it’s essentially estimating that 7 out of every 33 broken pumps (21%) would be working if we standardized technologies. This is only a preliminary finding though, as you note, and I might get a different estimate when I finalize this work.
Your other point is important: there are indeed many other factors that contribute towards pump breakdown. Some of these are observable in the data I use (see the list of predictors of pump breakdown in the blog, though these should not be interpreted as causal relationships), whereas others do not appear in the data (e.g. how good the maintenance training was, other community characteristics (wealth, education etc)). I am currently working on merging the pumps data with World Bank surveys to see if community characteristics in the World Bank surveys predict functionality, but this will likely greatly reduce my sample size, so I’m not sure I’ll find much that is statistically significant.
One interpretation of my findings so far might be something along the lines of: there is evidence that ‘fragmentation’ of pump technologies results in a lower functionality rate, but this only explains about 20% of the breakdowns. Even if we standardized technologies, pumps would still break down because of lack of community resources, poor installations, droughts, poor community engagement etc.
Best,
Rossa