Return to search

Assessment of bias, inter-rater reliability, and external validity in the use of mobile phone surveys for monitoring bed net coverage and use indicators in Tanzania

archives@tulane.edu / Introduction: Mass distribution of insecticide-treated nets (ITNs) is a core malaria prevention strategy that has proven to be efficacious and cost-effective in low- and middle-income countries (LMIC). Monitoring ITN coverage, use, and access has relied on household surveys which are expensive and time consuming. Recently, mobile phone survey (MPS) methodologies have emerged as a comparatively inexpensive alternative to large scale population-based household surveys and are becoming increasingly attractive considering the rapid growth trend of mobile phone ownership in LMIC. The overall research objective of the current body of work is to determine if interactive voice response (IVR) MPS can serve to rapidly and reliably monitor ITN indicators in LMIC.
Methods: Data collection used either household surveys or IVR MPS – all of which included a module on bed net ownership, access, and use. The first study aim analyzed data from the last five nationally representative household surveys conducted in Tanzania in order to assess and quantify the potential for bias as a result of using MPS over traditional household surveys in estimating bed net coverage indicators. The conceptual design compares surveyed households reporting mobile phone ownership, and thus the potential for participation in an MPS, against all other households regardless of mobile phone ownership over the course of a 10-year period. The second study aim was designed as an individual-level test of inter-rater reliability of bed net indicator estimates between a face-to-face household survey and a follow-up IVR MPS to these same households. The third study aim was designed as a population-level test of external validity comparing ITN coverage indicator results from a nationally representative random-digit dial (RDD) IVR MPS and the malaria module from a nationally representative household survey.
Results: Household mobile phone ownership increased by over 50 percentage points from 28.1% in 2007-08 to 81.5% in 2017. In more recent years, survey results show that bias in measuring ITN coverage indicators is minimal under a scenario that compares estimates calculated from DHS surveys for all households against those households reporting mobile phone ownership. For the four ITN coverage indicators assessed using the 2017 MIS data, national-level measures of bias did not exceed a 2.5-percentage point difference for mobile phone-owning households compared to the overall sample of households. Further, regional measures of bias for these same indicators rarely exceeded ± 3-percentage points in 2017. The second study aim, which compared bed net indicator estimates between the small-scale a household survey and a follow-up MPS, found that agreement between survey modalities was variable depending on the indicator, but was highest for household ownership of at least one bed net of any type (Gwet’s AC1 = 0.8). There was low agreement for indicators calculated from counts reflected in the low concurrent validity of key data elements used to calculate bed net use and access indicators. The third study aim comparing bed net indicator estimates from a national household and IVR RDD survey found that the external validity was variable but, in general, the RDD MPS tended to underestimate bed net indicators at the national level. Differences in bed net indicator estimates ranged from 3 to 23-percentage points but overall, it appeared that indicators non-specific to net treatment status demonstrated less bias in measurement through the RDD MPS when compared against the nationally representative household survey.
Conclusions: According to estimates, mobile phone ownership has increased drastically in Tanzania since 2007 suggesting that MPS could presently be used to track population-level indicators of ITN coverage, among others. The IVR MPS methodology we applied has the potential to serve as a mechanism that can accurately estimate certain bed net indicators – primarily those that would make use of data elements derived from binary response options. Their use could be scaled to much larger RDD surveys to collect discrete packets of information. At a total cost of approximately US$22,000 (2017 USD) to obtain nationally and regionally representative bed net indicator estimates, the cost-for-information benefit is promising, but more research needs to be done to optimize question sets in order to ensure RDD survey results are able to repeatedly track with face-to-face household survey results. / 1 / Matt Worges

  1. tulane:121734
Identiferoai:union.ndltd.org:TULANE/oai:http://digitallibrary.tulane.edu/:tulane_121734
Date January 2020
ContributorsWorges, Matt (author), Yukich, Joshua (Thesis advisor), Ashton, Ruth (Thesis advisor), Wisniewski, Janna (Thesis advisor), Hutchinson, Paul (Thesis advisor), School of Public Health & Tropical Medicine Tropical Medicine (Degree granting institution)
PublisherTulane University
Source SetsTulane University
LanguageEnglish
Detected LanguageEnglish
TypeText
Formatelectronic, pages:  183
RightsNo embargo, Copyright is in accordance with U.S. Copyright law.

Page generated in 0.0021 seconds