Synthesis of Immunization Service Experience Interventions and Monitoring and Measurement Approaches for Zero-dose and Under-immunized Children

"Immunisation SE [service experience] is a critical element of improving vaccine demand." 

This report shares the findings of a literature review that sought to identify existing interventions and tools for improving and measuring immunisation service experience (SE), particularly for zero-dose and under-immunised children in low-and-middle-income countries (LMICs). The literature review was carried out through the Vaccination Demand Hub Service Experience Workstream and is intended primarily for the United States (US) Centers for Disease Control and Prevention (CDC) and the members of the Vaccination Demand Hub, but also for global health researchers, practitioners, implementers, and donors who are interested in or working on vaccine demand challenges and on SE specifically.

Immunisation SE, described in more detail at Related Summaries, below, is defined by the Vaccination Demand Hub as "factors within and beyond the interactions between a health worker and an immunization client, which influence the experience and delivery of the immunization service. Ensuring positive SE is important for improving local vaccine demand and shoring up quality, equitable immunization services..." To advance the current understanding of SE, this review and synthesis of SE-relevant literature was carried out to answer two questions: (i) What SE-focused interventions exist that are focused on reaching zero-dose children and under-immunised children and have been used? (ii) What mechanisms or data collection tools exist to measure and monitor immunisation SE for zero-dose and under-immunised children at the country level (i.e., facility level up to national level)?

The review involved scientific, peer-reviewed and grey literature from selected organisations that work on global immunisation. It included 43 papers with 35 LMICs reflected. The literature was analysed using an SE framework that looks at the inter-related and cross-cutting elements at the individual, facility, community, and system levels that influence the immunisation service experience. These elements include, for example, community voice, input, and demand; healthworker empowerment; community actors and stakeholders; expectation and perception of SE; and outreach services. 

The following is a summary of the findings:

Key characteristics of the SE intervention literature
 

• The findings highlight various interventions that improve service experience, such as communication campaigns, community engagement, digital information systems, and pro-equity strategies. Most of the studies employ multiple, bundled interventions that focus on community engagement and health service delivery improvements in rural areas.
• Studies employ mostly quasi-experimental pre-post evaluation designs. Measurement approaches for interventions are not consistently described in this literature.
• Immunisation coverage features as the main outcome of emphasis across most of the studies, demonstrating marked improvements for most interventions. Outcomes around client knowledge/awareness of and satisfaction with immunisation, as well as service access, utilisation, and quality outcomes, are also prominent. However, the articles mostly do not report the relative effects of intervention components on coverage or other behavioural or service-related factors.
• Studies typically do not systematically disaggregate and describe specific effects on zero-dose and under-immunised children, though some studies do explicitly describe outcomes around the first and third doses of diphtheria, tetanus toxoid, and pertussis-containing vaccine (DPT 1 and DPT 3).
• The SE framework components with the most frequent representation in this literature are community voice, input and demand; community actors and stakeholders; integration of immunisation within services packages; and health worker empowerment. Interventions for health worker empowerment, including training on interpersonal communication, are prominent in this literature. Measures showing specific improvements in interpersonal communication (IPC) are not evident and hard to parse from general reporting on client satisfaction.
• There is a gap in attention to measuring important SE factors beyond health worker skills training, such as health worker attitudes, interpersonal dynamics, workplace culture, and work stress.
• Studies reviewed are mainly pilot interventions and do not report on efforts to scale or adapt interventions. Interventions using digital information systems appeared to be adaptations from previous experiences in other country settings with a technology provider.

Key characteristics of literature on mechanisms or tools to measure and monitor immunisation SE
 

• A broad set of options (tools, indicators, methods) for evidence gathering on SE is available, with evidence of use in pilot studies in countries at subnational levels. Their regular and systematic use for routine country monitoring and planning is not evident in this literature.
• Descriptions of tools and approaches emphasise monitoring activities but also indicate they would be suitable for quasi-experimental evaluations (e.g., pre-post or midline evaluations and for implementation research designs).
• The SE framework components with the most frequent representation in this literature are community voice, input, and demand; community actors and stakeholders; expectation and perception of SE; and quality of the interaction and services provided.
• Newly developed toolkits (e.g., Behavioural and Social Drivers Toolkit and Strengthening Immunisation Service Experience: Insight Gathering Tool) bundle together a range of SE relevant methods and metrics in a comprehensive way, which can be useful for SE-focused intervention studies. However, based on this review, it is apparent that they have not gained wide application in country studies to date. Toolkits that compile methods and metrics that are relevant to SE are available. However, standardised and validated SE indicators have yet to be formalised.

Based on the findings, the report makes the following recommendations: 
 

• Develop interventions that employ more diverse approaches to measuring and reporting results for SE, beyond coverage. Having greater diversity of evidence, including measurement of SE or quality of services, can help to better assess what works for reaching missed communities beyond typical or conventional approaches.
• Invest in adaptively scaling interventions for broader impact around improving SE among missed communities.
• Measure SE more broadly across health areas and in integrated services, beyond immunisation services, to obtain a more holistic view of the factors that influence positive and negative immunisation SE.
• Use participatory engagement and research to understand local meanings of SE, which could be instrumental for designing interventions and developing sets of SE indicators that are locally valued and validated.
• Beyond health worker IPC skills training, increase focus on measuring interpersonal/behavioural aspects of SE at the facility, particularly as it pertains to health worker dynamics, attitudes, and clinic culture.
• Balance the use of digital and analog engagement channels for reaching zero-dose populations, based on local context.
• Formalise measurement of SE (e.g., via a compendium of validated, standardised but adaptable indicators), drawing on existing service quality metrics and adaptive learning from participatory research and design activities in local settings and case studies.
• Advocate with decision makers at the country level to incorporate measures of SE into routine immunisation monitoring and continuous quality improvement methods.