BRIEF

Advancing the science of delivery: A proposed impact evaluation of inspection regimes in health care and their impact on patient safety standards

December 22, 2016



In Kenya, where clinics have limited budgets and patient safety standards are low, researchers are looking at how increased monitoring and providing information to patients about the clinics that serve them can improve the standard for patient safety. The Government of Kenya wants to develop better approaches for monitoring and improving health care throughout the country, and this evaluation will provide the government with evidence for policies to strengthen patient safety in private and public facilities.

Research area: Health

Country: Kenya 

Evaluation Sample: 1300 clinics 

Timeline: 2015 - 2019

Intervention: monitoring, health grades made, information campaigns

Researchers: The KePSIE team includes Guadalupe Bedoya (Principal Investigator), Jishnu Das (Principal Investigator), Jorge Coarasa and Ana Goicoechea (Operations Research), Amy Dolinger (Country Coordinator), Khama Rogo, Njeri Mwaura, and Frank Wafula (Implementation Team), supported by Benjamin Daniels, Seungmin Lee and Chenxi Yu from the World Bank Group.

Partners: The team works together with the Kenya Ministry of Health, the regulatory boards, and councils. 

 

Context

A growing body of experimental evidence points to the effectiveness of audits and monitoring (accompanied by rewards or sanctions) in improving the performance of service providers. In Kenya, where the government is working to expand health care access and improve quality, policymakers are interested in understanding what’s most effective and sustainable for strengthening patient safety standards in both public and private facilities.  This evaluation, implemented with the Ministry of Health, will pilot a program that has the potential to be scaled up within the government’s existing system, providing valuable evidence for future policymaking in Kenya.


Image
Photo © Dominic Chavez/World Bank

Intervention and evaluation

The project will operate in three Kenyan counties—Meru in the central region, Kakamega in the Western region, and Kilifi on the coast. These three counties were specifically chosen because they contain a combination of different types of health markets, ranging from stand-alone clinics to dense markets.  Across the three counties, there are around 1300 health facilities, representing nearly 10 percent of all health facilities in the country.

The facilities will be grouped into “health markets,” geographical clusters where facilities are in close proximity to each other. “Markets” of facilities will be divided into two treatment arms – each with around 430 health facilities divided into around 91 markets -- and a control group with the same number of markets and clinics. Both groups of facilities will be inspected according to a pre-determined checklist to determine patient safety standards. Researchers will then compare results with those from the control group, which will continue to receive the usual low-probability inspections as mandated by the country’s laws.

The first treatment arm will receive high-intensity inspections, with warnings and sanctions for non-compliant facilities. A health facility will be inspected with 100% probability more than once, and the inspection will be conducted in an electronic format by a group of trained inspectors working full time, and based locally. The inspections will have specific and standardized warnings and sanctions imposed on facilities according to their overall JHIC score.

The second treatment arm will receive high-intensity inspections with public disclosure of patient safety performance and aims to empower patients to make informed choices of health facilities based on their patient safety record. At the end of the inspection, each facility will be assigned a letter-grade -- provisionally A through D -- that will be prominently posted on the health facility door. Fraud control mechanisms will be put in place to ensure that clinics do not alter the information on the clinic door. The inspection results will be reported to the relevant boards and councils and they are able to apply warnings and sanctions to the clinics. There will be a dissemination campaign for patients to introduce them to the warnings and sanctions. At the same time, clinics will receive a detailed checklist and they can ask for a re-inspection if they want to change their grade.

All public and private health facilities in Kakamega, Kilifi, and Meru counties will be randomly assigned to one of the three groups.

The evaluation will specifically measure quality, patient safety, demand for services and prices for facilities across the three groups. This will include (1) Adherence to a checklist developed by the regulators and includes indicators of quality and patient safety related to protocols, infrastructure, and equipment; (2) Adherence to patient safety practices related to infection prevention and control; (3) Adherence to case-specific checklists of essential and recommended care for four medical cases, and (4) Prevalence of substandard medicines. Demand for health services will be measured through patient flow records from health facilities and administrative data, and prices will be measured through patient exit surveys. Measurement will include direct observations in the form of surveyors dressed as patients who are recording patient safety – for example, did the doctor wash his hands before seeing the patient -- and the accuracy of the doctors’ diagnosis and treatment given.

Results from varying facilities in “markets” will also be used to determine whether increased competition between facilities in the same geographic area affects patient safety standards. The evaluation team will also collect data on sub-groups both by facilities and patients, including gender.

Policy impact

The results will help the Kenyan Ministry of Health craft more effective health care policies that increase both accountability and competition to boost patient safety throughout the country. The evidence will also be useful to policy makers in other low income countries who are looking for innovative ways to provide higher quality health care to their citizens. Results will help health care practitioners develop a set of tools and instruments that can be broadly used in surveys and inspections across diverse low-income settings.