What is user feedback survey?
A user feedback survey is a simple tool where interviews are undertaken with people who have used public services such as health or education. It is premised on the understanding that citizens are the users of public services and can provide authentic feedback on their perceptions and experiences of quality, efficiency and adequacy of the services and the problems they face in their interactions with the service providers. The findings can inform actionable reforms to service delivery agencies to improve their services.
Where should a user feedback survey be conducted?
A user feedback survey can be conducted almost anywhere where a specific service, e.g. business registration, issuance of land user rights certificate or health services, is being delivered.
Four pilot surveys on seeking feedback and opinions of public service users, which have been supported by the World Bank in Binh Dinh, Thanh Hoa, Phu Tho and Vinh Phuc in 2014 and 2015,have surveyed the agencies at the district levels. The findings are thus relevant at that level, while aggregate comparisons across provinces cannot be made based on this data.
Why is user feedback survey important?
The basic message is that public services are delivered for the benefit of the people of Vietnam. Citizens have rights and entitlements to good services.
While service users may not be able to comment on the technical features and standards of the services such as the specifics of an health operation or the specific administrative procedures internal to the issuance of land user right certificates, they are well-placed to assess whether the service meets their needs, and whether the agency is perceived as responsive and reliable. The information generated through the surveys will help the service delivery agencies to improve services. For instance, if service users find that not information is available as regards the service e.g. opening hours or fees, it could put up a notice board informing about these issues or at the website of the People’s Committee
What is the process of the user feedback survey?
The below figure shows the steps in the process of undertaking a user feedback survey. Among other things, the survey relies on a very rigorous survey methodology where the respondents are sampled according to scientific criteria. To ensure the reliability of the interviews a certain amounts of the interviews are redone to check the quality.
How to ensure the quality and transparency of this process?
The quality of the survey is of paramount importance as described above. A training manual, based on good international survey practices and the particularities of Vietnam, was developed.
Transparency was ensured by informing every stakeholder about the specific steps of the survey in advance.
How can the findings from the user feedback survey help improve the quality of the service?
Each of the four survey reports included a standard matrix of recommendations. The action matrices set out each of the issues identified in the order in which they appeared in the provincial reports and pinpointed each surveyed location where the issue occurred. Further columns proposed solutions to each issue accompanied by their strengths and weaknesses. This proved helpful in condensing the mass of data and analysis into something that decision makers could deal with. In all four provinces the action matrix proved successful in directing decision makers to the things that could be done to improve the service, often at minimal cost.
The survey provided specific information to service agencies, for instance, 32% of surveyed respondents in Vinh Phuc wanted extended clinic times. For each and every province, an action matrix with recommended action(s) to address each shortcoming identified by the survey findings was prepared.
What are the findings?
As the survey objectives aimed at generating actionable findings at the agency level in particular, a summary of the findings across services, provinces and dimensions is illustrative only.
The four 2014/5 surveys produced a prodigious amount of data on every aspect of the selected services. This was analyzed and presented in four provincial reports in the Vietnamese language for use by the provinces concerned. Examples drawn from these reports can be found in the forthcoming synthesis report and highlight different aspects of the outcome of the surveys under the four condensed thematic areas used in the survey questionnaire: accessibility; responsiveness; cost of services; and feedback mechanisms.
One notable finding was the differences in perception established between aggregated data (at the Province level) and disaggregated data (at the District or service delivery point level). This confirmed the need for disaggregated data at the point of delivery level in order to clearly define practical problems that could be overcome.
For instance, in Binh Dinh with 11 district offices, only 7.7% of respondents were satisfied with the information kiosk; 5.5% with entertainment equipment; and barely 50% with the fans and air conditioners. However, on closer inspection District by District it quickly becomes apparent that it is likely that the only information kiosk is in the main city office and that seems to be largely inaccessible. Office waiting areas are generally satisfactory although only marginally so in two districts, possibly due to overcrowding. The provision of tables and chairs is satisfactory but noticeably less so in one district.
A second finding was the difference between the availability of facilities and the level of satisfaction of respondents with those facilities. Most notable is the dissatisfaction with rest rooms at all the locations, despite their availability. This would suggest that they were poorly maintained. In contrast, whilst only 41.3% of patients across the province said that drinking water was available, 72.8% of them were satisfied with this situation.
One of the most noticeable aspects of hospital services in Vietnam and in the survey outcome is overcrowding due to too great a demand against the level of services provided. This is most visible in the sharing of hospital beds. In Vinh Phuc 32.2% of patients had to share beds; in Thanh Hoa 13.6%; and 29.3% in Binh Dinh. Despite this, most patients were satisfied with their treatment and the hospital facilities. This is most probably a question of the current expectations of patients attending hospitals and is likely to change as expectations grow over time.
As is typical around the world, public service opening and closing times tend to follow normal business hours and are often inconvenient for working people. However, one example we examined was hospital out-patient service times. Currently, patients attend hospital only when needed and without any prior appointments system. The majority of respondents wanted the examinations to be available on Saturday, following by the preference of having earlier starting working hours.
The survey tested the attitude and quality of staffs. A very high level of 42.5% of respondents felt that doctors and nurses were only normal or were not dedicated. However, most respondents reported that doctors are generally more dedicated than nurses. However, nurses also have far more interaction with patients than doctors and are therefore more likely to receive such comments.
How can other provinces learn from those peers?
Initial capacity has been developed to undertake user feedback surveys. Much has been learned from undertaking the surveys that is of use to others in deciding what works and what does not. A peer learning mechanism through a pool of experts has been established through which cities and provinces with experiences provided technical support to others. The survey materials have been widely published and shared at various workshops and conferences, and study visits between provinces might be a way of learning from the peers.
All four provinces have allocated the funding of future surveys into their annual budget, thus ensuring the sustainability of the process for continuous improvement of service delivery. This has been achieved by ensuring the provinces’ ownership of the exercise, building local capacity and establishing a peer learning mechanism.
