1. Inspired by a private sector practice of conducting client satisfaction surveys, a small group of people in Bangalore , concerned about the city' deteriorating standards of public services , initiated an exercise in 1993 to collect feedback from users. User perceptions on the quality, efficiency, and adequacy of the various services were aggregated to create a 'report card' that rated the performance of all major service providers in the city. The findings presented a quantitative measure of satisfaction and perceived levels of corruption, which, following coverage in the media, not only mobilized citizen and government support for reform, but also prompted the rated agencies themselves to respond positively to civic calls for improvement in services. This exercise was repeated in 1999, and has been replicated in at least five other Indian cities, as well as the State of Karnataka in the interim. By systematically gathering and disseminating public feedback, report cards may serve as a "surrogate for competition" for monopolies - usually government owned - that lack the incentive to be as responsive as the private enterprises to their client's needs. They are a useful medium through which citizens can credibly and collectively 'signal' to agencies about their performance and pressure for change.
Process
2. Supported by a small advisory group of local leaders and funded through local donations, Dr. Samuel Paul with the help of a market research agency called Marketing and Business Associates (MBA) Ltd. first planned the initiative in 1993 to seek answers to three main questions, i) how satisfactory are the public services, ii) what aspects of the service are satisfactory and what are not, iii) what are the direct and indirect costs of acquiring these services? This informal exercise has since been institutionalized as one of the core functions of the Public Affairs Center, a non-profit society established in 1994 in Bangalore, with a goal of "improving governance in India by strengthening civil society institutions in their interactions with the state" . Principally, PAC has focused its past six years on combining research with action, by supporting networking and capacity building of citizen initiatives to respond to knowledge revealed through systematic research on public policies and services.
3. Specifically on the 1993 initiative, after undertaking mini case-studies on some key urban problems, focus group discussions were held with two different sets of households - middle income (called general) and lower income (largely slum dwellers) to draft and finalize respective questionnaires. These were then pre-tested. The city was divided into six regions, and from each region, a random sample of households that had interacted with at least one of the service providers in the past six months was chosen. Around 480 households were drawn from a pool of middle and upper income groups and around 330 households representing the slum dwellers. Selected interviewees (men to women ratio was 7:3) were then asked by trained investigators to rate their level of satisfaction with a particular service provider's overall performance as well as allied dimensions such as, i) staff behavior, ii) number of visits required to complete a task, and iii) frequency of problem resolution. People were asked to assign a rank from a scale of 1 to 7 (very dissatisfied to very satisfied). These were then aggregated to compute averages for overall perception of the quality of service. The extreme scores indicated what percentage of the people were either completely dissatisfied or completely satisfied with particular service providers. The list of the agencies had not been pre-determined, that is, people were not asked what they thought of agency A, but were rather questioned about whichever services they had availed of in the past 6 months. Overall, the methodology employed was what has been described as a merger of statistically valid techniques with qualitative participant observation . When the exercise was repeated in 1999, there was some attempt to use the 1994 results as a benchmark by using similar scales for some questions, although sample size was increased this time to 1339 from the middle income category and 839 from the slums using a 'multi-stage stratified sampling plan'. The way respondents were selected was quite elaborate too. A list of all polling booths for local assembly elections in 14 constituencies of the city was obtained from which 90 were randomly selected to serve as 'starting points' for investigators to go to and ask between 5 and 10 people about the services that they have encountered. Similarly for the urban poor, 80 starting points were identified from four different categories of slums . The collection of responses was then entered into a computer and analyzed using a software package.
4. On dissemination, unlike in 1994, when all key findings were flashed through the media, in 1999, the Public Affairs Center decided to first present mini report cards to four of the key service providers (telephone, water, electricity and the municipality) to solicit their initial reaction. In the interactions, the agencies did not dispute the findings at all, and instead felt the pressure to be defensive about their performance by presenting constraints they had to work with. After these selective meetings, the 1999 report was circulated to all public agencies and senior state government officials. This was followed by a launch ceremony for the press - a crucial ally in the process. After letting the findings sink in through a heavy media coverage, a two-part workshop was organized involving senior officials from the agencies and the public. The first part allowed the agency officials to interact and learn from each other on what some of the more responsive agencies were doing to best address the criticisms. The second part involved the head of the agencies answering questions from assembled citizens on what steps were being proposed to improve the quality, efficiency and adequacy of their services.
Findings
5. Over the 5 years between the first and the second report card initiatives, partial improvement in services such as the telephones and the hospitals were noted . However, overall citizen satisfaction remained low (with even the better performers scoring less than 50% satisfaction rating). People seemed even less satisfied with the way staff interacted with the clients. Bangalore Telecoms, for instance, had the highest overall satisfaction rating of 67%, but this dropped sharply to only 30% among a sub-sample of people who interfaced with the agency personnel to solve a specific problem. The scale of corruption was perceived to have grown with both the number of people paying bribes and the amount they were paying increasing. 92% of the respondents said they visited the agencies in person to solve a problem and two-thirds of the time, they needed to make two or more such visits. Over half the cases involving bribes were extortionate in nature, while a third had been voluntary 'speed' payments.
6. A separate report card on the slum dwellers in 1999 also found that they were more happy with services such as transport, schools, electricity and hospitals than with the police, water supply and garbage clearance. The urban poor had to visit the agencies more often than the middle class to solve a similar problem, and had lower problem resolution rates, although, probably because of low expectations, they seemed more satisfied with the same services than people with higher incomes. The proportion of people who were made to pay a bribe in order to access the services was also higher for the poor than the middle-income group: in 1993, a third of the urban poor (surveyed) said they had paid a bribe. While this ratio dropped to 25% five years later, average amount paid had increased three-fold to Rs. 1245 per case, largely to the police.
7. The report cards indicate that there is a clear link between petty corruption and inefficiency of service providers that have non-transparent procedures, and arbitrary decision making powers vested in officials. A better access to information, clear specification of service standards and customer rights, and sustained public scrutiny and monitoring of performance through institutionalized report card initiatives has been suggested to be helpful in curbing both the vices by shaming the 'baddies' and creating public awareness and building up pressure.
Results
8. The report cards have forced the hitherto apathetic public agencies to, at least, listen and react to citizen concerns. While anecdotal evidences on the incompetence of public agencies have always existed in plenty, quantification of perceptions has brought with it a credible indicator that lays down the extent of (dis)satisfaction and allows inter-agency comparison, triggering internal reforms. Although no dramatic improvement in quality of service was witnessed between 1994 and 1999, of the 8 agencies covered in the report card in 1993, four did make attempts to respond to public dis-satisfaction. The worst rated agency - the Bangalore Development Authority (BDA) - reviewed its internal systems for service delivery, introduced training for junior staff, and along with the Bangalore Municipal Corporation, began to host a joint forum of NGOs and public agencies to consult on solving high-priority problems such as waste management. The Karnataka Electricity Board, too, formalized periodic dialogues with resident associations to garner feedback from users. Two others tried to strengthen their grievance redressal systems. However, the remaining four stayed indifferent to the results demonstrating how public outcry alone is not adequately strong to overcome bureaucratic inertia. It is quite clear that report card initiatives will need to be complemented by politically-driven systemic reforms from the inside for optimal results.
9. Public awareness on issues of quality service delivery and corruption has substantially heightened following heavy coverage of the report findings in the mainstream media. In 1994, the country's premier daily, The Times of India, for instance, ran a weekly feature for two months focusing on one interesting finding at a time. Civil society groups are increasingly realizing that their collective pressure has enough power to usher in change. Instead of remaining passive individual receivers of inefficient services, the report card initiative has, for example, inspired the undertaking of a unique state-citizen Swabhimana Initiative in Bangalore to improve quality of city life through innovative solutions to old problems. By the time the second report card was released, the new Chief Minister of Karnataka formed the 'Bangalore Agenda Task Force' consisting of prominent city residents to come up with suggestions to improve the city's quality of services and infrastructure, and thus, 'rejuvenate' the city. This was the first time a chief minister was responding to persistent citizen demands, expressed through the report cards, the media and the NGOs, to make service providers perform better (more efficiency, less corruption) and be more accountable (through citizen charters). One immediate result of this initiative has been the introduction of a system to self-assess property taxes. This has brought in transparency, speed and simplicity to an otherwise corrupt and arbitrary process .
10. The fact that the report card exercise has been replicated not only in other Indian cities, but also internationally in countries like the Philippines and Ukraine, and cities like Washington D.C., endorses the usefulness of the 'model' in exacting accountability from public enterprises. Inspired by the pioneering initiative in its own capital, the State of Karnataka undertook a similar exercise in 1999 with the help of the Public Affairs Center. Feedback on five services - drinking water, education, health, food distribution and road transport, as well as key policy issues was collected from 4250 respondents (3600 rural, 650 urban, in addition to 583 independent) in six different districts. Some findings were surprising enough to be useful for future policy making. Investment in infrastructure was appreciated the most: 87% availed of the public water systems (but half of them could only tap for less than 2 hours), 75% had access to road transport, over 80% of the students attended government schools. Only 20% were happy with the quality of ration food. 65% in rural areas had no access to a local medical facility; 80% of the schools had no toilets. Most interestingly and on a different plane, 50% also said they didn't know that everyone was equal before the law and discrimination was punishable, and a tenth said they had been denied the right to vote in elections. In addition to a survey, the Karnataka exercise also introduced the idea of 'spot checks' to validate accuracy of data collection from people . That a model tested at a municipal level was adapted for use at the state level shows the versatile applicability of the idea of report cards. However, the legitimacy of the findings vis-Evis the domain of exercise is important - one especially needs to watch out for caveats such as using nation-wide data to infer solutions for local problems, or vice versa.
Concerns
11. Unlike opinion polls, report cards necessarily present views of people on issues that have actually been experienced first-hand by the interviewee. This is a useful merit, but because people's expectations of the standards of service vary across regions of residence (urban, rural), income groups, and education levels, among others, aggregation of 'subjective' satisfaction ratings of different people to compute an 'objective' indicator of a quality of service is theoretically problematic . At a practical level though, if a sample is dis-aggregated well to reflect fairly homogenous socio-economic characteristics such as income levels, this methodological flaw can be minimized, and useful insights can still be extracted on the general perception of a group of 'similar' people. Report cards to date have also concentrated more on the quality of service offered, and not so much on whether people actually have access to a service in the first place. The scope of inquiry thus ought to be broadened, especially if the survey covers rural areas where issues surrounding access remain a big problem.
12. Although report cards are a seemingly uncomplicated method of channeling citizen feedback, they require a competent intermediary that is technically versed to pilot and administer a survey methodology, like PAC. By being dependent on intermediaries with sophisticated skills, the process, by extension, requires adequate financing, time commitment, and interest on the part of local residents to sustain the exercise from formulation through monitoring year on year, in addition to a conducive socio-political climate on site (in non-democratic countries without a vocal free press and civic activism, this exercise will need to be done differently). These are all demanding requirements, and simpler ways will have to be sought if 'indigenization' of the exercise in developing countries means making the exercise cost-effective and easily manageable. This, however, entails tradeoffs. The benefit of making the process manageable by limiting the exercise through a small sample size, for instance, must be weighed against costs of non-representativeness and inapplicability of findings on a broader plane (e.g., it is difficult to make the sample size of 2000 representative of the 5 million residents in Bangalore). Relaxing of difficult constraints can thus compromise the credibility of the report, making it easier for the government to ignore the findings at the time of dissemination.
13. It seems that one of the biggest challenges for initiatives like the report cards is to ensure an operational link between information and action. Unless implementation is not followed through through a sustained movement of action and research, probably along the lines of the PAC model, report card initiatives, especially those that arrive as one-off experiments will serve little long-term purpose. How these efforts are to be institutionalized at non-governmental levels, possibly involving a coalition of grassroots, research, advocacy and media organizations, should thus be a concern warranting some thought right from the outset.
Annex: The PAC Model on Report Cards
Phase I: Planning and Preparation
Phase II: Statistical and Analytical Groundwork
Phase III: Dissemination
Phase IV: Advocacy and Monitoring
Source:
Samuel Paul, "Case Study on Report Cards on Public Services in Bangalore" Bangalore: Public Affairs Center, 2001
