Opinion poll and methodology
Transparency International Bangladesh (TIB) published the result of their survey on the activities of the members of parliament on October 14. They presented their finding in a press conference on the same day. According to their survey, 97% MPs are involved in "negative activities."
The prime minister questioned the motivation of
the survey taking place a year before parliamentary election. She also said that TIB had published the result of a similar survey in 2001, implying that the result
did not help her party then. She also said that TIB did not do any survey on the activities of her opponents. The speaker of the parliament said that the report was erroneous.
There was widespread coverage in the media and one daily published an article on the methodology of the TIB survey. One very experienced journalist praised the survey, saying that having felt one boiled rice you knew how the other boiled rice were in the pot (a comment which is totally opposite to the spirit of research). Every individual has his or her right to have an opinion. But to survey the opinion of many, proper methodology should be followed.
The methodology varies depending on what kind of research it is. Sociologists mostly use qualitative research, in which sample size is not important. Individuals or focused groups are interviewed. Detailed discussions take place on one's attitude. This type of study is subjective. TIB survey does not meet the criterion of qualitative study. This study is more like an opinion poll where survey method of research is followed.
A representative sample has to be taken to survey the opinions of a population. A few basic rules have to be followed. A population has many types of people, males and females, adults and elderly, working class and
middle-class, urban and rural and so on. Samples should be taken in such a way that everyone has a chance to be chosen. One type of choosing samples is "quota sampling." In this method the interviewer is asked to interview a certain number of a certain type of people, for example 9 men and 10 women, etc. Stratified samples are more scientific, in which subgroups are chosen in the same proportion as they are in the population
There is another method called "area sampling." The area is divided into smaller areas. For example, a town may be regarded as an area, and is divided into wards. In this method, specific persons are interviewed. The problem here is that an alternative person will not be accepted. If a person is not at home the interviewer has to go back to that house, which may be on the other side of the town. The methods of choosing the samples have their own advantages and disadvantages.
In developed countries telephone directories or jury's lists are used to choose the samples, which is not possible in our country. Researchers should explain how the sample was chosen. Unfortunately, that was not done in the TIB research. It was mentioned that 46 meetings were held in 40 districts in which 600 people participated. They were teachers, lawyers, journalists and other professionals. It was claimed that these people were neutral, educated and well-informed. The question is, among so many teachers how were those particular teacher selected? The same question may be asked about the others .Who are "other professionals?" It was not mentioned how many of which profession were chosen. Some people may think that TIB had something against the working class.
The next question is, what was asked or what was discussed? The answers given by the interviewees may vary depending on how the questions are asked. Answers may also vary on the order of the questions. There was no mention of any questionnaire. To prepare a questionnaire is not easy. Everybody must understand the questions. Words must be chosen carefully so that there is no ambiguity. A question, to be meaningful, must relate to existing opinions and attitudes on the part of the subject. The effect of the context has to be considered in framing a questionnaire. The question of explicit and implicit statement of alternatives has to be considered as well. Pre-testing the questionnaire helps prepare a good one.
We have no idea how the discussions progressed in the 46 TIB sponsored meetings. The general impression is that in any meeting a few people talk most, most people hardly talk and others in-between. The views of the most talkative ones should not be taken as the views of all the people present in the meeting.
The characteristics of the interviewer are also important. The interviewees knew that the meetings were conducted by the TIB. It has been seen in other studies that some interviewees consciously or unconsciously go along with the interviewers.
The next question is how were the interviewees' responses interpreted? The survey report has given a list of activities which were described as "negative activities." Some of them can be described as serious offences, for example murder. Some activities are not serious offences. Some activities are immoral but not offences in legal sense. It is not mentioned how many people reported about the activities of how many MPs and how many "negative activities." There is a possibility that a few people reported about many MPs and mentioned too many "negative activities." They were then added and the interviewer reported that they were given the information by all the interviewees.
We may not agree with the finding of a survey but there is no place for emotion. We must know why we disagree. A little knowledge of methodology may help us. The disagreement may not be because of motivation but may be due to the methodology. It was good to see the government asking for an explanation of the
methodology and a minister making comments on methodology in a press conference. The methodology of opinion poll has tremendously improved since
the days of "straw polls." We must remember that there is no such thing as "absolute truth." As Karl Popper said: "Identifying the truth is actually achieved by
moving further away from error, rather than
discovering truth."