Assessing Voter Needs

Educators have to anticipate and understand the needs of those for whom they are designing programmes. There are a number of different ways of assessing needs, and those planning an education programme will want to consider these on the basis of:

• available time;
• complexity of the programme and its geographic spread;
• whether the programme is being implemented for the first time or whether there is some institutional knowledge with respect to reaching the target constituency; and
• available finances.

Mainly, educators will want to secure professional assistance. This section provides the overview that they might need to determine what professional assistance will be required and how to manage that assistance. "Surveys" describes the most complex and most likely activities that an education team will commission out. But it is possible to use "Existing Data", and to make use of the education team itself to obtain information through "Interlocutors and Intermediaries".

Surveys

In order to develop effective programmes, voter educators have to know about voters or potential voters. They need to know what people know (or think they know). They also need to know how people feel about elections and voting. And finally, they need to know what is likely to encourage them or inhibit them from voting.

How can this information be obtained? One could look into a crystal ball. One could guess. It could be useful to look at past experiences. Talking to others involved in the field would be a good idea. What about simply asking voters, or potential voters, themselves? But it would be impossible to speak to everyone. So how can one be assured that the information obtained from a limited number of people will be worth anything? Will they be representative of all potential voters?

Using Surveys

Surveys are useful when it is necessary to learn about the attitudes, values, motivations, predispositions, and likely behaviours of large numbers of people. Surveys are also useful when the results need to be generalized to some larger population.

Focus Groups

With very small numbers of people - say, a few dozen - it is often better to use more qualitative, in-depth forms of research such as focus groups that allow people to speak at length about their feelings. Then, one can simply review the transcripts of the group discussions to learn their thoughts and attitudes. Focus groups often provide great insight into a topic. The richness of such transcripts is very difficult to quantify, but with a small number of cases, quantification is often pointless.

It is important to realize, however, that focus groups are not simply any meeting or unstructured conversation. Focus groups employ a specific methodology with respect to the selection of people to be in the groups. Groups should be as homogenous as possible, and the groups should be structured to reflect the key differences of potential interest. One group might include all young male first-time voters, for example, and another might include all young female first-time voters. Insight comes from both the carefully facilitated conversations in each group, as well as the differences between the groups. The insights generated from focus groups are often useful in order to identify key issues to be addressed in a larger quantitative survey, or to investigate questions unearthed by such surveys in more depth.

Designing Surveys

This discussion is intended to aid two types of people. First, many people might want to undertake a survey on their own and thus might find this a useful guide, or blueprint to all the key steps that they will have to work through. It would not, however, provide a sufficient "users manual" to take you through each step in depth.

Some detail has been included in these sections so that a person interested in the subject will have a complete guide to what may be one of the most expensive parts of a national voter education programme. Other readers may wish to ignore the technical information that follows.

The various dimensions associated with a survey are often beyond the capabilities and resources of any given individual or organization. Thus, many organizations would more likely want to hire a professional research firm experienced with surveys to do this work for them. At no point, however, should control of the process be relinquished. This description is intended to enable election authorities to maintain critical control in monitoring the project.

Thinking your way through a successful survey consists of a series of steps:

• Survey Design
  

• Question Framing
• Survey Preparation
• Sampling
• Selecting Participants
• Getting Into the Field

Survey Design

Deciding What Information is Needed

First a decision will need to be made about what information needs to be collected and why. This process can be initiated by asking a few simple questions: "What opinions or likely behaviours do you want to know about?" In social science jargon this is called the dependent variable. "What do you think are the causes?" These are the independent variables. The answers will provide some important foundations for the survey's content.

Suppose one wanted to know about the causes of voter turnout, or the dependent variable. A decision is made to test the varying impacts of potential causal factors such as:

• information
• motivation
• interest
• efficacy
• perceptions of the degree of electoral competition among parties in a given election

These will be the independent variables. Basically, then, five key factors have been identified that are important and need to measured. All that is left is to define each of these concepts, or factors, so that there is agreement about what is meant by such terms as "electoral competition" or "motivation."

A "conceptual framework", therefore, is created that should act as the blueprint for the entire project. At any point in the project, one should be able to guage whether what is being done is helping to measure an element identified in this framework. If it is not, one may discover that he or she has gone off track (which is easy to do), and is working on something peripheral to his or her real interests.

At the same time, while writing survey questions, it may become apparent that there are some really important things that one needs to know about, but that have not been included in the blueprint. At that point, one should not simply write a new question in an ad hoc fashion, but should go back and put the new concept into the blueprint.

Conceptualization is usually based on:

• your knowledge of the local context;
• reviewing what is known and published about the subject, e.g. voter turnout;
• consultation with experts in the field.

Before Idasa's Public Opinion Service (POS) conducted a survey on the Cape Flats about public views toward crime, policing and collective action, for example, it called in a range of criminlinologists, sociologists, social workers, and journalists with extensive experience on the ground as well as with the relevant academic literature. This helped in the identification of the key conceptual areas, and thus, the parameters of the questionnaire.

Operationalization

At this stage, the goal is to begin formulating a structured questionnaire by designing specific questions to measure the "real world" existence of the phenomena or attitude in the conceptual framework. In other words, the conceptual framework is being converted into an actual questionnaire.

Ideally, several questions should be designed to measure each key concept. One single question can often be an unreliable indicator of people's attitudes in that area. The ultimate goal is to be able to average the responses to all the questions about a concept to provide a valid and reliable aggregate measure or index of the concept (such as "interest"). The series of questions should not simply measure the same exact thing, but tap various dimensions or elements of "political interest".

A valid question, or series of questions, is one that actually measures what is meant by efficacy. One form of validity is called "Face Validity". That is, by reading the actual question wording, the question wording appears to be getting at what the intent. Another form is called "Construct Validity". This is when responses to the question, or series of questions, seem to correlate internally with one another, or with other questions that measure things that one would expect to be related to political interest.

"Reliability" refers to the extent that the questions would yield the same responses from one sample to the next, at any given point in time. Various types of statistical tests exist to help assess the extent of construct validity and reliability.

"Operationalization" is probably the most time consuming aspect of the survey process. Converting concepts into valid, reliable questions that measure exactly what they are intended to measure requires much thought and careful phrasing.

Framing Questions to Maximize Survey Data

In general, questions should measure precisely what they are intended to answer. They should be as clear and accessible as possible, especially in places with low levels of education and literacy. Wording and phrasing, therefore, are crucial.

A useful guide is to look at the work of respected analysts in an area, such as voter turnout, and to examine the types of questions they are asking. What do other people internationally ask in order to measure efficacy, or likely voter turnout?

Asking questions that have been asked elsewhere also enables one to compare one's results with what has been found in other places and at other times. This is crucial. Any given result from a survey - such as that 34% of South Africans are not interested in politics - may mean something on its own. On the other hand, it may mean more if we knew that it was much higher, much lower, or about the same as people in other countries.

At the same time, a particular context might demand a uniquely worded question. Thus, there is a fine line between developing questions that are meaningful in a specific context, and producing results whose comparability helps enhance one's understanding of local dynamics.
Beyond these general comments, there are a number of potential pitfalls the question designer should understand.

Open-ended Questions

These questions allow the respondent to answer spontaneously, on their own terms. Rather than asking people to rate the importance of several possible reasons for voting on a scale of "very important" to "not important at all"' for example, one might ask them: "What are the most important reasons to vote?" Then, they have the advantage of not presuming what needs to be proven.

Open-ended questions, however, are very expensive. Most survey companies will allow for only three to four in their normal quotations. A typical question, such as "What are the most important problems facing the country?", may get dozens of unique responses. All of them have to be examined and categorized or "coded" into broader categories that are considered useful. This is a very time consuming process and drives up labour costs considerably.

Closed-ended question, those that have a closed set of responses from which people may choose, also present a range of potential problems that are discussed in greater detail below.

Framing

Framing refers to how important issues are presented or "framed" in a survey question. Which aspects of a larger issue should be tapped? Which set of policy alternatives should be offered to respondents? Should a question on the location of a parliament tap costs and efficiency? Should South Africa have one administrative capital in Pretoria, for example, and one legislative one in Cape Town? Or should they be put together in one city? Or, should the question ask about changing the status quo? Again, as an example, should parliament be kept in Cape Town , where it presently is, or should it be moved to Pretoria or somewhere else? These question may yield quite different results, with very different political implications.

While different frames can yield significantly different results, decisions about which frames to use are almost impossible to resolve and will almost always invite criticism from some part of the political spectrum.

Question Order

Question order can shape responses by altering the larger context in which respondents think about an issue. Because answers to one question can be shaped by answers to previous ones, questions that are themselves fairly unbiased may create a very different effect when asked in combination. Questions on likely voter turnout, for example, may be biased in favour of higher potential participation if those questions were preceded by items asking people about people's duty to vote, thus reminding them of that duty.

Response Order

The order in which possible responses are listed may also have important effects on results. When extreme response items are placed before a more moderate response - the "contrast" effect - the preceding extreme responses increase the likelihood of choosing the following, more moderate response.

Order effects also differ according to interview method. With phone surveys or personal interviews that are read out, there is a "recency effect" where respondents tend to choose latter options from a list because they have more time to think about them. In contrast, visual presentations like showcards or mail questionnaires may have a "priming effect" where earlier alternatives tend to be chosen because people are more likely to think about the first alternative.

One-sided vs. Forced-choice Questions

"One-sided" questions ask people to agree or disagree with a statement, to favour or oppose some position, or to state some degree of an opinion. With "forced-choice" questions, the researcher attempts to provide balanced alternatives, such as, "Do you favour the government doing X policy, or should it pursue Y policy?".

"Agree" or "disagree" response sets tend to bias results in favour of the "agree" response, especially when knowledge is low. Less educated respondents with little political experience may be especially susceptible to these effects. When people have given little thought to a topic, they are less likely to develop counter arguments against one-sided statements and are more likely to acquiesce.

The typical solution is to offer respondents a second, or even a third, substantive alternative - a forced choice. This provides respondents with a counter argument. This generally decreases the number of people favouring the first alternative in a single-sided format and also changes the distribution of opinion. Yet the strength of the arguments and alternatives presented is important - not all are equally effective. Creating a second substantive alternative also places researchers in the awkward position of shaping public opinion by deciding which alternatives to include as well as the substance of those alternatives.

Double-barreled Questions

One pitfall typically to be avoided is the "double-barreled" question. Here a proposed alternative is coupled with a solution. An example would be, "Do you approve of a tax increase to end the budget deficit?" Respondents may not be clear about what their response will mean.

 
Does a "yes" mean they approve of the tax increase, for example, or getting rid of the deficit, or both?

A "one-and-a-half-barreled question" contains qualifications that lead respondents toward choosing a specific alternative.

Key Words

Another potential area of difficulty is the actual wording used to describe the object or referent of a given proposition. Is government funding designed to deal with "drug addiction" or "drug rehabilitation", "assistance to the poor" or "welfare," "assistance to the poor" or "improving conditions of the poor"? Are respondents being asked to approve of the "president's policy" or his "handling of the policy"?

What words describe the actual choice respondents are asked to make? Are they asked to "approve," "support," or "favour" something, or rate it on a scale of "excellent," "quite good," "only fair" or "poor"?

A special form of this problem is the association of "buzzwords", or words used to evoke emotional responses from respondents, with response alternatives. These words may inject partisan or ideological calculations into responses, as well as allow less-educated responses. In the United States, the word "communist" was well known for its effects on responses to foreign policy questions. The mention of the president almost always has a large impact on the results, usually in favour of whatever the White House has done. In South Africa, one might well obtain significantly different results if people are asked to compare their lives now to twenty years ago, that is, if they were asked to compare to "life under apartheid."

"Don't Knows"

Question format also affects the number of people who offer an opinion. Increased levels of "don't know" (DK) are obtained by agree/disagree questions, questions on remote and abstract issues, and ones with greater task difficulty (e.g., ones that require long explanations or require respondents to make future-oriented projections).

The level of DK can also be affected by the use of a "filter", such as, "Or haven't you had a chance to think about this?" Filters usually increase the absolute number of DK responses by legitimizing non-response. But filters can also affect the actual substantive distribution of opinion. Those who are likely to give an opinion when they really don't have one do not come from the various response options in a random way. The problem is made more difficult because "floaters" (those who give differing responses to different types and forms of questions) are difficult to predict and don't seem to be characterized by any single trait.

All of the potential difficulties reviewed in this section arise because of the way human beings think. The way in which humans process information is greatly affected by how that information is presented. This is no less true of the survey environment. People do not conduct exhaustive searches for representative instances of an opinion or an attitude in long-term memory. Rather, they look for the most accessible information from either the environmental context of recent history or experience, or from the immediate context of the questionnaire and interview.

There are no easy answers. A question's meaning is always partially dependant on where it is placed in the questionnaire. The way we consider our response is always partially dependent on what the response alternatives are and how they are presented to us.

Besides a few well-worn "rules of thumb," therefore, there appear to be no obvious solutions to many wording problems. While we may try to avoid "contaminating" questions, "clean" questions stripped of political context may be unrealistic and irrelevant. Buzzwords and double-barrelled associations are often what give a survey questions political realism.

Survey Preparation

Before going into the field with the interviewers, every questionnaire should be tested on respondents similar to those who will be interviewed in the actual survey. On the basis of this trial run, flaws in the questionnaire can be identified and necessary adjustments made.

While it might sound Orwellian in the extreme, most pilot studies are done in a small room with a one way mirror so that one can watch the actual interview, as well as see and hear exactly what goes on with regard to the questions, answers and body language.

This provide an excellent opportunity to check whether:

• instructions are clear to the interviewer;
• questions are clear to both the interviewer and respondent;
• questions read well;
• questions make the person uncomfortable or anxious;
• respondents tire and give less thought to their answers, and at what point this occurs;
• questions gather the types of answers in which we are really interested.

In testing questions on social identity, for example, Idasa found that the wording: "What do you call yourself?" was generating some very personal labels, such as "nice person" and "open minded", which was not the information being sought. So it was necessary to provide some sort of context in which the person was to answer the question. So the question was changed to: "Thinking of all the groups in South Africa..." (a listing of various types of groups was provided) "which group would you consider yourself belonging to first and foremost?"

Pilot testing usually leads one back to the drawing board to rewrite at least some questions, or to delete some questions. If it appears from the pilot test that the questionnaire is too long, the conceptual framework will need to be revisited. In the final analysis, some decision will need to be made as to whether to drop whole concept areas from the survey that may be interesting but not vital or to drop one or two questions from each concept area.

Translation

In a multilingual society, it is imperative that every respondent is able to answer questions in the language with which they feel most comfortable.

While it is extremely time consuming, the best way to ensure that questionnaires mean what they are intended to mean after translation is by using the "double blind" method. One set of linguists takes the original questionnaire and translates it into the desired languages. Then, a separate set of linguists take those versions and translate them back into English (or the original language).

At that point, the re-translated version needs to be checked against the original language. Any differences need to be reconciled by finding another word in either the original or the translated language that better expresses the key concept. Note, however, that if the original language is changed, all the other translations need to be revised accordingly.

Yet good translators do not simply translate every word. They must know when respondents may be used to hearing key words in a another language, such as "parliament", so that the proper term can be used.

Interview Method

The method of interview is key. Responses to survey questions are not necessarily independent of the way in which they are obtained.

Telephone

An increasingly popular method is to contact respondents by telephone. Telephone surveys are often cheaper, because they do not require interviewers to travel all over the country or region to people's houses, and they may be quicker.

A major problem is presented, however, by actual rates of telephone ownership. Even in the United States, it has been estimated as late as the early 1990's that five percent of the national population (and ten percent in some states) still did not have a telephone.

The big problem is that telephone ownership is not random. It is highly associated with household income. Those who do not have telephones are extremely likely to have social and political views that differ sharply from those who do have telephones. In developing countries, the low and highly uneven incidence of telephone ownership makes nationally representative surveys impossible, significantly under-representing lower income households.

In the U.S., the Gallup organisation once estimated that pre-electoral telephone surveys were five to six points more favourable to Republican candidates than to Democrats. A striking example in South Africa occurred in 1992, when the Human Sciences Research Council (HSRC) used a telephone survey to project that F.W. de Klerk had more support than Nelson Mandela in a hypothetical presidential race. The HSRC observed that they had correctly weighted the final responses according to the correct racial proportions of the country. What they forgot, however, is that those Africans with telephones were not very typical of Africans in general.

Telephone surveys also make it easier for people to opt out of the survey and, therefore, out of the sample. As noted below, it is important not to allow people to self-select themselves out of the sample. Moreover, telephone interviews rarely establish the rapport possible in personal interviews, which is necessary to allow interviewers to lead respondents into controversial topics.

People are simply more reluctant to express negative attitudes with unseen strangers. Because telephone samples are also likely to include alienated people, they are generally biased toward less negative data. In some situations, however, the lack of direct contact may render telephone surveys preferable.

Post/Mail

In a survey by mail, a questionnaire is simply sent to the respondent through the post. The questionnaire is then self-administered. Because they involve relatively minor labour and postage costs, they tend to be relatively cost effective.

On the other hand, mail surveys tend to get a very low response rate. To get a one-third to one-fourth return rate is considered a major success, and even these rates require a good deal of work contacting recipients a second or third time to coax them to fill it out and return it. Other incentives may be offered, such as prizes or chances to win prizes.

In some places, the effectiveness of postal surveys is also greatly hampered by both the quality of the postal service and high levels of illiteracy. For these and other reasons, postal surveys are usually only carried out on specific targeted populations, most often very educated audiences, top-level managers, or "elite decision makers."

Personal Interviews

Personal interviews can establish a relationship of trust with the respondent, allowing more sensitive questions and more in-depth answers. Because respondents can actually see their interviewer in face-to-face interviews, interviewer characteristics such as race and gender may influence the willingness of respondents to offer socially undesirable responses on issues of race and gender. Thus, in a place like South Africa, survey companies usually try to make sure that interviewers are of the same race as their respondent. If the survey was on sex, or gender issues, an effort would be made to ensure that interviewers were the same sex as respondents.

Personal interviews tend to be expensive, however, because of labour and travel costs. In addition, personal interviews face many logistical hurdles not encountered in the other methods. Most simply, there is the prospect of getting past someone's front gate, let alone through their door. Especially in South Africa, "bad" neighbourhoods, apartment building security systems and other minor problems, such as Rotweiller or Doberman Pinscher dogs, often prevent surveyors from contacting everyone in the sample.

Again, if unchecked, "non-response" (those who cannot be reached at home or who refuse to be interviewed) can play havoc with a sample because these people almost always differ from the general population in attributes and attitudes.

Sampling

Surveys are useful when we want to know about large numbers of people. The goal is to speak to some smaller number of people (sample) and to generalize to some larger group of people (population). Sampling is generally complex and usually requires lots of statistics and computers. But it is important to understand the basic logic so as to intelligently communicate to a fieldwork company what is desired and to adequately check on what they actually do.

For what population is information being sought and generalized? Is it all voters? Only likely voters? Only men or only women? Only young or old? Only black or white voters?

Drawing a sample of some larger population can be likened to making soup. When mixing up a big bowl of soup, any good cook will tell you that you only need two or three spoonfuls to get a reasonably reliable idea of what the entire bowl tastes like. Of course, this assumes that the soup has been mixed well and that, as a result, all the salt is not clumped in one corner, or that all the potatoes are not lying at the bottom of the bowl, or that all the garlic has not moved to the side of the bowl. Any of these possibilities would mean that the spoonfuls were likely to be unrepresentative of the whole bowl.

Again, assuming a well-mixed bowl, about the same number of randomly drawn spoonfuls will give a good idea of the taste regardless of whether the soup comes from an ordinary black pot on a home stove or one of those industrial-sized pots in a restaurant. The same number of spoonfuls should do if the bowl is well mixed. The number of desired spoonfuls may increase slightly, but not nearly as fast as the increase in the size of the bowl.

But few populations are "well mixed": there are often groups (or strata of people) whose attitudes differ significantly from other people (just as there are different vegetables and seasonings that taste differently) who are not randomly scattered throughout the population but tend to be clustered together in certain regions, cities or neighbourhoods.

Any possibility that a sample would miss, or under-represent, any of these groups or strata in a purely randomly drawn sample should be reduced to the greatest extent possible. In effect, while attempting to draw a sample that is representative of the whole population (or bowl of soup), one will probably also want to "stratify" the sample so as to draw mini, subsamples of each desired subgroup (thus ensuring adequate subsamples of potatoes, rice, and tomatoes).

This means paying attention to representing people of all race and language groups, all regions, or rich and poor, or urban and rural. Usually, these strata should be constructed so that their size is proportionate to the size of the stratum in the actual population. Thus, if the rural component of some desired population is 52 percent, the rural component of the sample should be the same.

Once a decision is made to stratify along more than two dimensions, however, deciding upon the actual composition of the sample can get quite complicated. A national sample in South Africa, for instance, might dictate that a given number of mixed-race, rural people from the Western Cape is needed as well as a given number of African and white rural people from that province. This would also mean getting numbers of metropolitan people from each group from that province as well. Since this can become quite difficult, a trained demographer or mathematician can help work this out.

In some instances, however, it may be desirable to have a disproportionate random stratified sample. This usually occurs when some desired subgroup comprises a very small proportion of the desired population. A proportionate sample in South Africa, for instance, would consist of only 9 percent mixed-race respondents and around 2 percent Indian-background respondents. But if, for reasons of cost, the national sample is only 2,000 people, this would result in less than two hundred actual mixed-race respondents and around forty Indian-background respondents.

It may not be possible, however, to base any reliable statistical estimates on a subsample of forty people. Even with two hundred people, the statistical margin of error may be so large that projections about mixed-race voters would not be very helpful to guide a voter education programme targeted to these communities. This would become even more important if one wanted to examine the differences between men and women, or urban and rural people, or party supporters, within mixed-race and Indian-background subsamples. The numbers of respondents within these subgroups would start to become unhelpfully small.

Thus, an "over sample" of small groups such as these might be considered. In this case, while some small group might merit only forty interviews on a strictly proportionate basis, a decision could be made to conduct a hundred interviews in order to have a more reliable base of information. Once all the data is selected, this disproportionate sampling is corrected by "weighting" the hundred interviews downward by the appropriate ratio so that they represent the correct proportion of the entire sample.

Weighting is also useful with regard to other important demographic attributes for which information exists, but will not be known until respondents open the door. The number of men and women in the desired population as well as the number of people in various educational strata may be known, for example. But it may not be possible to stratify the sample according to these traits a priori, because the interviewer will not learn until someone answers the door or the telephone, whether the respondent is male or female or how much education they have.

Once the sample is done, it can be compared to the actual population along demographic lines for which information is available. The people in the sample can be weighted upward or downward in the appropriate direction. Imagine, for instance, twice as many women being contacted in the sample than actually exist in the overall population. In this case, each woman in the sample would ultimately be weighted downward by half to bring the proportion of women in the sample to its proper proportion.

Constructing samples is a key determinant of the cost for a given survey. Samples that require a high proportion of rural respondents will tend to be relatively expensive given the costs of getting interviewers deep into rural areas.

To be able to survey representative samples on a national basis usually requires a considerable amount of infrastructure and personnel. Thus, most nongovernment organizations, even if they are able to design the project and are capable of analysing the results, will still contract a professional organization to conduct the actual interviews.

Selecting Participants

Getting To A Household

Once the desired construction of the overall sample, e.g. total sample size, plus the number of interviews to be conducted in each sub-stratum has been decided upon, the next step is to translate that desired sample into actual interviews.

There are at least two very different ways of proceding at this point. The key distinction is to decide between a random probability sample or a quota sample.

Random Probability Sample

Here, every person in the population has an equal and known chance of being selected in the final sample. This presumes that the size of the overall population is definately known. If that size is "n", then the person's probability of selection = 1 / n.

If there is a list of all the people living in a given population, pure random probability sampling means randomly drawing out the total number of names from that list until the desired number is reached. Or, if the sample has been stratefied into subsamples (e.g. urban and rural people), X number of names from the list of urban people, and Y number of names from the list of rural people would then be drawn. Once the sample is drawn, these persons would simply be visited, contacted by telephone, or mailed the questionnaire.

Even where there is a complete list of every person in the population, personal interviews using pure random sampling tends to be inordinately expensive. Getting interviewers out to each spot randomly selected by the random sampling procedure regardless of how remote it is from the other interview sites is costly. Thus, most personal interview strategies use clustered random sampling. That is, travel costs are minimized by sending a group of interviewers to some randomly selected location and then conducting a series of interviews at that location.

Clustered sampling is widely used because it reduces costs, but also because, very often, a list of names is not available. Many countries, or provinces, or municipalities, have no such list, or if they do, will not share them with a researcher.

Thus, although the size of overall population and the number of people living in various regions or in various subgroups might be known, there may not actually be a list of individual names. Clustered sampling around sampling points helps researchers get to individual households in a way that maintains randomness and an equal probability of selection.

This involves the selection of a series of what are called "primary sampling units" (PSUs). PSUs are the smallest units from which final sampling points will be randomly drawn. PSUs consist of the smallest geographical units for which there is reliable population data (and for most surveys, this means the population 18 years and older). In some counties with good census data, these may be called "Enumerator Areas."

Final sampling points cannot be randomly pulled from these PSUs because the PSUs will almost always have different population sizes. Even where there are census determined Enumerator Areas consisting of an set number of households each (for example, in Zimbabwe, EAs have 100 households each), the number of people in each household will differ. Thus, each potential PSU must be weighted by the actual number of people living in it. That is, the chance of selecting a final sampling point from a PSU must be proportionate to the actual population size of the PSU.

Once each PSU has been weighted by its population size, final sampling points can then be randomly selected from the list of PSUs. The actual number of final sampling points is determined by the number of interviews to be conducted at each point and the total sample size. Most surveys conduct between five and seven interviews at each point. Thus, if five interviews will be done at each point, and the overall sample size is 2500, a list of 500 final sampling points must be randomly selected.

Now we know where we want to go. For instance, a generated list might reveal 350 suburbs, some populous ones might be selected more than once, and 150 rural magisterial districts. Survey researchers will then find maps for each of those areas, and then randomly select a point in a suburb. This can get quite elaborate, as some researchers will lay over a transparency of randomly numbered points, then select a number at random, and then look for the street on the map it overlays. That is where they finally will send the interviewers.

In many areas, no good maps exist. Or, rural maps might be so large that they only show the locations of towns, but not streets within the towns. In this case, one might resort to a rule such as starting at some common point, such as a church, school, municipal building, or water tap.

Once interviewers know what point they have to go, then they should follow a set of rules that allows them to start picking houses, again at random. For instance, they might go to the agreed upon point, face the sun, or face east, and then proceed ten houses, and then interview at every fifth house. The rule should be random, but all your interviews should follow the same rule. The whole point is that the interviewer should play no role in the selection of the household.

The very last step involves selecting an actual, real live respondent. Again, giving every person an equal chance of selection demands that interviewers do not only speak to people who answer the door, or the telephone. If interviewers are working from a sample chosen from some grand population register, then they need to speak to the specific person whose name appears on the list.

If there isn't such a list, once interviewers are inside the door, or have someone on the phone, they will need to "enumerate" the household, or make a list of people who live in the household (and, normally, are citizens above the age of eighteen). Then they need to choose one name at random and interview that and only that person. A common way to select that person randomly is to ask which person in the household had the most recent birthday. In rural areas, people may often be irritated at not getting the chance to express themselves (especially if the head of household is not selected, particularly if it is a man), and they may not understand the birthday method. One visible way to display the logic of random selection is to distribute a series of colour coded cards to everyone who is eligible, then gather them and ask someone in the house to randomly pull a card from the stack: the person who had held that card is the one to interview.

However, not every door we knock upon nor every telephone call placed will result in a successfully completed interview. Many people will not be at home, many will be at home but remain inaccessible for a variety of reasons, and many people will simply refuse to speak to interviewers. As mentioned above, it is important that interviewers, to the greatest extent possible, not allow people to self-select themselves out of a sample. This is because those who are not at home or are unwilling to participate are likely to be different from the overall sample in important ways. The people more likely to be at home, especially if interviews are conducted during weekdays, are disproportionately likely to be young people, unemployed, housewives and the elderly. Those unwilling to talk to interviewers tend to be more alienated. Those types of people need to be represented in any sample. This is especially true if alienation is likely to be related to the topic of interest, like voting.

"Non-response" can play havoc with the representiveness of a sample. In the U.S., "non-response" rates have doubled since the 1950's, going from between 12% to 22% to anywhere from 30% to 55% for personal interviews and 25% to 35% for telephone samples. In South Africa, non-response rose well over 100% in some conservative white communities for surveys conducted in 1993 and 1994.

One method often used is to correct for non-response by "weighting" the obtained responses according to known census statistics. So, if not enough middle-aged men were actually interviewed, the responses of those middle-aged men may be "weighted upwardly" by some fraction. So, for example, if there are only half as many of this group as we need in the realized sample, we simply multiply each case by 1.5.

This is problematic, however, because it assumes that those who were not included in the sample or refused to be contacted are similar to those questioned across the entire range of attitudes tapped by the survey. However, by the very fact that the person was out of the home (probably working or shopping), or the very fact that they refuse to speak to the interviewer, probably makes them different from those people who are at home, or want to speak with the interviewer.

There are a few things that can to done to minimise the incidence of those who refuse to speak. Interviewers need to be extensively trained so that they are courteous as possible. The questionnaire should also feature an introduction that makes the survey as interesting as possible to the potential respondent, as well as convey to them the importance attached to their views. Finally, interviewers should ask if they have come at a convenient time, and if not, offer to make an appointment at a better time when the respondent is not busy and can spend some time concentrating on the questions.

Interviewers can try to minimize the effect of people not being at home in several ways. First of all, they should try and do a large share of interviews in the evening and at weekends. Weekdays are difficult because workers are out of the home, but housewives may have more time to speak to you. The absolutely worst time seems to be supper time, when people are busy either making dinner, or eating, and are most irritated at being disturbed.

Secondly, interviewers can devote a lot of attention to what are called "call-backs." If the person on the list or the person randomly selected with, for example, the birthday method is not available, interviewers should ask when that person is likely to return and then come back at that point in order to get the interview. Most survey companies require interviewers to make at least two, if not three "call backs" to get the originally selected person. Some large surveys providing marketing information on media and product usage actually require four call backs.

Only once the interviewer has made the required number of call-backs and still had no luck, should they be allowed to "substitute" another person for the original respondent. Furthermore, they should not substitute with someone else from that house. Rather, they need to follow some rule, such as going two or three houses to the right or to the left, or dialling a new number below or above the original number in the telephone listing, and go through the whole process again.

The whole point is to make an extra effort to ensure that those likely to be out of the home are not easily allowed to slip out of the sample, and that they are not easily substituted with the types of people that are more likely to be found at home.

The advantage of a random probability sample is that it allows researchers to take advantage of the mathematical laws of sampling for the purpose of generalizing sample results to the larger population. These laws tell us that the average (mean) of any randomly drawn sample will tend to equal the mean of the overall population from which it is drawn. More specifically, for any given sample size, these laws provide the formulae to calculate the exact margin of error around any sample. That is, for a given sample size, a sample estimate will be within plus or minus the true mean of the overall population 95 per cent of the time. This is because, if a large number of samples are drawn, the laws of probability indicate that about five percent would fall outside the normal margin of error. However, 95% of the samples would fall within a calculable range, or band, around the true population mean. The larger the sample, the more narrow that band.

Quotas

An alternative method is the quota sample. Here, the overall sample is constructed to represent the overall population along all the important lines of distinction. For instance, a decision is made that the sample should have certain percentages from each province and from each city, certain percentages of men and women, of each language group, and of each race group. However, the final selection of respondent is left up the interviewer.

Each interviewer is given a quota to fill in their area: that is, a list of the number of people they must find and interview who fit different demographic categories. So an interviewer may be told that they must find five African men and six African women who live in urban areas, and seven African men and eight African women who live in rural areas. However, they are not told which houses to go to, or streets to go to, or given any random process to follow. They must simply find people that fit the desired categories.

Because interviewers are relieved of the duty to go through all the random processes described above, they are able to obtain the desired number of interviewers much more quickly and with far less travel costs. This makes a quota sample considerably cheaper than a random probability sample.

However the major drawback is that, because the equal and known probability of inclusion that characterizes a probability sample has been dispensed with, mathematical theories of probability cannot be applied to make any inferences from a quota sample to the overall population. The frequency of responses from a quota sample can be calculated, but strictly speaking, the degree to which those results are representative of the true values in the overall population cannot be determined.

Getting Into the Field

Getting into the field requires the selection and training of interviewers as well as their deployment and management.

Interviewers

Whenever possible, only trained and experienced interviewers should be used. In any case, training is extremely important. If professional interviewers are not being used, the process of selecting respondents in the household needs to be reviewed thoroughly. The questionnaire should also be covered extensively, including how questions should be read and certain words emphasized. The importance of not prompting respondents inappropriately should be stressed and interviewers should be cautioned against giving their own opinion either explicitly or implicitly through clothing, facial expressions, or body language. Even experienced interviewers must be trained extensively with a new questionnaire.

Because they often work cheaply, university students are often hired to conduct interviews. A word of caution about this option is in order, however. Students who are interested in sociopolitical surveys often are politically active and may be more prone to communicate their own preferences to respondents in implicit and even explicit ways.

As mentioned above, one should make sure that interviewers are from the same background as their respondents, especially if the survey touches on related matters. On some occasions, however, it might be desirable to follow a different procedure.

Fieldwork

Fieldwork should be done under the strict supervision of field supervisors. Most respected companies will conduct call backs to at least 10 to 15 percent of the households interviewed, find the person interviewed to confirm they were actually interviewed, and also go through some of the questions to verify that the answers recorded were actually their answers.

Probably even more important than doing this, is that the interviewers know that it will be done and that their payment will depend on getting satisfactory results from the call backs. As a cross section of society, fieldwork interviewers are no different from an ordinary cross section of society. And, unfortunately, many a story has been told by exasperated researchers of finding their field workers sitting under a tree and filling in questionnaires with fictitious names, addresses, and answers. Call backs can be done by phone if such service is widespread in a population. If not, they need to be done in person and probably before an interview team leaves an area.

Field supervisors should also check all questionnaires before the team leaves an area to make sure everything has been filled in completely and correctly, and if not, send the interviewer back to that person and obtain the necessary information.

Receiving Data

Actual responses will then need to be entered into a computer readable format. There are several statistical software packages that provide accessible data entry features and can also read and manipulate that data once it is entered. SPSS (Statistical Package for the Social Sciences) is one widely-used programme.

Fieldwork companies typically will enter the data and even provide a technical report. Even so, the organization paying for the survey should obtain its own data set on computer disk, preferably compatible with SPSS format. The organization may also want to do its own analysis, or if it is inexperienced in statistics, contract another individual or entity to to do this. The organization should, however retain the freedom to monitor and evaluate what is provided. Just as important, there are probably much larger sets of data manipulations, cross tabulations, or correlations that could be run, but that a survey company will not present in its technical report.

Time

To do a proper job, the length of time from conceptualization to actual data analysis and report writing can be unexpectedly long. Even when it is absolutely imperative to go into the field quickly to capture public reactions to some fast-breaking event, it is hard to imagine doing a good job in less than six weeks.

Personal interviews of a national sample could often take the most experienced company several weeks to complete. Larger projects, such as attempting to test some model of voter participation, will usually take at least several months or even up to a year if fastidious academics become involved.

Costs

Project costs will include administration costs, data entry, and general overheads. The largest portion of these costs usually is for fieldwork and includes costs of transport, lodging, and the actual labour of interviewers and field supervisors. The latter are determined by the number of interviews to be done, the number of telephone calls or household visits required to realize the number of actual interviews, the number of hard-to-reach areas to be visited, and the length of each interview. Thus sample size, stratification, interview clustering, and whether a probability or a quota sample is used are important both methodologically as well as financially.

Given the costs associated with conducting independent surveys, however, it is possible to buy space for one or two, or sometimes even a dozen or so, questions in on-going market research surveys. Market research organizations tend to conduct surveys on a regular basis, and, because a number of their clients may include questions, the costs of the survey are shared. Costs often can be calculated on a "per question" basis, sometimes with an initial "buy-in" charge. Many organizations decide to "piggy-back" their questions onto ongoing omnibus surveys. This is very efficient when one only wants to put a few questions to a representative sample, such as checking on current levels of interest in the next election, or current levels of registration. Also, the frequency of such omnibus surveys allows one to check these issues on a more regular basis and to monitor trends over time.

Some questions, like "Why are people uninterested?" or "Why are they not yet registered?", lead to many other questions. The more questions, the higher the costs. In addition, it may be desirable to get respondents focused on issues of voting, elections, and democracy in order to obtain more thoughtful and considered responses. With an omnibus survey, there may be no control over whether respondents are answering a question about the competitiveness of elections immediately after being asked about their monthly consumption of motor oil. Finally, due to reasons of cost and client interest, ongoing market research surveys may not be done in remote rural areas or poorer areas.

Using Existing Data

Educators trying to establish educational needs and to identify target constituencies and country infrastructures may, at first glance, assume that they have to do this from scratch. Perhaps they will, but a little time spent investigating existing data sources will have a number of benefits:

• less time needed for conducting an investigation
• lower overall costs
• gaps identified that allow more focused research
• general data gaps discovered to consider for more general developmental purposes

There are very few places where nothing is known. And there are very few places where there is no existing source of information. The trick in some closed or authoritarian societies, however, may be getting access to existing information.

International organizations will want to work closely with indigenous ones to access existing data. Often they are unaware of existing repositories of knowledge, particularly if it is of an oral and traditional nature, and they are often amongst the first to assume that something new and impressive must be done. But those working on election programmes need to understand their work within the larger democratization framework. Good information is needed by those who govern and by civil society. And elections should stimulate the gathering and availability of that information.

Paper, People, or Bytes

Information available in computer databases has the advantage of being easily updated and maintained as well as moved from one place to another. When carefully collected and where software and hardware planning has been done, databases can be remarkable. They can also be frustrating, however, especially in developing countries. Information may be stored in incompatible formats, or it may be out of date, incomplete, and often locked away. Computers can be a curse as well as a champion of open democracy.

Information on paper does not suffer from compatibility problems. Here the problems are ease of handling, whether the documents and publications are available, preservation, and the cost for revisions.

People See Only As Far As Their Horizon.

Those looking for data will want to look at questions of reliability, accessibility, and cost. In some settings, for example where there there have been authoritarian systems, there may be no official culture of "freedom of information." Government bureaucrats may be unresponsible to requests for information. They may consider it priviledged or even a state secret. There is also a danger that data collected by the government has been manipulated for political reasons. So even if it is made publicly available, it may be of little use to educators.

But even accurate data presents educators with a variety of useability issues. This is because there is a virtual sea of information that requires careful selection. Educators will want to establish precisely what information they require and how they intend to use it prior to even beginning the search. There will be iterations on these questions, because once some information is available it leads to further questions. Nevertheless focus is essential.

Look in the Obvious Places

Voter rolls and related data provide an immediate starting point if these rolls have been collected nationally or regionally. They will provide basic information about numbers of voters and geographic dispersal. In developing countries and transitional societies, however, voting rolls may be of poor quality. Educators in these types of situations will need to assess how accurate and current the voting rolls are in making a determination about their usability.

In order to establish the rolls, information shoud also exist about registration officials, places where registration has taken place, and possibly even places that were evaluated and then not used. Amongst these places will be many public venues, such as libraries, schools, community halls, clinics, and government offices, as well as more temporary structures linked to community gathering places, such as sports fields, markets, and so on.

Other basic information will be available in forms that can vary from the rudimentary to the highly computerized. Telephone directories can be useful as well as government directories, income tax mailings and address lists (where these are public documents), television and radio licence lists, and market research.

Beyond this basic address and geographic information, there will be government yearbooks and reports on a wide variety of subjects. In poor countries, these reports may have been done by international agencies or international companies interested in development plans and opportunities.

In addition to reports with a developmental focus, many countries have tourist bureaus and tourist publications that contain basic country and travel information. Bus and train timetables, hotel listings, contact offices for local information bureaus, all increase the amount of information about the country infrastructure and basic governance.

With the burgeoning of the Internet, it has become possible to do worldwide searches for information about countries. While not all of it may be held inside a country, it is surprising what information may be held in an academic institution. At present, access to such institutions in the western and northern hemispheres is greater on the web. but these institutions often host information servers that link organizations and networks in the southern hemisphere.

Beyond these basic sources of information, there may be libraries, government departments, research units linked to national, regional, and local governments, and national and regional statutory research institutes. All of these collect information, some of them will release it upon request and perhaps for a fee. International and domestic NGOs have vast amounts of personal experience. have collated information about countries and are often willing to make this available more freely than government departments.

Perhaps most useful of all, but not always accessible, is the data collected in political and marketing polls. The reason it is useful is its direct bearing on individual and group attitudes and insights into issues that relate to elections. If it is possible to develop a relationship with collectors of such survey information, it is possible to request them to reanalyse existing data to address particular questions voter educators may have.

All the sources and organizations listed above have been collecting information not for electoral purposes but for a variety of other reasons over an extended period of time. The information, therefore, has breadth and depth that officials preparing for a specific election period cannot hope to replicate.

At the same time, it suffers from the fact that it has to be reorganized in forms that are useful to educators. This may be difficult, time consuming, and costly. Data mismatches, information collected from different periods and with varying degrees of reliability, patchy information with biases towards cities and men, revenue producing activity, and old political debates all conspire against the compiler.

There may even be occasions when the task of assembling this information is larger, more time consuming, and more costly than going out and getting it anew. But this is unlikely in the sphere where educators are working. Focusing on developing an understanding of the voting population, the country infrastructure available to support the educational programme, and getting a grip on educational needs faced by various groups and audiences will all help to ensure that the data available can be more cost effective than anticipated.

Interlocutors and Intermediaries

While some educators are preparing focus groups and surveys, others prefer to go into the field and talk to people who are working with the target audience or constituency. This has the advantage of being quick, if adequate care is taken in establishing with whom conversations should be conducted. It also helps give educators access to a range of nuances and undercurrents that are difficult to achieve in any other way. Another benefit is that such people provide a fund of local knowledge about educational conditions, the political environment, and the identification of educational issues.

This consultation or conversation is conducted at a practitioner level, educator to educator; or at the level of educator and community leader. So it also ensures that ownership of the programme is developed from the very outset. There are disadvantages to this approach, particularly if it is relied on to the exclusion of additional data collection. But for educational purposes, where local knowledge and local ownership are so important, it is a potent and relatively cost-effective way to get the programme into the field.

It can be extended in effectiveness at limited additional cost by adding two related techniques. The first is the consultative conference where a range of people come together and discuss educational needs and educational conditions in a structured programme. The structured programme can either be very formal in nature, with different speakers addressing different topics, or more informal and dialogue oriented, with brief introductions to issues followed by facilitated round table discussions.

The specialised focus group selection of membership is based on practitioner competence and local knowledge.

Two Different Types of People

Going into the field and talking to people at random is not adequate. Care must be taken in the selection of people. Understanding the use of the two terms "interlocutor" and "intermediary" gives some insight into the selection that needs to be made. The terms also indicate some of the difficulties and limitations that can be encountered and allude to the care that must be taken. Interlocutors speak in the place of the target constituency or on behalf of them. Intermediaries stand between the educator and the audience and act as a bridge between them.

Educators will develop a list of people with whom to converse on the basis of their assessment of effective community education and nongovernmental organizations operating within the sphere of investigation. The sphere of investigation may be national, regional, or local. In addition, they will identify community leaders based on their legitimacy within the particular community.

Finally, they can engage in fruitful conversation with individuals who interface with the community and the world of the educator, such as students, academics and members of diplomatic bodies. The latter can be a particularly helpful group where there is a wide gap between the educator group and the community: as for example when an international programme is being planned or when the educator group has to work in a part of the country where they have no previous experience. Indeed it will be essential to identify such people who can join the educator team on an extended basis if possible, even as interpreters and drivers if not as educators themselves.

Identify People

Once a tentative list has been gathered, it can be assessed in cooperation with individuals who have already been identified. In other words the collection of the list of people is an iteractive process. Educators identify a first round of people, perhaps based on advice from a trusted NGO, or even as a directive from the election authorities. This group of people then suggests others whom the educator should contact.

The second list will grow and also contain people nominated on a regular basis. A second round of conversations will take place and the list will grow. At some point in this exercize, the list will become circular. In other words, new references will be made to people with whom the educators have already spoken.

Educators will want to take care to maintain good records of the conversations they have had and the details about those they have interviewed.

Confidentiality

In situations where these discussions are undertaken in contexts of conflict, and where those involved are discussing the needs of members of their own constituencies, there will need to be an understanding that the information being collected will be handled confidentially. Especially when conversations are being conducted between practitioners, there will be critical and reflective comment on organizations operating within the community and with the given constituency. The assumption of these conversations is that programmes are being developed to assist target audiences. Any other use of the information can have an impact on the relationships that exist between those being interviewed and the communities within which they operate.

Limitations

The techniques being proposed here are based on a methodology used in evaluation studies and described as "triangulation". This term is used in establishing the position of a place or person on a map. In other words, information is obtained which establishes a particular direction. Knowing where the direction is taken from enables one to draw a line across the map. Then a similar direction is taken from another position. If this is done three times from different points, a small triangle will be formed on the map. That is where the person or place will be found.

In the case of conversations and interviews that take place with a variety of interlocutors and intermediaries about the same community, the educator will be taking notes both about the information being given and the source of that information. In other words, they will judge the information relative to the interests and position of the person giving the information.

If this is done with care and if the same conversation is conducted with a range of people, the data about the community will become more and more reliable. It will be possible to place the community within a map of data, some of which confirms and expands while some of which establishes scepticism and negative implications.

As mentioned, there can be problems. These can be overcome, however, if this particular technique is coupled with the gathering of information from other means, such as surveys, existing data and focus groups. It is also possible to test the data being gathered with a reference group.

Reference Groups

Educators can establish a small reference group of trusted organizations and individuals with whom they can review the information they are obtaining in the field. Such groups meet regularly but do not have a direct interest in the proposed direction of the programme or its intended outcome.

Collusion and Unreliability

There are times when it is in the interests of some people and organizations that educators have a particular view of the community. There may be a perception that the educator team has access to money that will be spent in the community, or that the educator team should develop programmes in a specific way that benefits the community or even a particular political party. If the educator team is comprised of outsiders, they may not even be aware that those they are interviewing are meeting one another and discussing implications of the programme amongst themselves.

Such collusion need not be undertaken in order to diminish the reliability of the information being provided. People have an interest in being considered intermediaries or maintaining their prestige within a community. They may not be willing to admit to areas of ignorance and may overplay their level of influence in order to impress the educator team.

Groupthink

There may also be a dominant view amongst those selected about local issues that doesn't entirely match the present reality. During transitions and crises, there are substantial shifts in conditions of reality, and organizations in particular cannot always keep up with these shifts.

Or there may be dominant political organizations and ideas that are taken for granted. These may be real. A single party may well have the support of all members of a local community. But in such positions of dominance, often it is easy for dissent to be suppressed and to become invisible. Of course, this raises the interesting proposition that members of minority support parties can also make claims that cannot be tested.

Gatekeeping

Finally, there are those who act as "gatekeepers" rather than guides. They control access to community information. Some are admitted to the community, others are not. And the reasons for this gatekeeping may be political, ideological, or personal. Educator teams will develop internal diversity in order to ensure that they are not kept out because they are all men, or all from a particular country, or of a particular cultural and ethnic background.

This alone will not prevent gatekeeping. But the development of an iterative approach can assist in overcoming it. In traditional societies, educators may have to be patient if they want to get through the gate. There is a range of strategies for dealing with this, but perhaps the most effective is the development of a relationship of trust with an intermediary who can introduce the educator to the traditional leadership.

Testing Information

Educators moving into situations where they suspect the information may be coloured by any of the above will be looking for reflective individuals who are willing to be fair to all political points of view and who can demonstrate the reliability of their opinions by pointing to supporting evidence. Or they may choose to conduct interviews that include members of the target audience directly on the basis of a small sample, just for verification purposes, rather than conducting a full survey.