Deciding What Information is Needed
First a decision will need to be made about what information needs to be collected and why. This process can be initiated by asking a few simple questions: 'What opinions or likely behaviours do you want to know about?' In social science jargon this is called the dependent variable. 'What do you think are the causes?' These are the independent variables. The answers will provide some important foundations for the survey's content.
Suppose one wanted to know about the causes of voter turnout, or the dependent variable. A decision is made to test the varying impacts of potential causal factors such as:
- information
- motivation
- interest
- efficacy
- perceptions of the degree of electoral competition among parties in a given election
These will be the independent variables. Basically, then, five key factors have been identified that are important and need to measured. All that is left is to define each of these concepts, or factors, so that there is agreement about what is meant by such terms as 'electoral competition' or 'motivation.'
A 'conceptual framework', therefore, is created that should act as the blueprint for the entire project. At any point in the project, one should be able to guage whether what is being done is helping to measure an element identified in this framework. If it is not one may discover that he or she has gotten off track (which is easy to do), and is working on something peripheral to his or her real interests.
At the same time, while writing survey questions, it may become apparent that there are some really important things that one needs to know about, but that have not been included in the blueprint. At that point, one shouldn't simply write a new question in an ad hoc fashion, but should go back and put the new concept into the blueprint.
Conceptualisation is usually based on:
- your knowledge of the local context;
- reviewing what is known and published about the subject, e.g. voter turnout;
- consultation with experts in the field.
Before Idasa's Public Opinion Service (POS) conducted a survey on the Cape Flats about public views toward crime, policing and collective action, for example, it called in a range of criminlinologists, sociologists, social workers, and journalists with extensive experience on the ground as well as with the relevant academic literature. This helped in the identification of the key conceptual areas, and thus, the parameters of the questionnaire.
Operationalization
At this stage, the goal is to begin formulating a structured questionnaire by designing specific questions to measure the 'real world' existence of the phenomena or attitude in the conceptual framework. In other words, the conceptual framework is being converted into an actual questionnaire.
Ideally, several questions should be designed to measure each key concept. One single question can often be an unreliable indicator of people's attitudes in that area. The ultimate goal is to be able to average the responses to all the questions about a concept to provide a valid and reliable aggregate measure or index of the concept (such as 'interest'). The series of questions should not simply measure the same exact thing, but tap various dimensions or elements of 'political
interest'.
A valid question, or series of questions, is one that actually measures what is meant by efficacy. One form of validity is called 'Face Validity'. That is, by reading the actual question wording, the question wording appears to be getting at what the intent. Another form is called 'Construct Validity'. This is when responses to the question, or series of questions, seem to correlate internally with one another, or with other questions that measure things that one would expect to be related to political interest.
'Reliability' refers to the extent that the questions would yield the same responses from one sample to the next, at any given point in time. Various types of statistical tests exist to help assess the extent of construct validity and reliability.
'Operationalization' is probably the most time consuming aspect of the survey process. Converting concepts into valid, reliable questions that measure exactly what they are intended to measure requires much thought and careful phrasing.
