An obligations-based assessment framework is most useful
when groups can find ways to collect data that correlate reliably with those
obligations or standards. One basic tool of observers in this regard is the
election day checklist. Short-term observers (STOs) use these forms to record
their findings at each polling station where they observe. Since Garber
included a sample checklist in his 1984 handbook, forms have become more
sophisticated but have retained common elements. Observers still record when
they arrive and depart, whether ballot boxes are properly sealed, whether
voters are intimidated, and whether the setup of the polling station preserves
secrecy of the vote, for example. But most groups now use different checklists
for different parts of election day, including opening procedures, polling,
closing and counting, and tabulation of votes. Groups also customize questions
for particular country contexts. A question that asks about unauthorized
persons present in the polling station, for instance, will reflect national
laws on the subject and will exclude “security personnel” if no restrictions on
security personnel exist. The range of topics covered also has expanded.
Observers may collect information on numbers of female polling officials and
other indicators of gender inequality, as well as on accommodations for
speakers of minority languages and voters with disabilities.
The most important issue guiding checklist design, however,
is how to collect accurate information that can be compared meaningfully from a
large number of observers reporting from different locations. Forms are now
engineered to elicit the most specific and objective information possible,
allowing less space for individual observer bias. What was once asked as “Are
voters identified as prescribed by law?”[i]
might expand to a series of questions that guide observers through discrete
procedural steps (e.g., “Did the PEC [precinct election commission staff] check
the voters’ IDs?” “Did the PEC sign and stamp the ballot?” “Did the voters sign
the voter list?”).[ii]
Detailed checklist questions help focus observers on specific aspects of what
can be a chaotic process, but they also enable staff analyzing reports to
identify where procedures are breaking down or where laws are consistently
violated. Most checklist questions today also give observers a range of answers
to specify the frequency of observed irregularities.
A shift away from open-ended questions to yes/no and
multiple-choice questions is another critical methodological development. A
question such as “How were unused ballots disposed of?” is better worded as,
for example, “Were the voter list, unused ballots, and spoiled ballots packed
in separate envelopes and sealed?” Open-ended questions complicate data
analysis and synthesis, making it difficult to reach meaningful conclusions
about the fulfillment of obligations. They also give observers leeway to
evaluate the process subjectively and potentially miss crucial data points.
However, most forms do instruct observers to elaborate on irregularities they
have observed so that reports of misconduct can be traced and reviewed later.
Expert staff then review the data as a whole and identify patterns.
While observer groups continue to tweak their checklists to
extract ever more precise data, significant challenges remain. The most
important one is the development of an overall evaluation question that provides
clear and reliable information regarding the general quality of the process at
each polling station or tabulation center. Determining the best way to word
this question in order to receive consistent and reliable results is a
microcosm of the methodological challenge discussed at the beginning of this
section: How do we weigh different parts of the process in order to arrive at
an overall assessment? Some scales may elicit more or less reliable information
than others. But how bad is “very bad”? Or, more challenging, what makes a
polling station good enough? Observers themselves may view violations they have
witnessed more or less seriously depending on comparative personal experience
or preconceived dispositions toward the process.
[i]
Garber, Guidelines, 49.
[ii]
OSCE/ODIHR, Election Observation Handbook, 111.
