Educational programmes have intended outcomes. They have plans that are being followed in
order to achieve these outcomes. These plans consist of a range of components working together
to ensure their successful implementation.
This section deals with ways in which monitoring and evaluation take place and some of the
dilemmas faced by those who conduct evaluations. It looks at methodological and organisational
questions and at the key question faced by all evaluators: the extent to which programme
implementers take advantage of the evaluation recommendations. The section following offers a
general overview of evaluation and its relationship to educational testing.
By implementing monitoring plans and evaluating their outcome once
completed, educators seek to ensure that they are being accountable to their stakeholders, true to
their intentions, and that they themselves will learn from disciplined reflection on the experience of
the programme for further work that they might do.
Terminology and Meaning
While it is common to use the twin terminology of monitoring and evaluation, this section merges
these terms into the single generic term evaluation, and talks about formative evaluation (that
conducted with a view to changing a programme while it is in progress) and summative evaluation
(that conducted to sum up the programme). But neither is entirely independent of the other.
The meaning of the word evaluation is a slippery one. Those requesting evaluations may be
trying to determine the value of a programme by measuring its impact. Evaluators may already have a view of the programme and
may be merely seeking external verification of that view. They may be trying to build support for a
particular course of action either to defray responsibility or, more generously, to obtain a second
opinion before acting.
What Can be Evaluated
Everything about a programme is worth evaluating, and there is no restriction on the locus of an
evaluation but the feasibility of doing it. There may be an assessment of the contextual analysis
that is done, the relevance of purpose, aims and objectives, the cost effectiveness of the
programme, the extent to which it met its objectives, the process by which it did this, the
competence of staff, the relationship between materials and objectives, and so on. At its heart,
people expect evaluation to tell them something about the difference the programme made to the
participants or intended beneficiaries.
Such evaluation inevitably takes place, whether planned or unplanned. People develop opinions
about aspects of the programme or its success. These opinions may be well-informed, or they may
be informed by prejudice, a particular limited experience, or a restricted perspective on the
programme. Good evaluations are designed to ensure that the most reliable picture is obtained of
a programme. Educators should ensure a professional rather than merely intuitive evaluation for
all programmes that are complex in their number of components, cost, length, or long-term
implications.
Educational and Programme Evaluation
Educational evaluation has often, therefore, consisted of evaluating the learners and their
achievements or progress over the period of the educational programme. Depending upon the
type of intervention, this can be done in a number of ways. First, this can be done by testing
competence or knowledge through written and oral presentations or examinations. Second,
information might be achieved by collecting baseline data and conducting of tracking polls throughout the course of the intervention tp identify changes in awareness levels, attitudes, and even behaviors. This qualitative data might be further supplemented by qualitative feedback collected through focus groups. Some electoin statistics might also provide useful information. With the case of some polling data and election statistics, however, educators need to keep in mind that it may not be possible to isolate their intervention as the sole reason for a particular result.
Programme evaluation goes beyond the assessment of individual learning. But educational
programmes should not escape the need to ensure that this is assessed. In nonformal educational
programmes this may not be best done by examinations, although tests of competency are essential
for training of educators.
Indicators
Evaluators establish a range of indicators by which they will be able to judge whether or not the
necessary learning has taken place.
Similar indicators are established for other components of the programme. In each case, the
achievement of these is deemed (unless there is evidence to the contrary) an indication that the
programme has been successful. Sample indicators might include: A significant increase in the number of
registered voters, in the number of first time voters, or of voters using a special voting service might all be indicators that an education programme has affected awareness levels and behavior. Similarly, decreases in the number of spoiled or invalidated ballots might also be used as indicators of voters' better understanding of the voting process.
Establishing these indicators is a difficult task and has to be done at the same time as the setting of
objectives. Failure to do this leads to the development of programmes whose objectives are not
able to be evaluated or the development of indicators that are alternatively insignificant or
established after the fact.
Evaluation Controversy
Because evaluation is seen as the important component in validating a programme to outsiders, and
because it contains these technically difficult activities (see Measuring Impact and Importance Of Prior Planning) and such a range
of stakeholder expectations and fears, it can become a highly charged and political
activity especially if the programme has a high stakeholder investment. This could be the case if
it is especially expensive, or if the programme has been endowed with societal significance (for
example a founding election or the tryout of new technology).
Educators should retain control over the evaluative process, over its planning, implementation and
costs. The best way to do this is to ensure that it is built into the programme from the beginning
and does not become an activity imposed upon the programme at some later date. When it
becomes part of the standard operating routine, evaluation becomes a valuable tool. When it is an
afterthought, a management tool for external managers, a decision-making or diagnostic tool
considered at a crisis moment, evaluation becomes a burden and a controversial and difficult
activity.
