Educational programmes have intended outcomes. They have plans that are being followed in order to achieve these outcomes. These plans consist of a range of components working together to ensure their successful implementation.
This section deals with ways in which monitoring and evaluation take place and some of the dilemmas faced by those who conduct evaluations. It looks at methodological and organisational questions, and at the key question faced by all evaluators: the extent to which programme implementers take advantage of the evaluation recommendations. The section following offers a general overview of evaluation and its relationship to educational testing.
By implementing monitoring plans and evaluating their outcome once completed, educators seek to ensure that they are being accountable to their stakeholders, true to their intentions, and that they themselves will learn from disciplined reflection on the experience of the programme for further work that they might do.
Terminology and Meaning
While it is common to use the twin terminology of monitoring and evaluation, this section merges these terms into the single generic term evaluation, and. It talks about formative evaluation (that conducted with a view to changing a programme while it is in progress) and summative evaluation (that conducted to sum up the programme). But neither is entirely independent of the other.
The meaning of the word evaluation is a slippery one. Those requesting evaluations may be trying to determine the value of a programme by measuring its impact. Evaluators may already have a view of the programme and may be merely seeking external verification of that view. They may be trying to build support for a particular course of action either to defray responsibility or, more generously, to obtain a second opinion before acting.
What Can be Evaluated
Everything about a programme is worth evaluating. thereThere is no restriction on the locus of an evaluation but the feasibility of doing it. There may be an assessment of the contextual analysis that is done, the relevance of purpose, aims and objectives, the cost effectiveness of the programme, the extent to which it met its objectives, the process by which it did this, the competence of staff, the relationship between materials and objectives, and so on. At its heart, people expect evaluation to tell them something about the difference the programme made to the participants or intended beneficiaries.
Such evaluation inevitably takes place, whether planned or unplanned. People develop opinions about aspects of the programme or its success. These opinions may be well-informed, or they may be informed by prejudice, a particular limited experience, or a restricted perspective on the programme. Good evaluations are designed to ensure that the most reliable picture is obtained of a programme. Educators should ensure a professional rather than merely intuitive evaluation for all programmes that are complex in their number of components, cost, length, or long-term implications.
Educational and Programme Evaluation
Educational evaluation has often, therefore, consisted of evaluating the learners and their achievements or progress over the period of the educational programme. Depending upon the type of intervention, this can be done in a number of ways. First, this can be done by testing competence or knowledge through written and oral presentations or examinations. Second, information might be achieved by collecting baseline data and conducting of tracking polls throughout the course of the intervention to identify changes in awareness levels, attitudes, and even behaviours. This qualitative data might be further supplemented by qualitative feedback collected through focus groups. Some election statistics might also provide useful pollinghoweverinformation. However, educators need to keep in mind that it may not be possible to isolate their intervention as the sole reason for a particular result in the case of some polling data and election statistics.
Programme evaluation goes beyond the assessment of individual learning. But educational programmes should not escape the need to ensure that this is assessed. In non-formal educational programmes this may not be best done by examinations, although tests of competency are essential for training of educators.
Indicators
Evaluators establish a range of indicators by which they will be able to judge whether or not the necessary learning has taken place.
Similar indicators are established for other components of the programme. In each case, the achievement of these is deemed (unless there is evidence to the contrary) an indication that the programme has been successful. Sample indicators might include: Aa significant increase in the number of registered voters, in the number of first time voters, or of voters using a special voting service might all be indicators that an education programme has affected awareness levels and behavior. Similarly, decreases in the number of spoiled or invalidated ballots might also be used as indicators of voters' better understanding of the voting process.
Establishing these indicators is a difficult task and has to be done at the same time as the setting of objectives. Failure to do this leads to the development of programmes whose objectives are not able to be evaluated, or the development of indicators that are alternatively insignificant or established after the fact.
Evaluation Controversy
Because evaluation is seen as an important component in validating a programme to outsiders, and because it contains these technically difficult activities (see Measuring Impact and Importance Of Prior Planning) and such a range of stakeholder expectations and fears, it can become a highly charged and political activity, especially if the programme has a high stakeholder investment. This could be the case if it is especially expensive, or if the programme has been endowed with societal significance (for example a founding election or the tryout of new technology).
Educators should retain control over the evaluative process and its planning, implementation and costs. The best way to do so is to ensure that evaluation is built into the programme from the beginning and is not an activity imposed on the programme at a later date.Evaluation becomes a valuable tool. when it forms part of the standard operating routine. On the other hand, when it is an afterthought or used for crisis management, evaluation becomes a burden and a controversial and difficult activity.
