Educational programmes will be evaluated, whether professionally or intuitively. The question is, who will be responsible for planning and conducting the evaluation.
Monitoring is a responsibility of those who manage the programme. They may use a range of tools to ensure that it is done accurately and adequately. But evaluation implies a distance from the programme that, it is assumed, cannot be achieved by the staff themselves.
Nevertheless, there are aspects of an evaluation that can appropriately be done by staff, and there are evaluation designs that benefit from staff participation.
Independent Evaluators
External evaluators provide independence and disinterest in the evaluation outcome necessary to ensure the study produces reliable results. Such results are unencumbered with the bias, shortsightedness or defensiveness that comes from those who are too close to the programme.
Outsiders can find it easier to obtain objective comment from participants and access to those who have been excluded from the programme. They also can provide the additional prestige that an evaluation requires, to be taken seriously and in order to ensure that the recommendations of the
study are implemented.
They will have the time to conduct the evaluation that is unlikely to be available to staff who are involved in ongoing programmes.
And finally, commissioning external evaluators may be the only way of obtaining the necessary expertize for the evaluation of a large and complex programme.
Self-Evaluation
Those who are close to the programme bring their own intimate knowledge of the programme and of the intended and actual outcomes to an evaluation. They have an interest in the outcome and,in most cases, also a professional commitment to improvement of their work, so that implementation of evaluation findings is more likely.
Internal evaluations increase the awareness of staff of the programme issues which have to be considered, such as the cost-effective use of resources, the need for clear objectives, the importance of collecting appropriate information throughout the programme.
Weaknesses of Both Approaches
- External Evaluators. External evaluators can take a considerable amount of time to understand the context within which the programme is running, to be able to enter the world of the programme stakeholders in order to interpret their responses to questions and their reaction to evaluator reports, and to follow the programme plan and its implementation.
They are costly, although the cost of employing outsiders can be offset by their ability to conduct the evaluation in a shorter period of time, assuming their familiarity with the context and the concepts.
They do not have to live with their recommendations, and they do not have to live with the consequences of their reports. At the worst, this can result in unrealistic recommendations which do not adequately reflect an understanding of the environment within which a programme has to be delivered.
- Insiders. Insiders, on the other hand, can get too close to the programme. They have relationships that have to be protected and personal careers that can be jeopardised. They may have a vested interest in a particular outcome or recommendation.
As mentioned, they often do not have sufficient time, although it is possible to give the task to a specific team within the organization. It may be difficult for them to get reliable and objective information, either from their colleagues or from programme participants.
Combining Insiders and Outsiders
Effective evaluation requires a combination of insiders and outsiders. It may be that the final report is prepared by the outsiders, while the insiders act as liaison and facilitation staff. Alternatively, both insiders and outsiders can declare their interests in the report.
An effective evaluation design encourages stakeholder participation, concentrates on skill development, and establishes the usefulness of the report and its recommendations. It uses a variety of data collection and analysis tools that encourage participation and ownership.
In such an evaluation, the roles of evaluators and of all programme stakeholders is carefully described, and the determination how to make use of staff insiders or independent outsiders is based on what is most effective for the particular evaluation study.
