Educational programmes will be evaluated, whether professionally or intuitively. The question is,
who will be responsible for planning and conducting the evaluation.
Monitoring is a responsibility of those who manage the programme. They may use a range of
tools to ensure that it is done accurately and adequately. But evaluation implies a distance from
the programme that, it is assumed, cannot be achieved by the staff themselves.
Nevertheless, there are aspects of an evaluation that can be suitably done by staff, and there are evaluation
designs that benefit from staff participation.
Independent Evaluators
External evaluators provide independence and disinterest in the evaluation outcome necessary to
ensure the study produces reliable results. Such results are unencumbered with the bias,
shortsightedness, or defensiveness that comes from those who are too close to the programme.
Outsiders can find it easier to obtain objective comment from participants and access to those who
have been excluded from the programme. They also can provide the additional prestige that an
evaluation requires to be taken seriously and in order to ensure that the recommendations of the
study are implemented.
They will have the time to conduct the evaluation that is unlikely to be available to staff who are
involved in ongoing programmes.
And finally, commissioning external evaluators may be the only way of obtaining the necessary
expertise for the evaluation of a large and complex programme.
Self-Evaluation
Those who are close to the programme bring their own intimate knowledge of the programme and
of the intended and actual outcomes to an evaluation. They have an interest in the outcome, and
in most cases, also a professional commitment to improvement of their work, so that
implementation of evaluation findings is more likely.
Internal evaluations increase the awareness of staff of the programme issues which have to be
considered, such as the cost-effective use of resources, the need for clear objectives, the
importance of collecting appropriate information throughout the programme.
Weaknesses of Both Approaches
- External Evaluators. External evaluators can take a considerable amount of time to
understand the context within which the programme is running, to be able to enter the world of the
programme stakeholders in order to interpret their responses to questions and their reaction to
evaluator reports, and to follow the programme plan and its implementation.
They are costly, although the cost of employing outsiders can be offset by their ability to conduct
the evaluation in a shorter period of time, assuming their familiarity with the context and the
concepts.
They do not have to live with their recommendations, and they do not have to live with the
consequences of their reports. At the worst, this can result in unrealistic recommendations which
do not adequately reflect an understanding of the environment within which a programme has to
be delivered.
- Insiders. Insiders, on the other hand, can get too close to the programme. They have
relationships that have to be protected and personal careers that can be jeopardised. They may
have a vested interest in a particular outcome or recommendation.
As mentioned, they often do not have sufficient time, although it is possible to give the task to a
specific team within the organisation. It may be difficult for them to get reliable and objective
information, either from their colleagues or from programme participants.
Combining Insiders and Outsiders
Effective evaluation requires a combination of insiders and outsiders. It may be that the final
report is prepared by the outsiders, while the insiders act as liaison and facilitation staff.
Alternatively, both insiders and outsiders can declare their interests in the report.
An effective evaluation design encourages stakeholder participation, concentrates on skill
development, and establishes the usefulness of the report and its recommendations, and uses a
variety of data collection and analysis tools that encourage participation and ownership.
In such an evaluation, the roles of evaluators and of all programme stakeholders should carefully
described, and the determination to make use of staff insiders or independent outsiders is based on
which is most effective for the particular evaluation study.
