Whether an evaluation is done by an outside team or the staff of the programme, it is necessary to write terms of comprehensive Terms Of Reference (TOR) document.
Such a document enables all stakeholders (and a number of them are likely to become involved in the evaluation as recipients of the report, as interviewees, or as providers of core information) to understand and agree to the areas under investigation, the manner in which the enquiry will take place, the time schedule and, when the TOR constitutes a contract with external evaluators, the costs.
A TOR document typically covers the following areas:
- The Background.: A short section setting out the reason for the evaluation and the background to the programme.;
- Primary Purpose of the Evaluation.: A statement descriptive of the overall intention of the evaluation so that it can also be judged according to the extent to which it achieves the goals of those who request the investigation.;
- The Methodology to be Adopted.: The most extensive section which, identifies whether the evaluation is to be primarily qualitative and descriptive, whether it is going to be quantitative and survey based, and the manner in which the evaluators should go about their job. This section can also include decisons about meetings of stakeholders to be convened, whether interviews are to be face to face, group, or telephonic, structured or unstructured.
- The Time Line.: An evaluation typically must be conducted in a very tight time frame. To alert all participants to this, a detailed time frame may be included with the TOR. If this is not possible, then starting and report delivery dates will be specified.;
- Additional Information.: Each evaluation is different. It is likely there will be other conditions that have to be specified. These might include restrictions on who does the evaluation, whether the team includes or excludes staff, and other evaluator qualifications. There may also be instructions about the accessibility of certain stakeholders to the evaluators, and existing data that should be taken into account. and;
- Contractual Obligations.: If the TOR is to be used as a tender document or as a contract between an external evaluator and the education programme, then the TOR will include information about cost, the manner in which the report will be submitted, the obligations of the evaluator to submit preliminary reports, confidentiality and disclosure of information and so on.
A TOR document is an essential tool in conducting an evaluation. It is likely to go through a series of drafts before being accepted by those whose acceptance is required.
Discussion on this matter is also facilitated by the development of a TOR. When an evaluation is determined upon, it might be assumed that the appropriate stakeholders are the educator team, their organisation, and possibly the programme financer. As discussion ensues, other stakeholders can become apparent and the draft can be shown to them.
Care needs to be taken over the development of a TOR document, because it is the document to which evaluators refer during the evaluation. Many evaluations and evaluators are attacked (see Legitimacy and Reliability) and they must inevitably refer to the TOR in their defence.
The more explicit the document, the better for all concerned.
