A good technology management strategy will include an evaluation and audit strategy to ensure not only that the right technology was implemented according to plan, but that it also continues to operate according to plan throughout its useful life.
Evaluation and audit are essentially two sides of the same coin, however they have different emphases. Audit functions generally encompass an element of external or separate monitoring of an organisation or system. Evaluation, while used as an auditing tool, is not restricted to external or separate scrutiny and can be used by managers and users to measure the success of their own systems and improve their own performance.
Evaluation
Evaluation strategies may be built into the design and implementation strategy of any technology project. What is the technology intended to do? How do we measure whether it is meeting its intended purpose? What outputs of the technology are measurable? What performance measures can we set?
If evaluation issues are considered before technology is purchased, this will help not only to measure its effectiveness after introduction, but it may also help to clarify whether the technology is appropriate for its intended task.
One of the best ways to evaluate the effectiveness of technology is to set realistic performance measures and use them. Traps to fall into include setting performance measures that are not easily measurable, and not collecting or reporting on those measures that have been set.
Evaluation procedures may also be built into the routine maintenance and reporting cycle of technology. Relevant managers should be responsible for ensuring evaluation is undertaken and for receiving that evaluation and acting on it if necessary. There is little point in evaluation if steps are not taken where problems are identified.
Once problems are identified, evaluation results can be used to identify possible solutions to problems and to rectify them. In the next evaluation cycle after improvements are made, evaluation can be used again to judge the effectiveness of the changes.
Audit
As with evaluation plans, an audit strategy can be built into the design and development stage of a technology implementation program.
Auditors take the results of evaluation procedures and apply external or disinterested scrutiny to them. The role of auditors in electoral administration is particularly crucial where technology is used for election processes that require transparency and trust. Auditors can serve to validate electoral procedures by providing an independent assessment of the effectiveness of the technology.
Audits can be undertaken by internal or external staff. Where audits are undertaken internally, it is customary for auditors to not have a direct interest in the process being audited, in order to demonstrate the credibility of the audit. Internal auditors can be valuable because they can be expected to know the intricacies of an organisation's operations. However, internal auditors usually do not have the benefit of the credibility that totally independent external auditors can bring.
External auditors have the advantage of having no interest in the outcome of an audit beyond their own credibility and the fees earned. Consequently external auditors tend to have greater veracity than internal ones. Another point in external auditors' favour is that auditing is a skill, and the best auditors are highly trained and very experienced. This kind of expertise may not be held by internal staff unless those staff are specially recruited for their auditing skills.
External auditors can be used to scrutinise any aspect of electoral technology that is capable of evaluation. In particular, where technology is used for crucial election functions such as recording and tallying votes, external auditors can have a very important role to play.
Computerised systems for recording and tallying votes are only as effective and secure as the computer code used in their calculations. External auditors can be used to scrutinise the code used in these systems and verify that they perform appropriately. Computer code that has been externally audited can then be 'escrowed', that is, kept in secure off-site storage in an independent authority's control, so that the escrowed version of the code can be compared to the 'live' version of the code used for an electoral event. In this way, it becomes possible not only to verify that computer code is free of any hidden flaws or deliberate attempts at manipulation, but also to verify after the code has been used that it has not been changed or tampered with since it was audited.
