As discussed under System Verification, Testing and Maintenance, ensuring the accuracy of hardware is a crucial component of a well-functioning system.
While this and the two following sections separate hardware, software and communications into three topics, their operation is often interdependent, and the following verification, testing and maintenance procedures may need to be carried out with all three elements in combination.
Hardware verification
For a highly important system such as an electronic voting system, it is appropriate to employ an independent testing authority to perform system verification tests. For less important systems, system verification could be conducted in-house.
Hardware verification tests (otherwise known as qualification tests) could include:
- testing hardware under conditions simulating expected real-life conditions, including storage, transportation, operation and maintenance environments
- ensuring the hardware conforms with local environmental requirements, including shelter, space, furnishings and fittings, electrical power supply and relevant extremes of temperature, humidity and pollution
- ensuring appropriate documentation is adequate and complete
- verifying that hardware is capable of performing under expected normal conditions and possible abnormal conditions
- ensuring appropriate security measures are in place and that they conform to appropriate standards
- ensuring that appropriate quality assurance measures are in place
Hardware testing
Hardware testing is usually more detailed and thorough than verification. Testing is needed to ensure that every component of a system is operating as it should, and that the system is performing exactly in accordance with the specific local requirements.
A comprehensive structured testing program is one that ensures that all aspects of a system are tested. This is especially important for key systems such as electronic voting systems. Testing measures that could be followed include:
- developing a set of test criteria
- applying 'non-operating' tests to ensure that equipment can stand up to expected levels of physical handling, such as transit drop tests
- examining if appropriate any code 'hard wired' in hardware (this code is sometimes known as firmware) to ensure its logical correctness and to ensure that appropriate standards are followed
- applying functional tests to determine whether the test criteria have been met
- applying qualitative assessments to determine whether the test criteria have been met
- conducting tests in both 'laboratory' conditions and in a variety of 'real life' conditions
- conducting tests over an extended period of time, to ensure systems can perform consistantly
- conducting 'load tests', simulating as closely as possible a variety of 'real life' conditions and using or exceeding the amounts of data that could be expected in an actual situation
- verifying that 'what goes in' is 'what comes out', by entering known data and checking that the output agrees with the input
Hardware maintenance
After hardware has been verified, tested and implemented, it must continue to be maintained. Maintenance routines will vary depending on the type and complexity of the technology. Many hardware items will come with a maintenance schedule or program recommended by the manufacturer or supplier. Maintenance could also be provided by the manufacturer or supplier as part of the purchase agreement.
Systems should be maintained to ensure that they continue to perform to the level demonstrated during the testing stage.
Ongoing monitoring or testing plans may need to be put in place to ensure that maintenance needs are identified and met when necessary. Where systems are in long-term use, a mechanism could be put in place to monitor feedback from users as another means to determine the need for maintenance and modification.
Where modifications to hardware are made as a result of system maintenance or upgrades, it may be necessary to instigate further rounds of system verification and testing to ensure that standards are still met by the modified system.
Reference: Performance and Test Standards for Punchcard, Marksense, and Direct Recording Electronic Voting Systems, [United States] Federal Election Commission, US Government Printing Office, Washington DC, January 1990
