Measuring electoral quality is like chasing a moving target. Just as election management professionals improve their skills, other factors change and evolve, impacting the quality of elections. The introduction of new technologies (including voting machines and the internet), the use of social media, the role of private sector service providers, public tender, and procurement processes, are only a few of the ever-changing factors. Measuring electoral quality assesses the integrity of an elections' adherence with international standards as the baseline.

The tools and methodologies used for measuring the quality of elections are many and diverse; election observer methodologies, the electoral cycle, performance indicators, and election integrity assessment methodologies are all common practice when measuring electoral quality. The electoral cycle, for example, offers a framework for measuring electoral quality that is universally agreed upon and which organizes all the components of an electoral process into three phases: the pre-election period, the election period and the post-election period. Various scoring systems, both quantitative and qualitative methods are utilized to assess distinct elements of electoral processes. The findings of the various assessments and evaluations of elections are used by election management professionals, political parties and candidates, voters, and election observers to improve their skills and inform their experiences. The practice of supporting and measuring the quality of elections remains a central act in democratic life.

The Measuring Electoral Quality Encyclopedia topic is an introduction to existing methodologies used to assess the conduct of elections under the rubric of “measuring electoral quality.”  Although most of the techniques examined here are well known, they will be grouped into several basic typologies to distinguish the qualities they share, how their methodologies differ and the range of findings they produce. The topic will also introduce the range of individual, national and international actors involved in measuring electoral quality, as well as the various approaches they implement in doing so.  

1.1 Changing Terminology and Contested Definitions

Most people have a personal understanding of democracy and a ‘good’ election but these terms are often difficult to describe, imprecisely defined, and often subjects of disagreement. The United Nations Universal Declaration of Human Rights refers to “genuine elections” but does not provide a working definition of the phrase nor does it provide a breakdown of the indicators or means to achieve them. There is an extensive set of international public law dedicated to giving expression to the fundamental human rights bound in the phrase genuine democratic elections but its application varies considerably by country. Yet, while nearly all countries in the world describe themselves as democratic and defend the quality of their elections there are many different frameworks for evaluating the quality of elections. [1]

“Free and fair elections” is a common phrase employed to judge the quality of an election but it too is subject to a range of definitions and a lack of consensus as to its components and sub-indicators. In the absence of a shared definition or codification of a “good” election, various other phrases have been adopted in efforts to shape a definition (e.g. “open and transparent” or “credible”).  

The phrase “electoral integrity” has also emerged as a concept to define electoral quality. [2] The Kofi Annan Foundation defines electoral integrity as "any election that is based on the democratic principles of universal suffrage and political equality as reflected in international standards and agreements, and is professional, impartial, and transparent in its preparation and administration throughout the electoral cycle.” [3]

The term “standards” is often misunderstood, suggests an imposition of external criteria, or is subject to multiple definitions. [4] In this article, the phrase international legal obligations is used. 

The absence of consensus on the definition of democracy, electoral integrity and indicators or benchmarks to measure them does not invalidate elections and democracy as development goals, but their intangible and contestable terms make it challenging to offer measures that will be accepted (and meaningful to the country and institutions and individuals within it) and applied.

1.2 Framework for Measuring Electoral Quality 

The concept of the electoral cycle helps to organize our thinking about elections since it locates election day within the full context of the many moving parts of an election, reminding us of the importance of electoral indicators such as election management, boundary delimitation, voter registration, the role of political parties, media and security forces and the judicial system as well as the participation of women, minorities or other marginalized groups and so on.

Many individuals who study democracy and elections opt for a procedural definition of democracy with strong emphasis on the electoral aspects and in particular the administration of elections. A procedural definition of democracy simplifies the task of measuring electoral quality since it is can be broken down into constituent parts, phases, events and specific actors.  However, elections remain a complex political activity contingent not only on election procedures but also a bundle of human and political rights, the activities of civil society, the functioning of political parties, the administration of rule of law and justice and so forth.

The use of performance indicators - measures of inputs, processes, outputs, outcomes, and impacts – to assess elections holds appeal for many election stakeholders, especially for those with technical or managerial responsibilities in the electoral process. [5] Establishing performance measures helps individuals and organizations (especially EMBs and technical assistance providers) to set performance targets and assess their progress toward achieving them and to remedy problems when they are identified.

For example, IFES has developed an "Election Integrity Assessment Methodology" that innovatively blends qualitative analysis with a scoring system to measure a range of vulnerabilities and their possible impact in the conduct of elections. The methodology assesses distinct elements of the electoral process (e.g. voter registration, voter education, electoral dispute resolution) and provides assessments of the potential for fraud, malpractice or systemic manipulation. The resulting report can become a planning and management tool for an election management body, technical assistance provider, donor or other electoral stakeholders. [6]

Bridging this set of concerns – the full context of an election as summarized in the electoral cycle, the procedural breakdown of the sub-components of an election and an appreciation for the political context within which an election takes place – to arrive at an overall measure of electoral quality is perhaps beyond the reach of any single methodology. But before reviewing the prevailing approaches, there are other dimensions that bear mention.

First, democracy and governance, including the conduct of elections, have become important foreign policy goals for individual states and multilateral and regional organizations. For example, following the 2009 Lisbon Treaty, the EU is committed to “define and pursue common policies and actions, and shall work for a high degree of cooperation in all fields of international relations, in order to consolidate and support democracy, the rule of law, human rights and the principles of international law.” Many other regional and multilateral organizations share similar goals and may provide various types of election assistance as well as deploy election observation missions. Assessment of democratic governance and the quality of elections has therefore become an important element of statecraft and foreign policy.

Second, international organizations, diplomats, donors and electoral assistance providers also have an interest in monitoring and evaluation of electoral quality to make judgments about how their time and money is being spent.

These two concerns indicate that political and technical considerations are both at play and not only tend to influence one another but they may be difficult to reconcile within a single methodology.

A final consideration is the degree to which an approach to measuring electoral quality balances the twin (and often competing) demands of being comprehensive and being concise. An election observer checklist with 10 questions is likely too few, while 100 questions may be too many to yield data that can be analyzed in the time when observation findings are most effective.

[1] For a useful effort to develop a more rigorous analysis in this regard see, Jorgen Elklit and Andrew Reynolds, “A Framework for the Systematic Study of Election Quality,” Democratization. 12:2, 2005.

[2] For a detailed ongoing research project on electoral integrity see The Electoral Integrity Project.  See also Pippa Norris, Why Electoral Integrity Matters, Cambridge UP, 2014, Pippa Norris, Richard W. Frank, and Ferran Martinez i Coma, eds. Advancing Electoral Integrity, Oxford UP, 2014, Daniela Donno, Defending Democratic Norms: International Actors and the Politics of Electoral Misconduct, Oxford UP, 2013.

[3] Report of the Global Commission on Elections, Democracy and Security, Deepening Democracy: A Strategy for Improving the Integrity of Elections Worldwide

[4] See Nils Meyer-Ohlendorf, “Forget Standards: Why Election Observers Should Talk About ‘International Obligations and Commitments When Evaluating an Election”  Democracy Reporting International, 2010.

[5] For a critical discussion of the uses and misuses of data in democracy promotion see Gerardo Munck, Measuring Democracy: A Bridge Between Scholarship and Politics, Johns Hopkins University Press, 2009. Available online at Google Books.

[6] IFES has conducted electoral integrity assessments in Afghanistan, Georgia, and Pakistan.

Before surveying the various approaches used to measure electoral quality it is useful to identify some of the primary actors interested in these measurements and their objectives.

• Election management bodies (EMBs) have a legal obligation to properly implement the electoral law and will be primarily concerned with the activities of its own personnel and the conduct of their responsibilities in delivering the elections.
• Police and the judiciary may need to collect data on violations of the electoral law and develop monitoring and liaison operations with other actors in the electoral process. The judiciary may have a constitutional or legal responsibility to review and declare election results or to resolve election disputes.
• Political parties and candidates are not only interested in winning or losing elected office but may also have intermediate goals to build their party’s base, extend their presence into new parts of a country, spread their message to new voters or increase their share of the vote. While political parties and candidates tend to focus their energy on the voter register, ensuring their supporters get to the polls, and assessing the conduct of polling and counting operations through the deployment of individual poll watchers (agents or witnesses).
• Civil society and international election observer missions (EOMs from multilateral and regional organizations, governments, international non-governmental organizations and national, non-partisan, civil society organizations) conduct direct observation of some or all parts of the electoral process.
• Election technical assistance providers not only assess the area in which they are working but may also evaluate other areas that intersect with their work (e.g. gender and electoral justice or training of political party agents and the conduct of the polls) as well as the credibility of the overall electoral process. Technical assistance providers want to assess the impact of the assistance to determine what has worked, what may be improved in the future and how processes have changed over time. They may also want to contribute to the ongoing development of best practices in many aspects of election administration.
• Media – domestic and international – combine first and second hand reports gathered from a wide range of sources to cover elections and often commission public opinion polls. The media is mostly driven by reporting on real-time and immediate political and electoral events judged to be most pertinent, but they may also provide important voter information.
• Academics and scholars provide political analysis of many aspects of elections and democracy, through a range of methodologies, including election results analysis, public opinion surveys and focus groups.
• Intergovernmental organizations, governments, and donor agencies participate in election assessment in many ways, ranging from meeting membership obligations to supporting foreign policy to providing funding and other services to the conduct of elections.  They may also participate directly in election observation.
• Individual voters form their own impression of electoral quality, particularly from the point of view of whether or not their preferred candidate or party has won. They rely on a range of sources in addition to personal experience, drawing on talk with neighbors, what they learn from the media, political parties, EMB, government and others. Collectively they form the “public” which is often invoked by others who assess elections and with whom they may or may not have consulted in systematic ways.

At its most basic level, there are two fundamental approaches to measuring electoral quality, which may be implemented in combination or separately by the distinct methodologies described in further detail below:

• Quantitative (e.g. statistical analysis but also systems that use figures to convey information captured through a range of approaches such as numeric indices, scoring systems, public opinion surveys of perceptions and preferences, expert or mass questionnaires, and analysis of election results)
• Qualitative (e.g. normative, narrative basis to assess electoral process, often with legal and/or procedural focus, may use case studies or focus groups, questionnaires, comparative studies across multiple elections and/or regional/international comparisons). These methods seek to capture not only a record of events per se but also judgments and perceptions of these events.

Quantitative measures are often perceived as definitive or objective judgments (rightly or wrongly) and tend to be presented in a numeric scale of score. An election that receives a score of 7.5 out of 10 may be more convincing as a firm judgment than phrases such as “somewhat free and fair” or “largely met international obligations.” Yet both approaches require an understanding of the methodology and data, whether numeric or descriptive, and how the underlying analysis has been applied to arrive at a statement of measure.

Quantitative measures that use properly applied statistical principles are persuasive because the methodology is derived from scientific principles that are demonstrable and universal. If the indicators and the measures are reliable, then different observers watching the same process and using the same tool and data will evaluate that event in the same way. Thus statistical principles can generate more reliable statements about what is being observed. [7] A properly conducted statistical audit of the voter register, for example, can identify finite variations in the accuracy and completeness of the register.

For many quantitative methods, the use of a random sample survey produces quantifiable data that can be statistically analyzed with the aim of measuring, aggregating, modelling and predicting human behavior and relations. A random sample is smaller than the total population or number of cases but it replicates the characteristics of that population. A random sample could be useful to an election official who wishes to test the reliability of the voter register or election observers who wish to verify the vote count.

Qualitative methods typically use narrative analysis, sometimes based on case studies of varying degrees of specificity and may range from the national level to more detailed levels of analysis. Thus while qualitative methods may lack the statistical generalizability of quantitative methods they may be able to provide more in-depth and contextual understanding of the meaning and importance of human behavior, drawing on a range of social science and other disciplines. Qualitative research tools include analysis of various documentary sources, direct observation and participant observation, questionnaires and interviews and other means. These methods seek to capture not only a record of events per se but also judgments and perceptions of these events.

The following survey of approaches to measuring electoral quality includes tools and activities that may be of relatively limited value to assess the conduct of a specific election. For example, while an annual country report might conclude that power has been increasingly concentrated in the executive it might not indicate if that dynamic has affected the conduct of the election management body. Conversely, a technical report that assesses the introduction of a communications system for polling station officials to report the tally of votes may not necessarily include comments on tensions between two dominant political parties. Yet both perspectives – one that is broad and generalized another that is detailed and specific – can contribute to an overall measure of electoral quality.

3.1 Public Opinion Polls

An opinion poll is an application of statistics that elicits responses to questions posed to a randomly selected sample of the population. Opinion polls are usually conducted on the telephone or through in-person interviews. While larger samples are generally preferable to smaller ones (to reduce the margin of error) polls can be expensive and logistically complex activities, limiting their size.

National opinion polls are often conducted throughout the electoral process with a strong emphasis on voter preferences for candidates and parties. Political parties and the media focus particularly on identifying and tracking these preferences over the course of an election campaign. Such polls may also include questions that capture the views of individuals on electoral quality such as public confidence in the EMB, whether or not the individual has experienced voter intimidation or if they have been exposed to voter information about where and how to get on the voter register. [8] Opinion polls can therefore be useful to track popular perceptions of the electoral process and be combined with other approaches to measure electoral quality.

Two ongoing initiatives are:

• Afrobarometer is an independent, non-partisan research project conducted by university and NGO partners that measures the social, political, and economic atmosphere in Africa.  Afrobarometer surveys are conducted in 35 African countries. Because the instrument asks a standard set of questions, countries can be systematically compared and trends in public attitudes are tracked over time. [9]
• Latinobarómetro is an annual public opinion survey that involves some 20,000 interviews in 18 Latin American countries. [10]

In an exit poll, interviewers ask voters how they have voted as they leave polling stations and this data can serve as a form of vote count verification. Exit polls are often used by the media as a means to report early unofficial results and to predict or ‘call’ the election results. In some environments these efforts may help to build public confidence in the official results. However, as a means of verifying or predicting the election results, an exit poll also relies on the assumption of truthful responses from voters. Exit polls may also be controversial, ill-advised or even disallowed for multiple reasons, including: 

1. EMBs and many civic organizations work hard to reassure voters that their ballot is secret;
2. officials want to stem the flow of rumors or politically motivated misinformation that could lead to post-election violence, 
3. exit polls announced before the close of polls in all parts of the country may influence those yet to vote, or 
4. in a very close election even a properly conducted exit poll could prove unreliable or incorrect.

3.2 Democracy Assessment

     3.2.1  Global Democracy Surveys

Several global democracy surveys that score and/or rank countries on a number of indicators are conducted on a regular (often annual) basis. These assessments provide cross-country data, often with a score or ranking associated with them. Democracy and governance, including political rights and the functioning of government, are generally components of these surveys. Some of them are well publicized and though they may bear only indirectly on measures on electoral quality, their prominence suggests they may influence overall perceptions of countries, including the quality of their elections.

In addition, there are many indices that measure specific elements that relate to parts of the electoral cycle such as the Mo Ibrahim African Governance Index or the Civicus Civil Society Index that assess the four dimensions of structure of civil society; the external environment in which civil society exists and operates; the values practiced and promoted in the civil society arena; and the impact of the activities undertaken by civil society actors. The Polity IV Project is an annual assessment of  the authority characteristics of states, ranging from hereditary monarchy to consolidated democracy. The Fragile States Index uses political, social and economic indicators to score and rank countries according to the level of stability – useful for understanding a country’s political, social, and security context for elections. There are also tools that assess a specific element of democracy such as the functioning of legislatures which may be helpful to understand a country’s legislative or institutional environment. [11]

Two of the overall political surveys are described in more detail below:

The Economist Intelligence Unit Democracy Index is an annual index that rates countries based on five categories: electoral process and pluralism; civil liberties; the functioning of government; political participation; and political culture. Each category has a rating on a 0 to 10 scale, and the overall country score is the simple average of the five category indices. The index values are used to place countries within one of four types of regimes: full democracies, flawed democracies, hybrid regimes and authoritarian regimes.

Freedom in the World is an annual survey conducted by the non-profit Freedom House based on a methodology that generates a comparative assessment of global political rights and civil liberties. [12] Each country is assigned two numerical ratings—one for political rights and one for civil liberties—based on a 1 to 7 scale. The ratings are derived from detailed country assessments based on a 40-point scale for political rights and a 60-point scale for civil liberties. The survey also defines the study countries by one of three overall scores of free, partly free, not free.

     3.2.2  National Democracy Assessment

A national democracy assessment is another approach to gathering data on democratic progress and the quality of democracy. In an effort to move away from scores and rankings, International IDEA has developed a framework for democracy assessment that works from fundamental principles of democracy and the mediating values related to those principles and a range of questions about democratic performance. [13]

Multilateral and regional organizations have significant interest in the conduct of elections and democracy among their members. Aside from election observation missions (described below), these organizations may employ other forms of assessment of progress in democracy and governance. The use of peer review among member states is an opportunity to assess the national state of political rights, establish objectives for political and electoral reform and monitor progress through regular follow-up. Peer review therefore has the potential to move individual countries and fellow member states along the path of democratic development. Although the country reports generated by the mechanism may not always yield specific measures of electoral quality (especially when conducted outside of the immediate context of an election), they are guided by shared objectives to advance political rights, strengthen the functioning of government institutions and other areas that have a direct bearing on the electoral cycle and the conduct of elections.

For example, the African Peer Review Mechanism (APRM) was established in 2003 by the African Union guided by the framework of the New Partnership for Africa’s Development (NEPAD). [14] Any AU member state can join the APRM and may launch this regional self-monitoring exercise through an initial base review followed by periodic updates every two to four years or at other times for its own interests. The base document for the APRM provides detailed objectives, methodologies, and indicators for the review and directs members to meet codes and standards established by international and regional agreements including democracy and governance. [15]

For example, the 2013 Zambia country report includes comments that relate directly to the quality of elections including, the absence of legal provisions to regulate political parties and associations, the failure to enforce the Electoral Code of Conduct, the curtailment of the public’s right to political participation, the abusive use of state resources by the ruling party during elections, and unfair media coverage for opposition parties.

3.3 Election Management Assessment

     3.3.1 Post-Election Review

A post-election review (also called a lessons learned activity, audit, or after action review) is an important evaluation activity especially for an EMB but may also be conducted by other actors such as election technical assistance providers or foreign donor agencies if applicable. [16]  For the EMB, not only is a post-election review an important self-assessment of the conduct of the elections, it may also include consultation with political parties, civil society organizations and election observers and may receive verbal and written reports. [17] The review may also convene focus groups or workshops or a large conference. Most EMBs also produce annual reports which summarize their activities for the year and may include the content and recommendations of their self-assessment.

Post-election audits conducted by the EMB routinely check voting system performance, regardless of how close margins of victory appear to be. Post-election audits that detect errors might lead an EMB to conduct a recount or other remedy. For example, in the Afghanistan 2014 presidential election the two leading presidential candidates (Ashraf Ghani and Abdullah Abdullah) agreed to an international audit of all 8 million ballots cast for president candidates to determine the victor (and thereafter form a national unity government). If audits are to be used to increase the quality of elections, just as with election observation and other assessment methodologies, predetermined audit standards should be established, uniform application of the rules must be applied throughout, rights of appeal must be respected and the entire audit process must be conducted transparently.

An EMB may also adopt practices for ongoing monitoring and evaluation of its operations to ensure that its performance matches its responsibilities and the legal framework for elections. In one useful monitoring effort, IFES has generated a set of tools to encourage a range of electoral actors to adopt strategies to assess, deter, and mitigate fraud, malpractices and systemic manipulation throughout the electoral process. [18]

Post-election reviews of all types are challenging for several reasons, including the need for an EMB to plan for the activity before the conduct of the election, to collect data throughout its activities, and manage complex political relations that may need to be considered as part of an EMB’s operating environment.

     3.3.2 Certification of Elections

Certification is a third-party review of the conduct of an election. On rare occasions, the United Nations Security Council or General Assembly may ask the Secretary-General to play a certification role in which the UN certifies the credibility of all or specific aspects of an electoral process. Given the obvious sensitivities of a sovereign state sharing or even handing over such responsibility, the UN undertakes this role with caution. UN guidelines state that certification could include one or both of the following:

• A pronouncement on the legitimacy of the entire electoral process
• A pronouncement on whether the results reflect the will of the voters.

Recently, the UN has been mandated by the Security Council to certify elections in two countries: Timor-Leste in 2007 and Cote d'Ivoire in 2010.  In each case the UN was broadly involved in peace keeping and/or state-building activities in these countries and the holding of credible elections was an integral part of these processes (the UN Integrated Mission in Timor Leste, UNMIT and UN Operation in Cote d’Ivoire, ONUCI).  In Timor Leste, the Security Council mandated the UN "to support Timor-Leste in all aspects of the 2007 presidential and parliamentary process, including through technical and logistical support, electoral policy advice and verification or other means.” In Cote d‘Ivoire, the UN was to certify "that all stages of the electoral process provide all the necessary guarantees for the holding of open, free, fair, and transparent presidential and legislative elections in accordance with international standards".

In recognition of the inherently political nature of the electoral process, the UN guidelines call on the relevant mission to develop the tools to apply such mandates and explain how the certification findings will be determined. Although the specific methodology is to be crafted for the country in question, basic guidelines indicate that the certifying authority should be present in the country throughout the entire electoral process, collect data from all available sources, refer to relevant legal and procedural documents and conduct field visits throughout the country.

3.4 Election Observation

Election observation is perhaps the most widely known effort to measure electoral quality. Though it takes many forms and includes many different approaches, a useful definition of election observation is “the purposeful gathering of information regarding an electoral process, and making informed judgments on the conduct of such process.” [19] In practice, there are many variations in how those judgments are made. [20]

A strong community of practice has developed around election observation; both international and national (often called domestic observers and more recently “non-partisan citizen observers”). These communities have sought to better explain how and why they conduct election observation as a means of improving not only their own transparency (by being clear about their methodology) but also to bolster the credibility of their findings. [21]

Many national legal frameworks for elections provide for the presence of observers, both domestic and foreign, in addition to representatives of the media, political parties and candidates, to ensure transparency. EMBs have also established specific conditions for the conduct of election observation, with best practice emerging for access to all parts of the electoral process.  An important provision includes the need to be invited to observe elections which is especially important for international election observation missions (EOMs) which otherwise may lack standing in the country. It is notable, that in certain established democracies, the legal basis for observers is not established or not applied in all jurisdictions.

It is often assumed that the United Nations plays a principal role in election observation though this is rarely the case (the last UN election observation mission was in Fiji in 2001). The UN does provide a wide range of electoral assistance and generates a great deal of findings related to elections and democracy. Other multilateral and regional intergovernmental organizations observe elections (generally among their member states or in the case of the EU only outside its member states), including (with hyperlinks to respective election observer handbooks or sample observer mission reports):

• European Union (EU)
• Organizations of American States (OAS)
• African Union (AU)
• The Commonwealth
• The Organization for Security and Cooperation in Europe Office of Democratic Institutions and Human Rights (OSCE/ODIHR)
• Economic Community of West Africa (ECOWAS)
• Southern Africa Development Community (SADC)

Many individual governments, regional associations of parliamentarians and election administration officials also conduct international election observation as do many international NGOs and national civil society organizations (on their own or in collective associations). [22]

The following sections describe the basic methodological components of EOMs, including direct observation, legal analysis and procedural assessment. [23] Special thematic assessments follow as a separate approach to measure electoral quality.

     3.4.1 Direct Observation

Most EOMs aim to provide assessment through direct observation of one or more elements of the electoral process through some or all of the following approaches:

• Some organizations deploy a limited number of long-term observers (LTOs) for several weeks (or more) before the elections. LTOs conduct pre-election assessments that may include: voter registration and other election preparations, political campaigns, campaign finance, role of military, police and/or militias, and civic education.
• The best publicized aspect of election observation is the deployment of short-term observers (STOs). While the number of STOs may be determined based on several factors (e.g. budget, size and make up of country, and distribution of polling stations), ideally, STOs will be deployed for maximum coverage, and in some cases, based on a random sample to better generalize their findings. Election day findings are collected on checklists detailing election procedures.
• STOs generally depart the country within a few days of polling though some EOMs will continue to deploy their LTOs to observe any ongoing tabulation of votes, to be present for the announcement of final results and to track any election complaints or disputes that may arise.

The observers may be joined by additional election experts with specific expertise such as legal framework for elections, voter registration, boundary delimitation, gender, media, or other areas (described below in section 3.5).

Most EOMs have one or a small number of individuals to serve as the delegation leader (chief of mission, chief observer or other designation) and principal spokesperson for the mission. The leader(s) may conduct multiple visits to the country before during and after the election but are seldom present on a full-time basis. They meet with national political and civic leaders, and present public findings of the EOM.

The public assessment of EOMs generally pertains to the electoral process only and not to the outcome of the voting. As observers and not supervisors of the electoral process, their judgments are not legally binding on election authorities.

     3.4.2 Rights-based Assessment

A growing number of EOMs have adopted the use of relevant international legal obligations, to provide the basis for independent and impartial assessment of a country’s electoral process by reference to the principles to which the country has already agreed (in contrast to imposing foreign or other standards with which the country disagrees). [24] This analysis generally requires at least one or more experts to collect and review all laws relevant to the electoral process, including the constitutional and regulatory framework for the election, and any other relevant laws (e.g. media regulations, judicial system, etc.). [25]

Important distinctions should also be made clear to differentiate among the various sources used to identify a state’s obligation to a particular principle, including:

• international treaty obligations which are binding on the country;
• regional treaty obligations which are binding on the country;
• international or regional political commitments made by the country which, although not legally binding, are politically persuasive; and,
• the national legal framework for elections of the country.

Take, for example, the right to vote from the Universal Declaration of Human Rights: "The will of the people shall be the basis of the authority of government; this will shall be expressed in periodic and genuine elections which shall be by universal and equal suffrage and shall be held by secret vote or by equivalent free voting procedures.” This right is further expressed and included in many other international treaties and agreements as well as in cultural norms across the world, which therefore apply along with the country’s constitution, electoral legislation and procedures. In this way, the right to vote thus achieves the status of a universal principle as it is supported many times over by multiple sources of international and national law.

The list of legal obligations enshrined in international public law is extensive, including not only the fundamental freedoms (e.g. freedoms of assembly, association, opinion, and movement) and various rights associated with elections (e.g. universal suffrage, secret ballot, right to vote, etc.) but also the right to participate in public affairs, the right to security, the absence of discrimination and the right to a fair and public hearing, among others. [26]

While international benchmarks are helpful, those of multilateral and regional organizations may be more persuasive to member states and hold greater detail about the national means to achieve those principles.  Regional commitments may in turn help to sustain sub-regional or other institutional expressions of democratic rights and obligations. [27] For example, the Organization of American States Inter-American Democratic Charter illustrates the way in which declarations of principle can generate obligations for the state vis-à-vis its citizens and member states. The Charter affirms that democracy is and should be the common form of government for all countries of the Americas, and it represents a collective commitment to maintaining and strengthening the democratic system in the region.

Many regions of the world have adopted democratic obligations for themselves, bolstering the overall regime of international public law (e.g. AU, African Charter on Democracy, Elections and Governance, OSCE, Existing Commitments for Democratic Elections in OSCE Participating States). 

Moving from the general to the specific is an obvious challenge for this approach. To become an effective basis for measuring electoral quality, a rights-based approach must establish the links between a principle, the state’s obligation to uphold that right, the specific elements of the national legal framework for elections, the election procedures or rules, and specific indicators to measure the quality of performance in each step. 

In other words, the rights need to be applied to the procedural steps in the electoral process in order to arrive at a measure of how well the state obligations have been met. Specific indicators need to be developed so that an observer can link the general principle (e.g. secrecy of the vote) to the performance of a specific practice (e.g. polling station layout and other measures to protect vote secrecy).

     3.4.3 Procedural Assessment

As noted above, the use of a procedural definition of elections provides a methodical and chronological template against which one can measure electoral quality. It is listed separately here as not all EOMs who adopt this approach will necessarily ground their assessment in a rights-based approach. The breakdown of an election into its constituent parts has proved useful not only for election administrators who must match human, financial and logistical resources to each part of an election but also to election observers who can develop checklists to evaluate the conduct of each element. Although some categorizations may vary, a list of key areas to be taken into consideration in this evaluation could include:

• Legal Framework for Elections
• The Electoral System and Boundary Delimitation
• Election Management
• The Media
• Parties, Candidates and Campaigns
• Political Finance
• Gender Equality
• Equality of Minorities, Persons with Disabilities and Marginalized Groups
• Voter Education
• Voter Registration
• Voting Procedures
• Vote Counting and Aggregation of Results
• Electoral Dispute Resolution

Cross-cutting themes such as gender can also be applied throughout a procedural assessment, say for example, applying a gender analysis to the EMB itself and how it conducts its operations. [28]

3.5 Specialized Missions: Technical Assessment/Study Mission/ Election Panel

A technical assessment (sometimes called limited EOM, study mission, expert panel, etc.) generally involves the deployment of a small team of experts to conduct an in-depth analysis of one or more elements of the electoral process (e.g. political violence, media, women’s political participation, etc.). These teams may apply one or more of the methodologies used by EOMs (e.g. direct observation, questionnaires and checklists, interviews, rights-based and/or procedural approaches to structure their assessment.

Many of the themes of a technical assessment may also be a component of a broader EOM or a technical assistance provider to the election working in that particular area (for example, developing civic education materials and facilitating public workshops). It is likely that a technical team will provide in-depth and detailed assessment with a limited scope of inquiry. It may also be the case that a technical assessment’s terms of reference closely resemble the broader scope of an EOM. For example, the EU Electoral Assessment Team to Libya in 2010 used the following criteria during its assessment of the electoral process:

• the degree of freedom of political parties and candidates to assemble and express their views;
• the degree of impartiality shown by the election administration;
• the fairness of access to state resources during the election;
• the universal franchise afforded to voters;
• the degree of access for political parties and candidates to the media, in particular the state media;
• the conduct of polling, counting and tabulation of votes;
• other issue related to the democratic nature of the election (e.g. legal framework; campaign violence; rule of  law; campaign finance)

The technical assessment team may receive accreditation as election observers but it is important for all types of these missions to explain their limited mandate and refrain from commenting on the overall electoral process. Should a technical or expert team be deployed to a country where the organization is conducting other activities (e.g. UN Women deploys a gender assessment mission to a country where UNDP is also conducting other election assistance) it is important for such missions to explain the intent and scope of their mission to avoid the perception of a conflict of interest. Unlike election observation missions, some technical teams may choose not to make their findings public, rendering close examination of their methodology more difficult.

Examples of technical or thematic focus in measuring electoral quality (links to methodologies for the assessment) include:

• Media Monitoring –  OSCE/ODIHR Handbook on Media Monitoring for Election Observation Mission, NDI Media Monitoring to Promote Democratic Elections.
• Observation of Voter Registration - OSCE/ODIHR Handbook for the Observation of Voter Registration, NDI, Building Confidence in the Voter Registration Process.
• Observation of Women’s Political Participation - Manual for Incorporating a Gender Perspective into OAS Electoral Observation Missions, OSCE/ODIHR Handbook for Monitoring Women’s Participation in Elections, SADC ESN Gender Checklist for Free and Fair Elections in Southern Africa.
• Assessment of Political Finance – OAS Manual for Observing Political-Electoral Financing Systems . NDI, Money in Politics,  IFES, Political Finance Regulation: The Global Experience, IDEA, Funding of Political Parties and Election Campaigns: A Handbook on Political Finance.
• Observation of Electronic Voting – NDI, Monitoring Electronic Technology in Electoral Processes, The Carter Center Handbook on Observing Electronic Voting.

[7] NDI, The Quick Count and Electoral Observation: An NDI Handbook for Civic Organizations and Political Parties, 2002. 

[8] See, for example, Polling Report, a site that aggregates opinion poll findings in the US on a range of political topics.

[9] For example, in Botswana, the survey has asked “On the whole, how would you rate the last general election?” over the course of 3 elections in a ten-year cycle.

[10] Latinobarometer has asked the same set of questions for nearly 20 years in each country in Central and South America and the Caribbean, including questions of confidence in state institutions such as the judiciary, satisfaction with political parties and the the performance of politicians.

[11] UNDP Benchmarking and Self-Assessment for Democratic Legislatures, and SADC PF Benchmarks for Democratic Legislatures in Southern Africa

[12] See Freedom House, Freedom in the World for detailed survey methodology.

[13] IDEA, Assessing the Quality of Democracy: A Practical Guide, 2008.

[14] As of 2013, 33 of 55 countries have joined the APRM and 17 have submitted to a peer review.

[15] For more detail on the assessment principles and methodology, see “Objectives, Standards, Criteria and Indicators for the African Peer Review Mechanism.”

[16] Jorgen Elklit and Andrew Reynolds, “The Impact of Election Administration on the Legitimacy of New Democracies: A New Comparative Politics Research Agenda,” Commonwealth and Comparative Politics, 40:2, 2010.

[17] See OAS, Quality Management Systems for Election Authorities in Latin America, 2012.  At the time of writing, IDEA and the SADC Election Commissions Forum are drafting a guide to audit methodology.

[18] For a useful series of papers on electoral fraud from IFES, see Chad Vickery and Erica Shein, Assessing Electoral Fraud in New Democracies: Refining the Vocabulary, IFES, 2012, Staffan Darnolf, Assessing Electoral Fraud in New Democracies: A New Strategic Approach, 2011 and Rafael Lopez-Pintor, Assessing Electoral Fraud in New Democracies: A Basic Conceptual Framework, 2010. See also Chad Vickery, ed., Guidelines for Understanding, Adjudicating and Resolving Disputes in Elections, IFES, 2011.

[19] International IDEA, The Basics of Election Observation.

[20] For analysis of election observation, see Eric Bjornlund, Beyond Free and Fair: Monitoring Elections and Building Democracy, Woodrow Wilson Center Press, 2004, and Judith Kelley, Monitoring Democracy: When International Election Observation Works and Why It Often Fails, Princeton UP, 2012. 

[21] Declaration of Principles for International Election Observers and Code of Conduct for International Election Observers and Declaration of Global Principles for Non-partisan Election Observation and Monitoring by Citizen Organizations.

[22] International NGOs who conduct international election observation include the National Democratic Institute (NDI), The International Republican Institute (IRI), The Carter Center, Electoral Institute for Sustainable Democracy in Africa (EISA), the Asian Network for Free Elections (ANFREL), European Network of Election Monitoring Organizations (ENEMO) and others.

[23] There are many handbooks outlining election observation methodology and these tend to echo one another – reflecting the community of practice.  Examples include NDI, How Domestic Organizations Monitor Elections, SADC Parliamentary Forum Election Observation Guide for Members of Parliament, and multiple resources on election observation from the EU and OSCE/ODIHR.

[24] Avery Davis-Roberts and David Carroll, “Using International Law to Assess Elections,” Democratization, 2010 describes the use of a rights-based approach to measure electoral quality and includes an extensive list of references for further reading.

[25] For a detailed review of the legal framework for elections see, IDEA, International Obligations for Elections: Guidelines for Legal Frameworks and NDI, Promoting Legal Frameworks for Democratic Elections.

[26] See The Carter Center website for a comprehensive and searchable database of obligations and commitments that cross-references legal sources with principles and elements of the electoral process. See also IDEA, International Obligations for Elections: Guidelines for Legal Frameworks.

[27] For example, see SADC Parliamentary Forum, Norms and Standards for Elections in the SADC Region and EISA and Electoral Commissions Forum of Southern Africa, Principles for Election Management, Monitoring and Observation in the SADC Region.

[28] IFES Gender Equality and Election Management Bodies: A Best Practice Guide, 2014. At the time of writing UN Women has drafted Inclusive Electoral Processes: A Guide to Electoral Management Bodies and Women’s Participation.

[29] See NDI, The Quick Count and Election Observation and Democracy International, Vote Count Verification.

[30] For example, in the US state of Arizona, the law requires a random sample of voting precincts to be hand counted by election officials after the elections (ballots are machine counted). A Vote Count Verification Committee sets the acceptable variance rate between the machine and hand counts as well as early ballots (advance voting by mail).

[31] IDEA, Atlas of Electoral Gender Quotas and IDEA, The Implementation of Quotas

4.1 Comparative Assessment of Approaches

This section will provide a summary analysis of the approaches to measuring election quality described in the previous section and assess them based on a set of common questions. As with the review of approaches to measuring election quality this will not be an exhaustive assessment but will highlight several key elements to consider for all approaches.

Figure 3 classifies the approaches according to their strength as measures of electoral quality and their scope of inquiry. “Strength” is understood as the degree to which an approach follows a robust methodology, is focused on one or more aspects of an election, and includes a specific rationale for its findings. “Scope” relates to the degree of focus and detail related to electoral quality.

Table 1 provides a summary comparison of the approaches to measuring electoral quality described above against the following key points:

• The nature of the implementers
• Scope and focus
• Methodology for data collection
• Outputs
• Highlights of strengths/weaknesses of approach

 

4.2 Summary Conclusions About Approaches

     4.2.1 Public Opinion Polls

Opinion polls of all types have value in that they report on a range of perceptions on important issues related to democracy and electoral quality.  At the global level they may offer useful comparative insights and at the national level signal key areas of achievement or concern. Their specific value with regard to measuring electoral quality is relatively limited but they can serve as one of several tools that can capture important data about public perceptions. They may capture general public attitudes about democracy and the electoral process but also more specific feedback, for example, to a political party about how well their messages are being received by the public or indicate to an EMB that voter information is being disseminated and understood.

     4.2.2 Democracy Assessments

Global democracy indices provide a macro and comparative perspective on the broad questions of democracy and governance. They also offer the confidence associated with numerical scores and ranking even if those scoring systems rely on a great deal of qualitative (and subjective) analysis.

While a score based on a numeric figure may provide the impression of a solid value, it should also be interrogated.  An obvious challenge for any such framework is deciding on which factors to consider and how to weight them against the other factors included or excluded from the framework. The number of indicators included in a democracy survey (e.g. 10 vs 80) may also affect the level of detail and timeliness of the survey results.  

A second challenge is the level of detail captured by the index. A generalized democracy survey may include measures of fundamental human rights and political freedoms, but relatively few specific measures of electoral quality.

National level democracy assessments hold more potential to provide in-depth measures of the strength of government institutions, the rule of law and operation of the judiciary, respect for individual political rights, the operation of civil society organizations and political parties and the like. However, it is possible but unlikely for a national democracy assessments (whether conducted by intergovernmental organizations, national governments, or non-governmental organizations and scholars) to offer detailed assessment of electoral quality or electoral actors.  This may be a thematic area in which further work could occur.

It is notable that democracy and elections are key elements of so many intergovernmental and regional organizations. In terms of electoral quality, the value of either their EOMs or other forms of periodic review and assessment they may conduct rests with the degree to which they follow a clear methodology and commit themselves to holding one another accountable through follow up on reform and implementation.

     4.2.3 Election Management Assessment

Post-election reviews by EMBs, election assistance providers, and national stakeholders hold a great deal more potential than is currently the case and if fully implemented by for example, not only the EMB but also the judiciary and other government structures they could serve as effective platforms for electoral reform. In the case of donors and election assistance providers, they can provide incentives for policy changes and improvements.

Certification is a tool used infrequently and only in special circumstances such as transitional elections in which an outside actor (e.g. the UN) is involved in a range of peacekeeping and/or state-building activities. The methodology for certification is thus highly contextualized and perhaps unsuited to broader generalization. It draws however on other existing methodologies used by other actors.

     4.2.4 Election Observation

Election observation is subject to many variables that may affect its ability to render a clear and compelling judgment about electoral quality. EOMs require financial backing to mount a long-term, multi-faceted, comprehensive election observation mission with a sufficient number of observers to provide good country coverage. 

There is a long and familiar list of criticisms of EOMs – too long to be detailed here – but it includes questions about the duration of missions, which parts of the electoral process they observe, what training is received by observers, how many observers are present and where they are deployed, what specific methodologies are used for data collection and analysis. The absence of a common set of practices or consistently used measures and use of a common set of openly defined indicators for quality also complicate the picture. Even where multiple EOMs share a consensus judgment, the comparability of their reports may be weak.

Many observer groups have responded with an effort to better systematize and to explain how they assess elections (re: Declaration of Principles for International Election Observers and Code of Conduct for International Election Observers and Declaration of Global Principles for Non-partisan Election Observation and Monitoring by Citizen Organizations). 

Establishing a more formal community of practice enhances the credibility and transparency of election observation methodology and allows for more informed examination of its practices and means to improve them.  

The use of international public law to inform awareness of a state’s international obligations and commitments to democratic elections adds greater rigor to efforts to assess the quality of elections from within a framework of human rights in which the obligations and terms have already been agreed by member states. It provides a set of benchmarks that can be pointed to objectively but they can only be applied when clear lines are drawn that link obligations to national law, practice and implementation through careful data collection.

Most EOMs do not score or weight the different constituent parts of an election nor do they explicitly state that one element – voter registration, for example – is more important than another (e.g. vote counting) and would predominate in the judgment of electoral quality.

While EOMs will assign an informal relative value to the different elements of an election, their methodology has generally avoided assigning weighted or priority values to these elements nor have they provided overall scores.

While a checklist may help an observer to follow the voting procedures and allow the EOM to ensure all observers are reporting on the same elements, arriving at an overall evaluation of the conduct in an individual polling station as well as the election overall continues to rely on a balance of observation and judgment. The use of electronic checklists holds promise to enable EOMs to capture more data more quickly and to subject it to a range of analyses (e.g. to correlate the presence of party witnesses with lodging of complaints and the polling station results).

     4.2.5 Specialized Missions: Thematic Assessment / Expert Panels

Thematic assessment (conducted within or outside an EOM) can provide excellent focus and detail on key indicators of electoral quality and can prove to be a valuable tool for many electoral practitioners. A specialized or technical focus holds the strength of digging very deeply into one or more core components of an election. By the same token, such approaches are unable to comment on the overall quality of the election or areas outside its scope of study. For electoral assistance providers, donors, and beneficiaries such as EMBs, the findings of a technical or specialist mission can identify whether goals have been achieved, what worked and what might need to be changed for the next election. Finally, the findings of thematic and expert or technical missions can provide effective inputs to future policy development.

It is not easy to arrive at the conclusion that the will of the people was reflected in the results without the application of several forms of vote count verification and assessment of the relevant steps in the electoral process. A PVT is an important tool but may not be conducted in all elections. Moreover, a PVT will be unable to account for changes that may occur during the official tabulation (e.g. individual polling station results are disqualified in case tally sheets show signs of alteration, official tally forms are lost or damaged in transit, or there may be political interference by EMB officials or others).  

Careful assessment and observation in practice of the legal framework and procedures to ensure that ballot counting is accurate and free from manipulation are needed.

The following section suggests a small handful of the many questions for further discussion.

How can voters and their candidates for elected office be better served by measuring electoral quality?

Even if an assessment of election quality appears to be “correct” or enjoys consensus among a segment of election stakeholders are there ways to better include voter perceptions of the quality of an election? Services such as Ushaidi show some promise to harness open source software and provide the expertise to moderate and filter large data flows (e.g. public comments on quality of an election or reports of alleged post-election violence). Some EMBs have expanded their public outreach through social media such as Facebook and Twitter and they can now offer relatively low cost platforms for individuals to register their comments (e.g. Facebook accounts for EMBs in South Africa or India). These types of platforms need to be tested in more country circumstances.

Observation of all types, including technical or study missions, should continue to publicize their methodologies and especially take care to define not only how they assess but also to be clear about the scope of what they can and cannot measure.

Political parties and candidates present their own challenges since they tend to be primarily interested in what happens during polling and counting and therefore have a direct interest in real time reports of large scale rigging or other irregularities. While party agents may be trained and present at polling stations, candidates and parties in many countries are unable to make fully effective use of their presence in a systematic manner (e.g. compile reports to confirm a victory or dispute a part of the electoral process). Parties and candidates face the challenge of collecting and making sense of a large amount of data in a short time period and compiling it into a legally compelling case should they choose to submit an election petition.

How can we improve the development of a community of practice in election assessment?

Just as the flow of data and findings about the quality of elections has grown, so have the number and type of actors involved in the conduct and assessment of elections. The proliferation of tools and methodologies makes election assessment an ever-evolving field. 

As noted above, development of declarations of principles from observers is a useful step to clarify their methodology and establish the grounds for a community of practice. This process needs to continue and reach more EMBs and political parties and other key electoral actors with whom EOMs interact. From their side, EMBs should plan for and manage observer access to the full electoral process noting that it can be to the EMB’s benefit to have observers.

Scholars and practitioners have demonstrated interest throughout the world in learning more about the conduct and assessment of elections. Many online resources are available such as the ACE Practitioners Network, an online network of more than 1,000 election professionals and experts where expertise, experience and information may be shared.

There is also a growing number of training opportunities for election management and election observation, including:

• BRIDGE (Building Resources in Democracy, Governance and Elections), a modular professional development program offers a range of training opportunities to election practitioners.
• Basque Country University and several partners organizations offer a one-year online course in International Election Observation and Electoral Assistance.
• The European Commission provides observer training through the Election Observation and Democratic Support along with many individual country training programs.

Associations of EMBs and election officials exist on a global, regional and national scale, and offer a range of services such as conferences and workshops, election observation, skills exchanges, and assistance, including:

• Global Electoral Organization Conference (GEO Conference)
• Association of World Electoral Management Bodies (A-WEB)
• Association of European Election Officials (ACEEEO)
• Association of Electoral Administrators (AEA)
• Association of Asian Election Authorities (AAEA)
• Southern Africa Development Community Electoral Commissions Forum (SADC ECF)

Many other formal and informal associations, training course and other forms of information exchange and support are available through social media such as Facebook and LinkedIn.

What is the impact of efforts to measure the quality of elections?

While election observers offer rapid preliminary findings in the days after an election, followed by a more complete public final report, they are not responsible for the implementation of their recommendations. Measures of electoral quality may shape perceptions of election quality but their ability to effective follow-up is limited. In the case of some inter-governmental organizations there may be debate and even repercussions at a regional level (e.g. in an extreme case a country holding irregular elections could see its membership suspended) but successful follow-up may require domestic election reform and international assistance. Even when an opposition party gains power, their pre-election complaints about the electoral law may not be followed up with electoral reform.

David S. Pottie

Measuring Electoral Quality

full text on one page

Measuring Electoral Quality

Introduction

Who Measures Electoral Quality

Approaches to Measuring Electoral Quality

Assessment of Strengths and Weaknesses of Approaches to Measuring Electoral Quality

Questions for Further Discussion

Contributors