Every number needs a definition.

1. The published AAPOR evaluation

Presidential historical figures reproduce the 2024 task force’s revised Table 2.2.1. The office comparison reproduces main Table 2.1.1. The final interviewing date must fall 0–13 days before or on Election Day. In 2024 that means October 23–November 5. Analytic weights account for duplicates, reposts, and overlapping tracking estimates. State and office estimates are not population-weighted or equal-contest averages.

The raw margin is published Democratic share minus published Republican share. Undecided and third-party shares are not discarded by renormalizing. Positive signed error overstates Democrats; negative signed error overstates Republicans. Mean absolute error averages each estimate’s absolute error, so opposing misses do not cancel. The public replication repository exposes code but not the principal raw analysis file: these tables are transcribed published findings, not an independently reproduced AAPOR dataset.

Selected nonpresidential comparisons quoted on the office pages also come from the revised 2024 report. The report plots additional historical office values without a complete exact table. We do not digitize that image and present the results as exact published numbers.

2. Original calculations from the historical archive

Source polling questions come from FiveThirtyEight’s pinned pollster-ratings archive, licensed CC BY 4.0. National and state presidential records are distinct; Maine/Nebraska presidential congressional districts are excluded from this archive trend. Senate, House district, national House generic, and governor questions remain separate. Only pairs identified as Democratic versus Republican are included in directional comparisons.

For each race and firm, we keep the latest median fieldwork date inside each selected window. Multiple estimates tied on that date are averaged into one firm/race unit. Each unit receives equal weight; partisan and internal polls are retained. This reduces repeated-poll volume effects but does not make firms or races independent. It is our retrospective calculation, not a contemporaneously published polling average or FiveThirtyEight rating.

The final-window chart uses 1–14 days by the median field date. Countdown bins are 1–7, 8–14, 15–21, 22–35, and 36–60 days. The archive lacks consistently usable final-interview and publication dates, so these are not interchangeable with AAPOR’s final-fieldwork window. Each countdown window selects its own estimates. Changing firms, places, and public opinion confound a causal interpretation of “waiting helps.”

Source cycles are not always calendar election years. Odd-year contests remain identified, and January 2021 Georgia runoffs fall in the source’s 2020 cycle. General, special-general, runoff and special-runoff tokens travel with the data. Missing observations and excluded comparisons are never filled with zero.

Outcome matching and known source problems

Federal outcome matching reconciles the candidate pair, state, district, election stage, and year against FEC or election-research returns where a defensible match is available. Votes on multiple party ballot lines must be collapsed to the same candidate. Ambiguous stages, unmatched candidates, and identified corrupt benchmarks are excluded rather than guessed. Governor metrics remain provisional archive-supplied outcomes without independent count reconciliation. All New York and Connecticut governor records and Maine 2010 are excluded. Provisional governor races are excluded from the largest-miss ranking.

The downloadable archive includes the exact current reconciliation ledger, corrections, exclusions, and remaining source-supplied comparisons. An arithmetic check alone does not establish that a source supplied the correct election outcome. These data remain a selected archive, not a census of every public poll or a certified audit of every race.

What the trend can tell us

It shows observed errors among retained estimates in covered contests. It cannot establish that polling technology improved or deteriorated, isolate one method’s causal effect, measure unpolled races, or justify adding the same directional correction to the next election. Frequently polled races contribute more units than sparsely polled races. National House votes also depend on uncontested races and candidate availability; a generic-ballot margin is not a seat forecast.

The generic-ballot benchmark uses first-general-round votes for major-party candidates across the 435 voting districts, excluding DC and territories. Runoff votes are not counted again. This declared round rule can differ from the FEC’s published national party summary when that summary includes an additional runoff. Unopposed contests whose votes were not reported are qualified as missing, not invented as zero candidate support.

3. Earlier Gallup estimates and case studies

The 1936–2012 Gallup chart tracks one organization’s final published estimate. Sampling, mode, turnout modeling, allocation of undecided voters, and final-poll timing changed substantially. Both poll and outcome are normalized to the two named major candidates for that chart only. Earlier result shares are rounded, so calculated errors are approximate. The modern AAPOR raw-margin series stays separate.

Case studies identify whether they compare one poll or a stated aggregate. Their plots retain the stated denominator. The 1936 Digest case uses two-party normalization on both sides; other charts use raw D–R margins unless explicitly labeled. The gallery is a curated set of landmark cases, not a global all-time ranking. A point estimate showing the wrong leader is not automatically a high-confidence forecast of the wrong winner.

The archive ranking includes only independently matched federal races. It uses the absolute error of the mean selected polling margin, at least three firms, and recorded D + R share of at least 90%. It does not rank the worst single survey. This threshold excludes material third-party contests but cannot guarantee every candidate pair or withdrawal is correctly characterized. Causal explanations distinguish documented mechanisms, pollsters’ interpretations, and unresolved possibilities.

Coverage: what is included, and what is still missing

Sampling error is only part of polling uncertainty

A reported margin of sampling error does not automatically include nonresponse, coverage, weighting, turnout-model errors, or subsequent opinion changes. It usually refers to one candidate’s share. In a simple two-candidate random sample, uncertainty in the difference between shares is approximately twice the candidate-share interval. Polls sharing blind spots can remain systematically wrong even when averaged.