This article is made possible by Spotlight PA‘s collaboration with Votebeat, a non-partisan news organization reporting on local electoral administration and voting. This article is available for reprint under the terms of Votebeat’s Republication Policy.
A person approaches the table and takes a 10-sided die. she rolls
It is the turn of the next reel in the row.
The unusual dice, shaped like an elongated diamond, might seem like an oddity to those who use it most: “Board gamers and election officials,” said Jonathan Marks, Pennsylvania’s assistant secretary of state.
But the Rollers, employees of the Pennsylvania State Department, don’t roll dice to determine their next move in Dungeons & Dragons. Instead, they generate a long random number that will determine the course of Pennsylvania’s 2022 risk-limiting audit.
A risk-limiting audit is a type of post-election review designed to provide statistical confidence that an election result was correct. This year marks the first in which all 67 counties must take one of these exams before confirming their election results.
The math used to administer the exam is available to the public, although practitioners agree that it is difficult to understand. Election officials who used it said it took time for the audit to be understood, but they have confidence in it now, and they hope it will also give the public more confidence that election results are accurate.
What is a risk-limiting test?
As a result of a settlement in a lawsuit filed by Jill Stein, the Green Party’s 2016 presidential nominee, the state was asked to roll out risk-limited exams statewide by 2022, according to a fan of the process.
“At its core, it’s a test to confirm that a race’s scope hasn’t been overstated,” Marks said. “What we like about risk-limiting audits from the electoral authority’s perspective is … first, that it’s efficient, and one of the other benefits is that it’s adaptable. And that’s really the big key for me.”
Pennsylvania law has long required counties to conduct an audit of 2% of all ballots or 2,000 ballots, whichever is less, after each election.
But this type of testing has its limitations. For example, a polling officer might choose to only review ballots from one district. While this review might reveal an issue with that particular precinct or the ballots cast there, it won’t reveal much about the election as a whole.
A risk-limiting audit, on the other hand, is specifically designed to check a race’s margin to ensure that the declared winner actually won.
It is also “adaptable”. When a race has a smaller margin, the algorithm requires more ballots to be checked to ensure the result was correct. When the margin is wider, fewer ballots need to be checked.
The 2% audit is also only a local audit and not a statewide audit, meaning that each individual district conducts its own audit that has no impact on other districts, while the risk-limiting audit is a statewide endeavor that looks at competitions as a whole .
“I think county election officials like it because they only have to do as much work as necessary to confirm the outcome of an election campaign, while at a fixed percentage you do the same amount of work regardless of the facts about that election. ‘ said Mark. “In a county [2%] can be more than enough, and in another country it can not be enough.”
One fan is Jeff Greenburg, former longtime Mercer County director of elections and now senior advisor to The Voter Project, a Pennsylvania nonprofit focused on access to voting rights. Greenburg conducted one of the state’s first risk-limiting audits in 2019 as part of a pilot program.
“[Getting] A 95% confidence level could mean you only have to count several thousand ballots across the state,” Greenburg said of how the adaptive part works. “Logically it doesn’t make sense. That’s one of the questions I remember asking the state in 2019. “How do I explain to people that if I’m counting 2,000 ballots versus 200, how do I explain to them that 200 ballots actually give me a better number than the 2,000?” And that’s where the math comes in.”
After going through the process, Greenburg saw his value, and the State Department working group of which he was a member recommended that the Pennsylvania legislature conduct the legally required risk-limiting test. The legislator has not yet followed this recommendation.
Only 15 states currently conduct risk-limiting testing, and only three of them mandate the process by law.
How does it work?
A random 20-digit number called a “seed” is generated by rolling 20 separate ten-sided dice numbered zero through nine. This was done in a live webcast by State Department officials on Nov. 17.
At that point, uniquely numbered lots of ballots from all counties were cataloged in the state’s risk-limiting audit software. The seed is then fed into the risk-limiting verification algorithm to determine which ballots need to be verified.
Mark Lindeman is the Policy and Strategy Director of Verified Voting, a nonpartisan voting technology organization that helps states like Pennsylvania conduct risk-limiting audits. He said while the generation of the seed number is completely random, anyone can verify the work once that number is known.
Because the algorithm is open-source, meaning it’s available to the public, anyone could use this information and the seed to ensure the batches the algorithm dictates are actually being counted. Lindeman said this also prevents any administrator from knowing in advance and potentially manipulating which particular batch is being counted.
Once the lots are known, counties begin their part of the process: a hand count of the ballots in each specific lot to ensure the result is correct. The number of ballots in a batch varies by county based on how administrators organize the ballots.
“The exam math is tolerant of small deviations that are typically observed in hand counting exams,” Lindeman said, as if a ballot was miscounted by a machine due to a smudge on the paper. “Small errors in the machine count would not affect the result, but large discrepancies that cast doubt on the result could require further testing or possibly further investigation, including a full recount.”
This year, the State Department selected the governor’s race for testing. Because the margin of victory between Attorney General Josh Shapiro and Senator Doug Mastriano was very large – nearly 15% – only a few lots needed to be checked to ensure accuracy.
Nonetheless, all districts must be willing to conduct the audit as it is not known which batches will be selected.
A spokesman for the Pennsylvania State Department said the seed dictates that 25 lots be selected, which equates to 10,209 ballots. These batches were distributed across seven counties.
In Chester County, one of the largest in the state, Elections Administrator Karen Barsoum recruited six teams of three comptrollers each, with at least one Democrat and one Republican on each team.
Barsoum organizes their ballots by district and voting type (postal, in-person, etc.) in advance of the post-election period to ensure a smooth verification process. The teams were faced with seven different pieces of paper – each representing one of the decisions in the gubernatorial campaign, including written votes and cases where voters chose no candidate or more than one candidate.
“It’s unclear at the beginning how long it will take,” she said of her system on Friday morning, just before the audit was due to begin. “It depends on how many batches the seeds are telling us to grab.”
It didn’t last very long for Chester County. No batches were selected from the county by the algorithm.
“We’re disappointed,” said Barsoum with a somewhat dejected but contentedly resigned smile. She seemed eager to implement her well-organized, color-coded system.
“We were all ready to go.”
The selected counties have until Friday to upload their results.
Marks, the assistant secretary of state, understands that the risk-limiting test will take time for people to understand, but he hopes it will give Pennsylvanians at least some degree of greater confidence in the process.
“I am not mistaken in believing that [risk-limiting audits] will inevitably convince any skeptic,” he said. “But I’m confident that over time, as people see it work and people see it in action, they will understand it. That is certainly my experience.”
WHILE YOU ARE HERE… If you learned something from this story, pass it on and become a member Spotlight PA so that someone else can contact in the future spotlightpa.org/donate. Spotlight PA is funded by foundations and readers like you committed to responsible journalism that gets results.