Can you use Benford's Law to reliably detect election fraud?
This question has been raised often since the U.S. elections on 3 November 2020. The earliest mention of it we can find where it was used to question results in this election dates to 6 November 2020, at this Twitter thread, which featured this infographic looking at the city of Milwaukee's election results in the U.S. presidential race. That was followed by tweets looking at vote tallies in Michigan, Chicago, Allegheny County (Pittsburgh), and other cities and states with long reputations for election irregularities.
Much of the analysis behind these posts appears to have originated with Rajat Gupta's DIY Election Fraud Analysis Using Benford's Law tutorial from September 2020, which provides a guide for how to apply Benford's Law to evaluate election results using spreadsheet applications like Microsoft Excel.
As such tutorials go, it is relatively well done, providing step by step instructions that anyone with basic spreadsheet programming experience can follow to conduct their own analysis to see if their dataset of interest follows Benford's Law and generate a chart showing the outcome. But the tutorial has a huge problem, in that it skips over a huge factor that determines whether Benford's Law can even be successfully applied to evaluate whether the data represents the results of a natural outcome or has potentially been artificially manipulated.
To be successfully applied to detect potential fraud, the dataset must contain values that span several orders of magnitude. In the case of election results from voting precincts, the number of voters in individual precincts would have to number from the 10s, to the 100s, to the 1,000s, to the 10,000s, and so on.
But when you look at the precinct voting data covering the geographies in question, it most often covers data that predominantly falls within a single order of magnitude. This limitation makes Benford's Law an unreliable indicator for detecting potential fraud in this application.
Matt Parker, avid student of math gone wrong and currently the #1 best selling author in Amazon's Mathematics History subcategory, has put together the following video to explain why Benford's Law doesn't work well under these limiting circumstances. It is well worth the 18 minute investment of time to understand where Benford's Law can be successfully applied to detect potential fraud and where it cannot.
Benford's Law has recently been more successfully applied to challenge the validity of the number of coronavirus infections being reported by several nations. The difference in determining its success comes down to the virus' exponential rate of spread, which quickly generates an escalating number of cases that satisfies the requirement that the range of values subjected to analysis using Benford's Law span several orders of magnitude.
With that condition unmet for the raised election examples, we find that using Benford's Law to detect potential voter fraud is unreliable. If the authors of the analyses identifying voter fraud were aware of the deficiency of data used to support their findings, we would categorize these results as the product of junk science. We don't however think that harsh assessment applies for these cases, as most seem to be following an otherwise useful guide that omits presenting the conditions that must be satisfied to properly apply Benford's Law. That makes this situation different from other examples of junk science where the offenders clearly knew their data's deficiencies and failed to disclose them, sometimes with catastrophic effect.
We'll close by providing a short guide to our work covering related aspects of the topics we've discussed and pointing to academic work that provides more background.
Previously on Political Calculations
- When Benford's Law Breaks Down
- Benford's Law and the Trustworthiness of COVID-19 Case Counts
- Prime Numbers and Benford's Law
- The Limits of Microsoft Excel
- The Examples of Junk Science Series
References
Deckert, Joseph; Myagkov, Mikhail; and Ordeshook, Peter C. Benford's Law and the Detection of Election Fraud. Political Analysis, Volume 19, Issue 3, Summer 2011, pp. 245-268. DOI: 10.1093/pan/mpr014.
Mebane, Walter R. New Research on Election Fraud and Benford's Law. Election Updates. [Online Article]. 23 August 2011.
Mebane, Walter R. Inappropriate Applications of Benford's Law Regularities to Some Data from the Presidential Election in the United States. [Online Article (PDF Document)]. 10 November 2020.
Brown, Michelle. Does the Application of Benford's Law Reliably Identify Fraud on Election Day? Georgetown University Graduate Theses and Dissertations - Public Policy. [PDF Document]. 2012.
Labels: election, junk science, math, politics
"Are you better off today than you were four years ago?"
That question first became famous when asked in 1980 by then-presidential candidate Ronald Reagan. Every four years since, polling firm Gallup has asked that question whenever a presidential election is held in the U.S.
In 2020, the year of the coronavirus pandemic and a deep recession, they received a surprising response when asking that question of registered voters:
Gallup's most recent survey found a clear majority of registered voters (56%) saying they are better off now than they were four years ago, while 32% said they are worse off.
Gallup provides a chart showing the graphical results of their polling in the fourth year of the first terms of Presidents Ronald Reagan (1984), George H.W. Bush (1992), George H.W. Bush (2004), Barack Obama (2012), and Donald Trump (2020).
How could that possibly be? Thanks to state and local government lockdown orders that shuttered businesses and required Americans to stay-at-home in late March 2020, the U.S. economy has been experiencing one of the sharpest, deepest recessions in its history, from which it has only begun recovering in recent months as those lockdowns have been lifted. And yet, when asked during the two week period from 14 September through 28 September 2020, a clear majority of Americans stated they and their families were better off than four years ago.
We have unique data that explains that outcome, at least as it applies to the typical American household. Median Household Income is the measure of total money income earned by the American household at the exact middle of the nation's income earning spectrum. 50% of American households have higher incomes, 50% of American households have lower incomes.
Tracking how median household income changes over time can tell us a lot about the state of the typical American household. Not only can it tell us whether nominal incomes are rising or falling with economic conditions, if we adjust it for consumer price inflation, it can tell us a lot about the buying power of the incomes Americans earn.
We've visualized that information in a single chart that shows median household income measured in both these ways.
The chart covers the period from January 2009 through August 2020, which captures the eight years of President Obama's tenure in office and most of President Trump's. With the 2020 presidential election a race between Joe Biden, who served as Vice President during President Obama's terms in office, and Donald Trump, who is running for reelection, it seems most appropriate to focus on this period to evaluate the effect of their respective policies on the welfare of the typical American household.
We see that nominal median household income, shown as the red data series in the chart, declined from $50,608 in January 2009 to $48,559 in early 2010, which then rose at a steady rate through 2016, before accelerating after January 2017. It ultimately peaked at $66,639 in February 2020 as the U.S. economy peaked before the onset of the coronavirus recession in March 2020. Through August 2020, median household income has fallen to $65,602.
The inflation adjusted data, shown as the blue data series, tells a similar story but with meaningful differences. In terms of constant August 2020 U.S. dollars, the Obama-Biden era began with median household income at $62,299, which then fell to $57,209 in early 2011. The buying power of the median income-earning U.S. household then stayed flat until mid-2014, when it finally began recovering.
Inflation-adjusted median household income would go on to slightly surpass its January 2009 level in early 2016, and then largely stagnated for the rest of the year before peaking in December 2016 at $62,440. A few months after President Trump assumed office in January 2017, the stagnation ended and the buying power of the typical American household rose above the levels recorded throughout the Obama-Biden era. The inflation adjusted median household income ultimately peaked at $67,131 in early 2020, but has since fallen with the coronavirus recession to its current level of $65,602.
These outcomes help explain the difference in Gallup's polling results for both 2012 and 2020. For 2020, a clear majority of Americans are answering that they are better off than four years ago because they are better off, even with the negative impact of the coronavirus recession.
Speaking of which, we see indications the recessionary trend for median household income in the U.S. reached a bottom in August 2020. We anticipate September 2020's data will show the first increase in this measure since the coronavirus recession began, as the economic recovery gains traction. The data for September 2020 will become available on 30 October 2020.
Analyst's Notes
Sentier Research suspended reporting its monthly Current Population Survey-based estimates of median household income, concluding their series with data for December 2019. In its absence, we are providing monthly estimates of median household income based upon our alternate methodology. Our references and data sources are presented in the following section.
References
Sentier Research. Household Income Trends: January 2000 through December 2019. [Excel Spreadsheet with Nominal Median Household Incomes for January 2000 through January 2013 courtesy of Doug Short]. [PDF Document]. Accessed 6 February 2020. [Note: We've converted all data to be in terms of current (nominal) U.S. dollars.]
U.S. Department of Labor Bureau of Labor Statistics. Consumer Price Index, All Urban Consumers - (CPI-U), U.S. City Average, All Items, 1982-84=100. [Online Database (via Federal Reserve Economic Data)]. Last Updated: 10 September 2020. Accessed: 10 September 2020.
U.S. Bureau of Economic Analysis. Table 2.6. Personal Income and Its Disposition, Monthly, Personal Income and Outlays, Not Seasonally Adjusted, Monthly, Middle of Month. Population. [Online Database (via Federal Reserve Economic Data)]. Last Updated: 1 October 2020. Accessed: 1 October 2020.
U.S. Bureau of Economic Analysis. Table 2.6. Personal Income and Its Disposition, Monthly, Personal Income and Outlays, Not Seasonally Adjusted, Monthly, Middle of Month. Compensation of Employees, Received: Wage and Salary Disbursements. [Online Database (via Federal Reserve Economic Data)]. Last Updated: 1 October 2020. Accessed: 1 October 2020.
Labels: coronavirus, election, median household income, recession
What is the probability that Senator Norm Coleman is really leading Al Franken in the polls for the U.S. Senate seat now being contested in Minnesota?
To answer that question, we've adapted our tool for finding the odds that a given candidate's lead in the polls reflects what the final outcome of an election will be. That tool reworked the math presented by Supercrunchers' author Ian Ayres in an Excel spreadsheet that, in our view, wasn't the easiest thing in the world to figure out how to use. We did an extreme makeover of Ian's user interface, developing it into a much more user-friendly format!
Today, we're going to rework that tool again to answer our question! First though, let's see where the current vote counts stand:
| Minnesota U.S. Senator Vote Tallies, 15 November 2008 | ||
|---|---|---|
| Candidate | Votes Cast For Candidate | Votes Cast for Others or Unrecorded |
| Norm Coleman | 1,211,565 | 1,708,853 |
| Al Franken | 1,211,359 | 1,709,059 |
| All Others | 462,578 | 2,457,840 |
| All Candidates | 2,885,502 | 34,916 |
Taking the vote tally as of 15 November 2008 as kind of "superpoll," what we're really trying to find is the odds that Norm Coleman will retain his lead after all the ballots ruled valid in the original election count have been recounted.
What makes that less than a sure thing is the possibility of machine error in originally counting the ballots. Here, assuming that all the ballots originally counted for each candidate are valid, we need to take into account the number of ballots for which no vote for any U.S. Senate candidate was recorded, since this provides the pool of votes from which the final vote tallies produced in a recount may be drawn, provided that they contain a clear indication favoring a given candidate. Nate Silver indicates that the number of non-disqualified ballots is 34,916.
Combined with the number of ballots counted for all candidates, 2,885,502, this sum tells us the size of the total population of qualified ballots: 2,920,418. We'll take this total, as well as official vote tallies for both Norm Coleman and Al Franken and use our tool to find the likelihood that Senator Coleman will retain his lead, which has been certified by the state election board of Minnesota:
Using the tool with the vote totals recorded as of 15 November 2008, we find that the probability that Senator Coleman will retain his lead to be 55.8%. We do note that an additional nine votes have been counted in Senator Coleman's favor following an audit of Minnesota's voting machines, which have not been added to his vote total entered in the tool above - you're welcome to modify the values in the tool above to reflect these votes or other potential uncounted vote scenarios!
We find that 55.8% to be a fairly narrow, but positive margin, which perhaps explains why Al Franken has launched a number of lawsuits in an as-yet failing attempt to pad the total ballot count in order to maximize his chances of winning the recount.
Labels: election, politics, tool
Have you ever seen those television shows with the Masked Magician? The ones where he reveals the secrets of how magicians saw people in half, levitate or make elephants appear in the middle of an empty parking lot?
We have to count those shows as one of our guilty pleasures. In fact, it's such a guilty pleasure that we're going to feature the Masked Magician performing, then explaining, how to do a trick he's never done on TV!
We'll begin with the Masked Magician's audience, a thousand people, who each choose a colored marble. All of the marbles are the same size and weight and texture - the only difference between them is their color. There are ten different colors from which the audience members can each choose: red, blue, green, yellow, white, black, orange, violet, tan and pink. They inspect them all, then each choose their favorite color.
Next, they line up to place their marbles in the Masked Magician's magic marble bag, while the Masked Magician's able assistants keep a tally as other audience members look on to make sure there's no funny business. When the last audience member drops their marble into the Masked Magician's magical marble bag, his assistants reveal that of the 1000 marbles the audience has placed into the bag, 420 are red, 419 are blue, 152 are green 5 are yellow, 3 are orange and 1 is violet.
It's sad that the audience members have rejected the white, black, tan or pink marbles. Perhaps they believe those colors just aren't very exciting.
The Masked Magician's assistants then shake the bag, jumbling up all the colored marbles inside. They show the bag's contents to the audience members. And then the Masked Magician arrives.
His assistants blindfold him. The Masked Magician rolls up his sleeve and shows his bare arm to everyone in the audience.
He then reaches into the magical marble bag with his bare hand and removes a single blue marble.
The audience is unimpressed. Surely there's something more to this trick.
He reaches in again, and pulls another blue marble from the bag. And then another. The audience begins to shift uncomfortably in their seats. Where's the magic?
But the Masked Magician continues. One at a time, he reaches in and pulls out a blue marble. After 10, some in the audience are yawning, but some are just getting interested. How is that possible, they wonder? He should have pulled at least one other-than-blue colored marble out of his magical marble bag by now.
The Masked Magician continues only removing blue marbles from the bag. 15. 20. 25. Then 30.
Most of the audience now understands why this trick never made it onto one of his television specials. But some in the audience are truly amazed - there's just no way that someone could pull that many blue marbles in a row out of that bag at random! There's an almost equal number of red ones, and almost one out of seven in the bag are green! Sure, you can understand maybe not seeing the yellow, orange or violet marbles, but shouldn't you have seen a lot of red marbles or at least one green one by now?
What are the odds of that?
What Are the Odds of That?
How amazing is the Masked Magician's ability to only pull blue marbles from his magical marble bag filled with hundreds of marbles of different colors? You can calculate the odds yourself with our tool below:
If you play with the numbers, the more he only pulls marbles of the same color from the bag, the ever more unlikely it is that he should be able to do it! Clearly, you should never let the Masked Magician near the ballot box during a recount!
It's time for the Masked Magician to reveal how the trick was done. How could the Masked Magician defy the laws of probability and pull so many marbles of just one color from a bag holding a thousand marbles of different colors?
The secret is in the bag. Literally.
Hidden within the bag are hidden pockets, sewn into its padded lining. Fans of the show will know he has his suits made with similar features.
But here, long before the first audience member has even begun inspecting the different colored marbles, the Masked Magician has stashed blue marbles into the concealed pockets that are sewn into the bag's lining. Lots of them.
And then it's just a simple matter of appearing defying the odds to keep pulling marbles of the same color out of the bag.
But the odds would only matter if the selection of marbles from the bag were truly random and fair. And as we all know, the Masked Magician is neither random nor fair when it comes to producing an ever more unlikely outcome. The fix was in from the beginning.
Magic in Real Life
There's something potentially just as unlikely happening now in Minnesota, where a U.S. Senate seat is on the line. The table below shows the original tally of that race (as best as we could find in news articles from 5 November 2008), along with the current tally of the mandatory recount that is now underway, as of Sunday, 9 November 2008.
| 2008 Minnesota U.S. Senate Race Vote Tallies |
|---|
| Candidate | Original Tally (5 November 2008) | Current Tally (9 November 2008) | Difference |
|---|---|---|---|
| Coleman | 1,211,629 | 1,211,556 | -73 |
| Franken | 1,211,167 | 1,211,335 | +168 |
| Barkley | 437,377 | 437,385 | +8 |
| Aldrich | * | 13,916 | 0 |
| Niemackl | * | 8,906 | 0 |
| Write-In | * | 2,340 | 0 |
| Cavlan | * | 1 | 0 |
| Evan | * | 0 | 0 |
| Price | * | 12 | 0 |
| Shepard | * | 0 | 0 |
| Total | 2,885,348 | 2,885,451 | +103 |
As we were unable to find the correct vote tallies for the minor candidates in the original tally (indicated in the table above with an asterisk "*"), we've assumed that the current vote tally is the same in calculating the total number of ballots cast in that election.
But to really get good use out of our tool above, it would be best to know the vote counts recorded at the polling place or precinct level. If say 32 uncounted ballots were to turn up in the trunk of the car of a poll worker, you could find out just how unlikely it would be that all of those ballots would be for just one candidate out of all the votes recorded for their polling place.
If so, in the worst case of vote fraud, an extremely unlikely probability would suggest that these ballots favoring one candidate were either pre-positioned before election day to be used in case of a close election or are the remaining evidence of vote suppression, where 32 real ballots are all that remain of a group of roughly 76 ballots, where some 44 legitimately cast for other candidates have been destroyed and never added to their totals (assuming the same proportion of ballots cast per candidate as tallied statewide.) An alternative explanation is that it could be evidence of extreme incompetence on the part of the individual poll worker, their co-workers and those to whom they report.
Finally, here is our table presenting the odds of unlikely events, which have been extracted from Gregory Baer's Life: The Odds and How to Improve Them,
| The Relative Odds of Unlikely Events | |
|---|---|
| Possible Event | Probability of Occurring |
| Being Audited by the IRS | 175 to 1 |
| Writing a New York Times Bestseller | 220 to 1 |
| Dating a Supermodel | 88,000 to 1 |
| Being Struck by Lightning | 576,000 to 1 |
| Getting a Royal Flush on the First Five Cards Dealt | 649,740 to 1 |
| Winning the California Lottery | 13,000,000 to 1 |
| Dying from a Shark Attack | 300,000,000 to 1 |
| Having a Meteor Land on Your House | 182,138,880,000,000 to 1 |
Previously on Political Calculations
- Recounting the Odds
How likely is it that votes previously counted for one candidate are now being counted for another?
Labels: election, probability, tool
Will Franklin finds the political climate being faced by the Republican Party this year to be, in a word, terrible.
But how bad is it really? To quantify the odds, Will turned to the Electoral Barometer, which was recently featured in an article by Alan Abromowitz at Larry Sabato's Crystal Ball '08.
The Electoral Barometer is a mathematical formula that features the things we need to stand back and objectively account for two key factors when looking at the U.S. presidential campaign: the popularity of the incumbent, as measured by their Net Approval Rating (the spread between the President's favorable and unfavorable job approval rating) and the strength of the economy, as measured by the annualized real (inflation-adjusted) growth rate of GDP.
If you know us, you know where this is going! All you need is to enter the appropriate data into the tool below - we'll do the math so you can see how likely the likelihood of either the candidate of the current incumbent presidential party or the candidate of the challenging political party will go on to the White House:
The default data in the tool is current as of 17 June 2008. As such, the GDP data is that for the most current revision of the first quarter of 2008.
In the results above, a negative result indicates that the political climate favors the candidate of the opposing political party, rather than the the candidate hailing from the same political party as the current President. Likewise, a positive result suggests that the incumbent President's political party is favored to win the Presidency in the November elections.
Now, here's the thing. The data that drives this math changes frequently, so you'll definitely want to check back often as the political weather changes!
Labels: election, politics, tool
Last month, Supercrunchers
That sort of thing is the kind of thing that really gets us going, because each posted spreadsheet on the web is an opportunity to convert it into something truly useful and much easier to use!
So, that's what we did! We've taken Ian's spreadsheet, cut out all the noise distracting commentary and boiled it down to its very essence: a tool you can use to find the likelihood that a given political candidate is really ahead given the results of a particular poll! Just enter the indicated data below and we'll find out what the odds are that a given candidate is truly the front-runner (as always, the best source for the latest polling data is RealClearPolitics):
Of course, this probability depends upon the polling sample accurately reflecting the actual voting population. As we've seen as a recurring issue with a number of major polls, including this one that perhaps burned Ian Ayres in his original post on the topic, that isn't necessarily the case!
Occasionally, we get around to living up to our name here at Political Calculations, which we hope explains our latest project!
In a nutshell, we've decided to convert Election Projection's formulas for projecting 2006's U.S. Senate races into our newest tool!. (HT: The Blogging Caesar for developing and posting the math!)
Now, instead of having to rely on "professional" prognosticators, you too can project who will be the winners of 2006's statewide elections, just like the "pros!" You can even play "what if" games to your heart's content - you just need to enter the indicated data into the tool below:
2006 U.S. Senate Election State
State: There's something to be said for how the last presidential election went for the state in question. For this tool, that means you would need to find out the margin by which either Bush beat Kerry or Kerry beat Bush in the state to provide a general sense of the inherent political leanings of the state's voters. Rather than have you look it up and do the math yourself, we built the winning candidate margin for each state into the tool!
You'll note that we've listed all 50 states and not just the ones with U.S. Senate elections - that's because the same formulas work for any statewide race (our built-in bonus for you!)
Incumbent Party Data
In the U.S., incumbency matters. Incumbents not only have an edge going into every election they enter, they can even transfer some of their vote-getting power to members of their party in the case of an open election in which they are not running. This section is where we capture the power of incumbency in an election.
Incumbent Net Job Approval Rating: If a statewide incumbent candidate has a positive net approval rating, the odds are good that they, or the candidate of their party, will win the election.
Incumbent Political Party: There's something else about elections in the U.S. that we didn't mention already - there are only two political parties that matter. It's rare to find a three-way (or four-way, or five-way, etc.) race where there are more than two serious candidates, which is why there's only two choices to select from here. Select the party of the incumbent that best fits the affiliations of the two major candidates in the race (we'd be less vague, but there's that whole Lieberman vs. Lamont thing in Connecticut.)
State Polling Bias: This is an arbitrary adjustment for which you'll need to use your best judgment. First, you need to indicate whether or not statewide polls in the state in question skews in the direction of the incumbent's political party. Next, you'll need to estimate by how much those polls skew the results (in other words, how far off are the polls from the results of actual elections?)
A good example is New Jersey, where polls typically indicate 0.5% more support for Republican candidates than actually shows up in election results. For the state's 2006 senate election, you would indicate that the polls do not favor the incumbent party (Democrat) and you would enter 0.5 to show the amount of state polling bias.
Incumbent Party Candidate Data
Here's where we get into who's actually running! Again, to keep the amount of data entry that you have to do managable, we focus upon the data for the party of the current incumbent in the office.
Name of the Incumbent Party Candidate: Enter the name that will appear on voter's ballots or your own cute concoction! This will be used in text that you can copy and paste elsewhere....
The Most Current Average of Polling Results: For our money, there's no better source for this data than Real Clear Politics, whose polling results and averages for U.S. Senate races is just what the doctor ordered for this tool!
Are They the Incumbent?: As we've said before, incumbency matters. Here's where you identify if the Incumbent Party candidate is, in fact, the Incumbent!
Opposing Party Candidate Data
Name of the Opposing Party Candidate: Just like the name you entered for the Incumbent Party Candidate, enter the name that will appear on voter's ballots or your own editorial creation! This bit of information will be used in text that is provided in the results below, which you may copy and paste elsewhere.
The Most Current Average of Polling Results: Enter the opposing party candidate's RCP polling average!
And that's it - here's where you can enter all the data and find the margin by which one candidate is projected to defeat the other!:
A Quick Example
For no other reason than we can, let's pick on New Jersey again. Here, the incumbent senator, Robert Menendez (D) is running for re-election against opposing party candidate Thomas Kean (R). Incumbent Robert Menendez currently (September 12, 2006) has the following numbers:
Net Approval Ranking: -1% (SurveyUSA)
Incumbent Party: Democrat
Do statewide polls favor the incumbent's political party?: No
How much are the polls off? 0.5%
Incumbent Party Candidate Name: Robert Menendez
RCP Polling Average: 40.0%
Is this the incumbent?: Yes.
Opposing Party Candidate Name: Thomas Kean
RCP Polling Average: 42.3%
We get the following projection:
Thomas Kean is projected to edge Robert Menendez by a margin of 1.16%.
The other races are up to you - go to it, poll junkies!
Labels: election, forecasting, politics, tool
Welcome to the blogosphere's toolchest! Here, unlike other blogs dedicated to analyzing current events, we create easy-to-use, simple tools to do the math related to them so you can get in on the action too! If you would like to learn more about these tools, or if you would like to contribute ideas to develop for this blog, please e-mail us at:
ironman at politicalcalculations
Thanks in advance!
Closing values for previous trading day.
This site is primarily powered by:
CSS Validation
RSS Site Feed
JavaScript
The tools on this site are built using JavaScript. If you would like to learn more, one of the best free resources on the web is available at W3Schools.com.
Other Cool Resources
Blog Roll
