Can AI forecast the presidency? Part II
2024 election fun!
An election where traditional polling is failing.
It’s the morning of the 2024 U.S. Presidential Election. The news has been overrun by a relentless stream of cringe-worthy political ads that have somehow managed to annoy and alienate just about everyone, regardless of political affiliation. But after all the noise, speculation, and endless punditry, today is the day—or at least we hope.
So, who’s going to win?
Like most of our elections here in the US, the result will be determined by an increasingly small number of key battleground states. Strangely, there’s been no clear winner emerging from the polls of these states. Instead, we’re seeing each new poll parroting the results of the last while slowly converging on a 50/50 race. This has raised the eyes of many political pundits who believe that these pollsters are overly manipulating the assumptions in their model as to not be an outlier.
Why? In short, it’s safer in numbers - polling firms may find it safer to reflect the data of other polls than having an outlier that ends up being wrong. This has recently been documented here on Substack by Statistician Nate Silver (writer of the Silver Bulletin and founder of 538), in a long post, where he modeled that even if all 7 swing states are exactly tied, there’s only a 1 in 9,500,000,000,000 probability of so many polls showing such a close race.
So, what good are polls that predict a 50/50 election in predicting the future? Are there alternative forecasting methods? How about something using AI-driven insights to predict the winner?
Non-quantitative election tools for predicting the race
Back in early July, I posted an analysis here on Neural NeXus that ignored the numbers games of polling in favor of something more qualitative, based on Allan Lichtman’s “Keys to the White House” framework. Lichtman’s model has successfully predicted nearly every presidential race since 1860, and I was curious to see how LLMs like ChatGPT and Claude would handle it. The initial prediction? A Trump victory, with the AI models pointing to factors that seemed to weigh against the incumbent, Joe Biden.
A little over a week after that initial post, however, the Joe Biden’s X (formerly Twitter) account announced he was dropping out of the race. Was he reading my substack?? I’m going to take this as evidence for a correct AI election prediction.
Since Joe Biden’s dropout and replacement with Kamala Harris there has been a significant change in the election dynamics. Or has there? It seems like it is entirely worth revisiting this test of AI prediction powers on the outcome of 2024 election. Particularly because we will find out how accurate this is in the coming 24 hours, or at least we hope.
What are Allan Lichtman’s Keys to the White House?
We review this in the previous post, but in short, they are a checklist of thirteen true or false statement reflecting political, economic, and social conditions. These statements are directly asked in reference to the incumbent party candidate, which in this case is Kamala Harris now. True statements favor the election of the incumbent party candidate while false statements disfavor the incumbent candidate. It ahs been historically found that if six or more items on the checklist are false, the incumbent is predicted to lose.
A reminder of the 13 keys:
Incumbent Party Mandate: The incumbent party gains more seats in the House of Representatives in the midterm elections than it did in the previous midterm elections.
Nomination Contest: There is no serious contest for the incumbent party nomination.
Incumbency: The incumbent-party candidate is the sitting president.
Third Party: There is no significant third-party or independent campaign.
Short-term Economy: The economy is not in recession during the election campaign.
Long-term Economy: Real per capita economic growth during the term equals or exceeds mean growth during the previous two terms.
Policy Change: The incumbent administration affects major changes in national policy.
Social Unrest: There is no sustained social unrest during the term.
Scandal: The incumbent administration is untainted by major scandal.
Foreign/Military Failure: The incumbent administration suffers no major failure in foreign or military affairs.
Foreign/Military Success: The incumbent administration achieves a major success in foreign or military affairs.
Incumbent Charisma: The incumbent-party candidate is charismatic or a national hero.
Challenger Charisma: The challenging-party candidate is not charismatic or a national hero.
Revisiting the AI Prediction of the 2024 Presidential Prediction With Harris Against Trump
With Joe Biden out and Kamala Harris stepping in as the Democratic candidate, it’s time to put our initial AI-driven predictions to the test again. This change in candidate isn’t trivial, as it shifts the dynamics of the race and potentially influences the “Keys to the White House” that we used in the original forecast. Let’s see how large language models ChatGPT and Claude 3.5 handle this new matchup under the same framework. This is uncharted territory, but it’s precisely the kind of scenario where AI may have interesting insight.
For this analysis, I ran the Lichtman framework through two AI models—ChatGPT and Claude 3.5. Here’s what each model concluded when considering Kamala Harris as the Democratic candidate:
ChatGPT 4o: The model took the keys framework and highlighted several factors unfavorable for the incumbent party. With Harris, some keys, like “Incumbency” and “Incumbent Charisma,” shifted against the Democrats. ChatGPT leaned toward a Republican win, reasoning that Harris lacks Biden’s incumbency advantage and hasn’t demonstrated the same level of perceived charisma or national recognition that could tilt undecided voters (see below).
Claude 3.5: Claude approached the framework with similar logic but placed slightly more emphasis on economic factors. Despite favoring the Democrats on policies enacted during the Biden administration, Claude pointed to mixed results on economic indicators and the significant challenges Harris faces in swaying undecided voters as a new contender. Ultimately, it also forecasted a Republican advantage, citing the compounded impact of economic ambiguity and Harris’s late entry into the race (see below).
ChatGPT Scoring and Justifications:
Claud 3.5 Scoring and Justifications:
We get to test our hypothesis in real time!
Both AI models predict Donald Trump to defeat Kamala Harris. Perhaps you disagree with these assumptions and results. You wouldn’t be the only one - Alan Lichtman, the developer of the Keys to the White House has predicted the opposite of these AI, that Harris will win.
These large-language models are far from perfect, but it’s still fascinating to see where they align and diverge in their reasoning. Each has distinct approaches to interpreting qualitative data, and their own biases, which adds layers of complexity to their predictions. Do I agree with all their key assessment they make? No - far from it. But that’s part of the experiment. My role here isn’t to bias the results—it’s to observe and see if these tools can genuinely forecast the outcome.
The beauty of this setup is that we’ll have our answer within 24 hours—or, if 2024’s election drama unfolds as expected, by the end of the week.
What do Prediction Markets think?
Prediction markets have increasingly become a popular alternative to traditional polling, especially when polls offer little clarity. These markets are fueled by real stakes—participants are financially invested in the outcome, which some argue leads to more reliable forecasts. In theory, having “skin in the game” forces people to weigh all available information more carefully, making these markets an intriguing reflection of collective sentiment.
For the 2024 election, I examined two major prediction markets: PredictIt and Polymarket. PredictIt, known for its user-friendly interface and accessibility, provides a look at betting odds across different demographics. Meanwhile, Polymarket, with its more advanced trading structure, has seen nearly $3 billion in total volume, signaling serious interest and commitment from traders.
PredictIt: Trump 54c vs Harris 51c - accessed 12:00 CT Nov 5, 2024
Polymarket: Trump 59.7c vs Harris 40.4c - accessed Nov 5, 2024
Here’s where things get interesting. Both platforms show notable optimism for Trump, with Polymarket in particular skewing heavily in his favor. The overwhelming confidence on Polymarket’s end could stem from a variety of factors—some suggest it’s reflective of trader demographics, where libertarians are known for interest in prediction markets and skew conservative, while others point to sentiment bias based on recent polling. Still, the optimism for a Republican victory suggests a divergence between polling and market belief, possibly because traders are factoring in unique elements not captured in the polls, like turnout enthusiasm and last-minute campaign dynamics.
Are these markets more accurate than polls? That’s debatable, but they certainly provide an alternate lens on the race.
AI Qualitative Method and Betting Markets Both Favor Trump
The election is very close and this is merely a fun experiment in predicting an outcome that polling seems to have failed. Check back in as Election Day unfolds. We will find out if AI and the wisdom of crowds got this right or if Harris is being underestimated!
Support:
These newsletters take a significant amount of effort to put together and are totally for the benefit of the reader. If you find these explorations valuable, there are multiple ways to show your support:
Engage: Like or comment on posts to join the conversation.
Subscribe: Never miss an update by subscribing to the Substack.
Share: Help spread the word by sharing posts with friends directly or on social media.
Really interesting experiment! The issue with the keys (as Lichtman interprets them) is examining objective measures of economic health like GDP rather than economic sentiment. If people are unable to buy groceries or make car payments, they are more likely to vote for change—even if that change might result in an even worse economic climate.
Sadly/disastrously, it seems ChatGPT got it right! :-\