Robert Cahaly: How to Spot Bad Polls, Clear Winners, and Predict Elections Accurately
Animals,
Prediction markets matter because polls — those things that are supposed to explain what the public wants from its government — are completely broken.
The polling industry was incorrect by record margins in 2016 & 2020 and the consequences were huge.
Trump’s surprise win in 2016 and near victory in 2020 were enough to allow divisive conspiracy theories to run rampant in the country, thanks to the false narrative — “Democrats in a blowout” — that bad polls were selling.
But while most of the polling industry embarrassed itself, one pollster emerged as a contrarian whose numbers were on-the-mark:
Robert Cahaly, who helms The Trafalgar Group. He joined us today on SSG.
Robert Cahaly: How to Spot Bad Polls, Easy Winners, and Predict Elections Accurately
Robert successfully predicted Trump’s win in 2016 and after the 2020 election, FiveThirtyEight’s Nate Silver praised him as the nation’s 2nd most accurate pollster.
However, as Cahaly has risen in the public discourse, so too have the criticisms from his peers.
High-profile commentators like Nate Cohn (The New York Times), Matt Yglesias (Slow Boring), G. Elliot Morris (The Economist) have all questioned his methodology and with it, the accuracy of his data.
And within the political gambling community, there are some who accuse Trafalgar of worse.
This made for arguably the greatest interview in SSG history.
Literally, almost everything Robert said was quotable. You need to watch the whole thing, but here’s a tease of some highlights:
WHY POLLS MISS
- “People are shamed of their political opinions” and don’t report them accurately
HOW CROOKED POLLSTERS COOK THEIR BOOKS
- They feature “registered voters only” — rather than “likely voters”
- Their surveys… “Ask a ridiculous amount of questions. Because as you continue to ask long questionnaires, you weed out average people and include people who care about politics too much, who tend to skew left.”
HOW HIS METHODS DIFFER
- “I’m not near as much of the huge number cruncher. But I think that’s what’s wrong with modern polling.”
- Most pollsters missed in 2020 because they had “no reward for getting it right. Their reward was for “pushing an agenda.”
WHY HIS INCENTIVES TO BE RIGHT ARE HIGHER:
- “Most of our clients are businesses, organizations, and high networth individuals who want to know what is really going to happen outside of spin. So if we are consistently way off, then we don’t get paid anymore.”
- “I’m not hiding and trying to sell a bunch of nonsense… I’m not working for the corporate agenda. I’m working for the truth.”
IS NATE SILVER ANY GOOD?
- “I’m probably a little more lenient on Nate Silver. Because Nate Silver is an aggregator. He is only as good as the polls that he has to use. So, I think Nate Silver is an excellent statistician. I think that his choice in who he includes in his average leads to most of his problems.”
- “If you’re trying to get a polling average, then having groups that are very one sided make up a disproportionate amount of your polling average is going to skew your average.”
IS REAL CLEAR POLITICS ANY GOOD?
- “I’m constantly frustrated by the fact that there’s only a few of us out there that I think are putting out real numbers.”
WHY DOESN’T TRAFALGAR USUALLY RELEASE CROSSTABS?
- “Our clients… What they pay for is those crosstabs”
This is a big one.
GET ALL THE INFORMATION HERE:
1 thought on “Robert Cahaly: How to Spot Bad Polls, Clear Winners, and Predict Elections Accurately”