Prediction Markets Beat Polls

This is a shoddy analysis.<p>1. There&#x27;s a statistical gadget specifically for doing this—a &quot;scoring rule&quot; [1] which is a principled way to compare different probabilistic predictions. A bunch of scatterplots of random quantities against each other are... not that.<p>By comparing only binary win&#x2F;loss predictions instead of probabilities, like in the first chart, you throw away <i>almost all information</i> contained in the probabilistic estimates—if Democrats win a state, there&#x27;s no bonus for predicting (say) 95% Dem instead of 55% dem.<p>It&#x27;s plausible that 538 would actually win under a proper scoring rule, because betting markets were underconfident (relative to 538) in deep dem&#x2F;rep states (predicting e.g. &lt;95% Dem win in VT, vs 538&#x27;s &gt;99%). [2]<p>2. The calibration analysis assumes that different state win&#x2F;loss rates are independent, but that&#x27;s really untrue: 538&#x27;s predictions were specifically <i>not</i> independent because they assumed polling errors were correlated between states.<p>3. Many of the other scatterplots look outlier-driven and don&#x27;t include r^2 or p-values. With so few datapoints, it&#x27;s unclear if they are meaningful at all.<p>[1]: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Scoring_rule" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Scoring_rule</a><p>[2]: Maybe we should cut prediction markets some slack here because liquidity constraints make them inaccurate for small probabilities. If that&#x27;s the article&#x27;s position, though, they should address this instead of just... not using a scoring rule.

Why PredictIt isn&#x27;t efficient: smart money is restricted by the amount they can place on any bet. This leaves plenty of inefficiencies in PredictIt. One time, I bought all the contracts saying that every candidate of Democratic nominee would lose and it paid instantaneously. Easiest $50 I ever made.

Spent a lot of time&#x2F;money this election cycle betting on the spread between Predictit and 538. I did act more conservatively by only making a bet when it made sense given all 3 versions of 538s&#x27; model.<p>ROI around 12% (not including 5% withdrawl fee). Expect it to go a few points higher given that called elections are still trading at 90c, but PredictIt won&#x27;t close due to ongoing litigation.

Wait, what? Are you arguing the prediction markets worked? And what about this then? <a href="https:&#x2F;&#x2F;i.imgur.com&#x2F;247IJhe.jpg" rel="nofollow">https:&#x2F;&#x2F;i.imgur.com&#x2F;247IJhe.jpg</a> This was hours after NE-2 was called -- and at that moment Biden won, make no mistake -- and people were <i>still</i> betting against Biden very strongly so I thought I could make a statement here. I wanted to avoid people telling me later &quot;hindsight is always 20&#x2F;20&quot; so I put my money where my mouth is.

Just a second though, betting markets are intrinsically subject to manipulation! Both in the markets themselves and in the world that the gamblers are purportedly &quot;observing&quot;. The betters have active agency in the world they are betting on. The only thing stopping people who participate in these &quot;markets&quot; from manipulating outcomes in unethical, coercive ways is: nothing at all. The betting markets only serve to raise the stakes and create a medium for perverse incentives to play out. So yeah, in some cases or might be more &quot;accurate&quot;, but what about the risks? At what cost?

We like to assume that markets are efficient because if not, there&#x27;s an opportunity for arbitrage, but for betting markets, this just isn&#x27;t true. Predictit, the most often referenced one, has a fee structure that makes it horribly inefficient: they take 10% of all winnings on a given bet and 5% of all withdrawals from the site. The latter actually isn&#x27;t as big of a deal, but the 10% on winnings means that I can&#x27;t make any money off of the spread when you have, for example, a market where Biden has a 88% chance of winning and Trump a 15% chance (which are the odds on the site as of right now). In a market without fees, I&#x27;d buy a Trump NO for 85 cents and a Biden NO for 12 cents. If Biden wins, I make 15 cents on the first bet and lose 12 cents on the second. If Trump wins, I lose 85 cents on the first bet and make 88 on the second. Either way, I&#x27;m up 3 cents. But with a 10% fee on winnings, I make 1.5 cents if Biden wins and I actually lose 5 cets if Trump wins. Then I pay another 5% when I try to withdraw my money (which isn&#x27;t as bad of a fee, because in theory, I would just do a bunch of arbitrage to grow my pile fee-free and then withdraw at the end).<p>What ends up happening is that people on these sites don&#x27;t get any value from betting on events that are 90%+ likely. Which is why the Trump&#x2F;Biden market even puts Trump&#x27;s chances as high as they are. I personally think Trump&#x27;s chances of winning are less than 1%, but I&#x27;d lose money if I bet against him. Meanwhile, some of his biggest fans may not actually think that he has as high as a 15% chance winning, but if they win, they get a 600% return, so what&#x27;s the difference.

Anybody who follows prediction markets will probably see that this doesn&#x27;t mean prediction markets are any good, just that polls are bad. The valuations in prediction markets are simply a summary of what&#x27;s been in the press over the past two hours or so. If the press is terrible, the prediction markets are equally terrible.

Prediction markets ain&#x27;t <i>that</i> great either: <a href="https:&#x2F;&#x2F;thezvi.wordpress.com&#x2F;2020&#x2F;10&#x2F;18&#x2F;predictit-presidential-market-is-increasingly-wrong&#x2F;" rel="nofollow">https:&#x2F;&#x2F;thezvi.wordpress.com&#x2F;2020&#x2F;10&#x2F;18&#x2F;predictit-presidenti...</a>

If anyone followed the betting odds on election night at one point the prediction markets went from heavy Biden favorite to heavy Trump favorite as he won Florida and it looked like he was ahead in a lot of the swing states, but for weeks pundits had been saying it might look like Trump would be ahead in these states early on due to propensity for conservatives to vote in person on election day and for democrats to vote by mail, which turned out to be true.<p>if these markets were actually good predictors I don’t think the wild swings we saw overnight would have happened.

After the first election day, Trump was up on predictit ~60&#x2F;40. This tells me that the betters have no idea what they&#x27;re doing. While Trump was leading in swing states, most metropolitan areas only counted a small fraction of ballots, so the early votes had very little statistical value.<p>Also, there are often tiny arbitrages where buying &quot;no&quot; for Biden would cost a few cents less than buying &quot;yes&quot; for Trump. In theory, you can make a guaranteed 1% return on betting no for both Trump and Biden, and it was as high as 5%. You can&#x27;t take advantage of this arb due to fees on predictit, but you&#x27;d think rational investors would at least try to keep this even.

The betting markets are a function of the polls.

Prediction markets seem to be doing a really terrible job at predicting an election that already happened...<p>Right now predictit.org has Biden winning at 89% and Trump winning at 15%[1].<p>But if you look at it by &quot;electoral margin victory&quot; and add up all the Republican win margins, you get that Republicans have 28% chance of winning and Democrats have 95%.[2]<p>This is two weeks after the election that is pretty clear a Joe Biden&#x2F;Democratic victory. These markets may get things right, but they also get things really, really wrong.<p>[1]Yes, adds up to over 100. You could bet against both for a guaranteed profit, however I believe the markets aren&#x27;t allowing new traders and there are also fees and counter party risk to consider [2] <a href="https:&#x2F;&#x2F;www.predictit.org&#x2F;markets&#x2F;detail&#x2F;6653&#x2F;What-will-be-the-Electoral-College-margin-in-the-2020-presidential-election" rel="nofollow">https:&#x2F;&#x2F;www.predictit.org&#x2F;markets&#x2F;detail&#x2F;6653&#x2F;What-will-be-t...</a>

there are <i>many</i> other things you have to remember when interpreting prediction market data. first, the rules of the contract may be slightly different than your intuitive interpretation based on the market title. second, you need to consider how long to expiry and overall market volume- lots of times the shares will sit at otherwise strange odds because no one wants to park their money somewhere for so long, or because there isn&#x27;t liquidity. third, you have to consider the nature of the market itself- any gambling outfit can tell you how what people <i>want</i> to happen will bias the wagers. and fourth, your are looking at a very restricted view- for instance, right now donald trump has a 13% chance of winning the electoral college on predictit. a study over a longer period of time would tell us more. i&#x27;ve followed predictit for us politics for about 5 years fairly closely, and there are certainly lots of spurious prices. but its also true that polls can be wild outliers. since both methodologies are at least in the same ballpark as far as accuracy, i think you need a bigger more rigorous dataset to make any kind of claim. i believe someone did with the iowa electronic markets back in the day, but i havent read it

While I am an obsessive fan of 538 (I&#x27;ve even ordered a Fivey Fox hoodie), I&#x27;m a little amused at Nate talking down prediction markets.<p>It seems to me that if he thinks his predictions are that much better, he should be massively personally invested in those markets, using his better knowledge to clean up.

False. Betting markets claimed a Clinton victory in 2016. [1]<p>[1] <a href="https:&#x2F;&#x2F;www.realclearpolitics.com&#x2F;elections&#x2F;betting_odds&#x2F;2016_president&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.realclearpolitics.com&#x2F;elections&#x2F;betting_odds&#x2F;201...</a>

This is lacks a strong causal analysis. We still don&#x27;t understand why polls are consistently undershooting Trump support. Given the performance of polls in 2018, it seems highly correlated to Trump&#x27;s presence on the ballot, and unclear if it will recur in a future presidential election. Ascribing better analysis to markets than polls requires us to ask &quot;how?&quot;, and there isn&#x27;t a clear mechanism.

The amount of words that can be wasted on complete nonsense never ceases to amaze.

&gt;You also see betting markets are generally more Republican-leaning in ways that were ultimately born out<p>I think the first half of this is the money quote. The markets are definitely more Republican (or maybe just Trump?) leaning. I&#x27;m not convinced that the second half being true isn&#x27;t just a coincidence.<p>IMO the analysis about how PredictIt bettors followed 2016 polling trends seems to be ascribing them an excess of rationality. I think if you mirror that S-curve diagonally you can see that: The betting markets put really high odds for Trump in states like New Hampshire and Minnesota(which went 55% to Biden but still had 25% odds of going blue). If you look at the mirroring red states which went 55% to Trump, there were no bettors expecting a blue wave (Missouri and South Carolina had ~5% odds of going blue).<p>When you look at markets like &quot;Who Will Win California&quot; or &quot;Who Will Win the Popular Vote&quot;, I&#x27;m convinced that many people making bets are delusional.

Note that predictit gives Biden an 89% chance of winning the electoral college <i>now</i> and Trump 14% (why do the numbers never add up to 100% ?) . The election was two weeks ago. Moreover, I think the author is cherry-picking his data. Nevertheless, I did use predictit during the election as a kind of &quot;nowcast&quot;.

Prediction markets might well have some big players trying to hedge.<p>For example, imagine you are a carmaker. You donate to politician A so he makes carmaker friendly policies if he wins. Politician B refuses your donation, so instead you bet that he&#x27;ll win.<p>You size the donation and the bet so that whatever the outcome, your car company will make the same amount next year. Either through friendly laws increasing sales, or by winning the bet.<p>You&#x27;ve now removed political uncertainty from your business, and you can focus on designing the best cars.

Betting markets <i>currently</i> give Trump a 10% chance of winning this election. On election night, most betting markets had Trump a severe favorite (75-80% in many places), which in retrospect seems severely mis-calibrated.<p>This year betting markets were swamped, and continue to be swamped, by money supporting Trump. This year happened to also have a polling error that underestimated Trump&#x27;s chances. Those two facts aligned together to form a situation where the betting markets seemed to better predict the election results than polls. However, based on one very strange election I wouldn&#x27;t extrapolate out that betting markets are always better than polls.

Didn&#x27;t The Economist&#x27;s model ended up actually doing the best of them all?

Before the election, I applied the theory of “shy Trump voter”. Found the research that about 5% of dem leaning and 10% of gop leaning poll respondents basically lie that they are undecided. I took the average of polls from 538 and allocated undecided voters in accordance with numbers above.<p>The result was very well aligned with actual results and prediction markets. Adjusted numbers predicted that FL, NC, GA, AZ would go to Trump and Biden margins in WI, MI, and PA would be much thinner than predicted by 538.<p>Outside of GA and AZ that turned out to be on point and even in GA and AZ Biden margins were razor thin. Personally I think that Biden has to thank Stacey Abrams for Georgia, and I am not sure what happened in AZ, maybe people are more outspoken there than elsewhere.<p>Now, there are also opinions that there is no such thing as shy Trump voter and my calculation could be just dumb luck.

I <i>also</i> feel like &quot;this analysis can be misleading if you ignore election night itself in which the prediction markets were wildly wrong and had Trump&#x27;s odds of winning as high as 80% while a lot of the models still thought Biden had the edge throughout the entire process.&quot;<p>This seems to be a topic the trolls are heavily invested in, no pun intended, judging by the downvote:comment ratio.

You can make money right now on predictit.org betting that Joe Biden will win the election.

Of course they beat polls. Polls are overwhelmingly operated by partisans in big media that are trying to act as an influence on the vote, not trying to figure out which way it&#x27;s going to go. That&#x27;s the very obvious reason the polls keep missing in such a humiliatingly outsized way. They don&#x27;t care that they miss - otherwise they&#x27;d do the easy thing and adjust properly for factors like intimidated &#x2F; scared &#x2F; threatened voters - when their purpose is to try to manipulate the vote through propaganda.