A ragtag band of internet friends became the best at forecasting world events

This is very interesting. A discussion of the process of estimating and predicting future events, specifically going into their focus on &quot;base rates&quot;. Apparently they are significantly better than other groups that enter most predicting competitions. (who knew there was such a thing?)<p>I particularly liked the idea of base rates and averaging them. [1] I can see here that the idea of them is not to be on their own accurate (some of the methodologies individually are dumb), but do get an idea for the scales of numbers to talk about (the difference between a 5% event and a 10% event is very hard to notice as a human)<p>[1] &quot;One was the rate at which provinces claimed by China (like Hong Kong, Macau, and Tibet) have eventually been absorbed, peacefully or by force; another was how often control of Taiwan has changed over the last few hundred years (twice; once when Japan took over from the Qing Empire in 1895 and once when the Chinese Nationalists did in 1945); the third base rate used Laplace’s rule. Laplace’s rule states that the probability of something that hasn’t happened before happening is 1 divided by N+2, where N is the number of times it hasn’t happened in the past. So the odds of the People’s Republic of China invading Taiwan this year is 1 divided by 75 (the number of years since 1949 when this has not happened) plus 2, or 1&#x2F;77, or 1.3 percent.<p>Sempere averaged his three base rates to get his initial prediction: 8 percent&quot;

I see a particular problem with this kind of prediction which maybe someone else can explain. If you are predicting one off events like China invading Taiwan, or London being hit by a nuclear weapon, as a percentage, how can you ever know if the prediction was accurate? Whether the event happens or not does not validate or invalidate the percent chance of it happening. Is there some kind of aggregate that happens to all the predictions to validate them? If you give invasion an 8% chance and invasion happens it doesn&#x27;t mean you&#x27;re wrong, just that the world fell into that 8%.

The base rate arguing seems like specious reasoning. For example, if you had a volcano that erupts roughly ever 100 years, base rate reasoning using the past 99 years of data would suggest that the probability is 0 and using 1000 years of data would suggest it&#x27;s ~10% when in reality your base rate in the year following an eruption is 0 with every passing year your probability of an eruption would increase &amp; increase past 10% for every year past 100 that goes without explosion. Same goes for something like war where pressures build up and war becomes more likely rather than less. So getting judged that you&#x27;re better at predicting by giving low probabilities for rare events isn&#x27;t that insightful because you&#x27;d be outperformed by someone who predicts a black swan event because the magnitude of the event matters.<p>&gt; The prediction got some press attention and earned rejoinders from nuclear experts like Peter Scoblic, who argued it significantly understated the risk of a nuclear exchange. It was a big moment for the group — but also an example of a prediction that’s very, very difficult to get right. The further you’re straying from the ordinary course of history (and a nuclear bomb going off in London would be straying very far), the harder this is.<p>Yup, the group got it right but predicting a rare event doesn&#x27;t happen isn&#x27;t that difficult, it&#x27;s just notable because everyone was overly freaked out, particularly in the media due to self-repeated sensationalism. Peter Scoblic is correct that the risk is significantly understated because it&#x27;s not correctly adjusting for the impact of the black swan event happening (e.g. if a nuclear explosion were to occur, you&#x27;d expect nuclear retaliations).

RAND has been in the consensus prediction business since .. well.. RAND was a thing.<p>This is pretty much normal. It&#x27;s just that it carries on under the surface, most people are unaware where &quot;sources state that there is a 20% chance of ..&quot; comments from the military-strategic orbit come from.<p>These % figures aren&#x27;t very meaningful. I personally think they are mis-read more than they are understood. I would not cross the road on a 1 in 10 chance of death. I undertake activity which at least formally has a 1 in 100 chance of negative outcome frequently, and most of us do 1 in 1000 or upward without thinking about it.<p>The Anti-Vax Phobics were rioting on a 1 in 100,000 risk issue.<p>I tend to think China-Taiwan by 2030 is in the 5% bucket as well for a number of reasons. Principally, the join over news stating the Chinese Military is massively corrupt, and there are generals weeding out over-claimed capacity, the lack of visible improvement in their naval fleet beyond one Aircraft carrier, and the massive bright red wave of blood which will stem from an opposed landing from sea, in modern warfare. Blood is a remarkable thing, in terms of it&#x27;s influence on the people at large. China is not Russia, they are nothing like as fatalist about their sons and daughters. An occupation of Taiwan will incur massive loss in the generation which is the one-child policy outcome: Many chinese families will lose their investment in the future. Not to mention the missiles coming back the other way will make this invasion a weapon with two sides, one of which faces coastal investment on the mainland.<p>That and the highly internally facing quality of the rhetoric: Talking up Taiwan is seasonal politics as people jockey for control of the structures. It&#x27;s not preparatory to invasion.

This is what drives me mad about this kind of article:<p>&gt; The team had an answer, and it’s an answer that goes some way toward explaining why this group has managed to get so good at predicting the future.<p>Then a whole section and a half of stuff I don&#x27;t care about to get to the answer:<p>&gt; there was broad agreement that the base rate of war — between China and Taiwan or just between countries in general — is not very high

That reads like a puff piece. There should be a list of correct predictions at the very beginning

For people who are religious, how do you factor in an external entity (omnipotent, benevolent in nature, cares about guiding humanity&#x27;s karma, etc.) weighting the odds of certain events?<p>This is something I&#x27;ve been thinking deeply about for a while now. Curious to hear other people&#x27;s thoughts.

The &quot;base rate&quot; argument is a rather odd one to make. The base rate of war was hasn&#x27;t changed much from 2022, that didn&#x27;t stop Ukraine from happening. Neither did it stop people from calling it. (It certainly didn&#x27;t stop Samotsvety, who made that call correctly as well)<p>And given that each year sees ~50[1] new armed conflicts (for a total of 195 countries total) I&#x27;d posit that the base rate significantly exceeds their estimate of 22%.<p>Yes, of course, a war between China &amp; Taiwan is not just like any war, and there are significant characteristics that influence the likelihood that aren&#x27;t present in other wars, but at that point, talking of a base rate is just misleading.<p>And when you then &quot;average&quot; the &quot;base rate&quot; of hand picked special cases like &quot;how often did control of Taiwan change&quot;, you&#x27;re waving your hands. Wildly.<p>Which means the point isn&#x27;t the idea of base rates per se, but choosing the <i>right</i> base rates - or answering the question &quot;what is relevant here&quot;, and then analyzing from there.<p>The other success factor for forecasting orgs is choosing what you forecast. As such, I have questions about the actual forecasts made being locked behind a paywall: <a href="https:&#x2F;&#x2F;samotsvety.org&#x2F;track-record&#x2F;" rel="nofollow">https:&#x2F;&#x2F;samotsvety.org&#x2F;track-record&#x2F;</a> (It doesn&#x27;t mean they&#x27;re not good at forecasting, but it&#x27;s hard to judge how good)<p>[1] <a href="https:&#x2F;&#x2F;ourworldindata.org&#x2F;conflict-measures-how-do-researchers-measure-how-common-and-deadly-armed-conflicts-are">https:&#x2F;&#x2F;ourworldindata.org&#x2F;conflict-measures-how-do-research...</a>

I assume the people who are actually the best at predicting world events are making huge sums of money in the stock market. Or posting on &#x2F;pol on 4chan. Not entering prediction competitions.

How is this different from the scam where someone sends out stock tips for different companies to different people? The ones that were successful they send out another round of random stock tips. Some of those will be right. Then they can target the people they sent the right ones to and say, look we predicted these so we are successful.<p>Basically, if you have many people predicting something, just from randomness some will be more successful and some won&#x27;t be.

Xi Jinping is on the record, saying they are taking Taiwan. <a href="https:&#x2F;&#x2F;www.nbcnews.com&#x2F;news&#x2F;china&#x2F;xi-warned-biden-summit-beijing-will-reunify-taiwan-china-rcna130087" rel="nofollow">https:&#x2F;&#x2F;www.nbcnews.com&#x2F;news&#x2F;china&#x2F;xi-warned-biden-summit-be...</a><p>The only questions are when and how and what will the rest of the world will do about it.

While I think this is really interesting, I&#x27;m not sure how useful it is. The world doesn&#x27;t change because something may have a 5% chance of happening; it changes because it happened anyway. As a general rule, the most predictable outcomes have the lowest impact.

Vested interest skews your opinions more than if you don&#x27;t care either way. Who knew?

If I wanted to try my hand at this hobby, what’s the best place to get started?

Sorry, is there a TLDR? I found the article difficult to follow in their main argument. Where can we compete with this team? Who and how will evaluate our results?

It is meaningless to put <i>any</i> number on an event like this. Unless you can find the very large number of universes where that same situation arises, you can never know if you were right.<p>Maybe a prediction market-based approach can weigh the <i>beliefs</i> of people about the event, but that doesn&#x27;t mean it&#x27;s a real probability.<p>Probability theory has given people the mistaken belief that everything is like a card or dice game. It isn&#x27;t.

tl;dr<p>Bayes&#x27;

What is the point of forecasting events? The groups who need to act on the events will ignore the forecast or use it to self delude themselves into not acting, others will prepare for events by ignoring the forecasting.<p>Ukraine for example, prepared itself despite forecasts of low probability of an attack. Israel&#x27;s military intelligence on the other hand knew the details of Hamas attack to the point and chose to ignore it despite high probability of the event.