TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

The forecasting fallacy (2020)

91 点作者 rognjen超过 1 年前

21 条评论

diab0lic超过 1 年前
Much of the discussion here including the linked article fail to make an important distinction between domains. Prediction can be done quite effectively on thin tailed processes. A lot of the counter examples listed in the comments here are physical systems which are thin tailed. I see aircraft autopilot, collision detection, ozone depletion. These are all well understood physical phenomenon in which large deviations do not occur — your car doesn’t get teleported elsewhere in the middle of avoiding a collision. If a large deviation did occur, say a meteor striking between your car and the object it is attempting to avoid, the collision avoidance system would almost certainly fail. These events occur so infrequently that the system can just assume they won’t and boast a high success rate.<p>Meanwhile the examples from the linked article are fat tailed processes. Recessions, GDP, interest rates, exchange rates. These are all subject to large discontinuous jumps. Anyone doing a 5 year rate prediction in July 2019 would have been required to predict the pandemic in order to accurately forecast. This is a single example but predictions in this domain are regularly blown out by being teleported to a completely different world. Unlike the thin tailed domain these events happen frequently enough that they’re the only thing that matters for the forecast.<p>Knowing which class your generating process belongs to is critical to understanding whether forecasting will be effective or not. I’ll take collision detection and leave economic forecasts at the door any day.
评论 #38825499 未加载
评论 #38825720 未加载
评论 #38829424 未加载
civilized超过 1 年前
There&#x27;s a germ of insight here that could use some development and nuancing.<p>&gt; The future is uncertain. You cannot predict it. But you can create it.<p>For millions of years, prediction has been <i>the engine by which</i> humans have created the future. We don&#x27;t always call it that, but prediction is the engine.<p>Let&#x27;s start from the most basic facts of life. We know from experience (our own and others&#x27;) what plants will sustain us and what will kill us, so we can predict what present-day choices of food will create a positive future. We create our positive future by making choices in accordance with those predictions.<p>It seems that we need to figure out what separates the kind of prediction that is the engine of human life and progress from the kind that is just useless blather.
评论 #38824982 未加载
评论 #38828656 未加载
评论 #38825770 未加载
kbrkbr超过 1 年前
I think this article has two shortcomings that make its sweeping conclusions shaky.<p>First, it identifies forecasting with point forecasting. There are other ways to put forecasting questions, e.g. lower and upper level with a certain probability.<p>Also it mentions Tetlock, but only his negative findings, not his positive ones that lead to Good Judgement Project, which suggest the contrary of the conclusion of this article [1].<p>Thus I think it is not up to the latest research results.<p>See you over at gjopen.com, if you are interested and have lots of time to waste...<p>[1] <a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;The_Good_Judgment_Project" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;The_Good_Judgment_Project</a>
评论 #38824042 未加载
INGELRII超过 1 年前
A forecasting system in aircraft autopilot that can accurately forecast when the plane hits the mountain is always wrong.<p>Forecasting when the forecast depends on the actions of agents that can be informed by the forecast changes the game. If the Fed model forecasts recession and the Fed takes action to prevent it from happening, it changes everything. Only a forecasting model that is not observed&#x2F;believed by policy makers can predict without intervention.<p>Layman&#x27;s idea of forecasting: Predict what happens in the future.<p>Economic forecasting: Forecast is input for actions. Predict what happens in the future, using this model, these variables, and everything else stay the same. You can check afterward if the model is an accurate forecaster by removing the changes caused by variables outside the model.
评论 #38824630 未加载
评论 #38825610 未加载
cesaref超过 1 年前
If you want a counter example, go and investigate algo trading hedge funds - you&#x27;ll find they do a pretty solid job of predicting the future. Sure, some of them predict only a few ms into the future, some a few minutes (the one I worked for was in that category) and others will do interday strategies.<p>I&#x27;m pretty sure there are examples which have a track record of decent returns above the markets they trade in with longer term strategies.<p>So, i&#x27;d say there are examples of forecasting working, but generally the people who are good at it don&#x27;t write about it, and instead use their knowledge and ideas to quietly make money from their insights :)
评论 #38824625 未加载
评论 #38825754 未加载
评论 #38828677 未加载
sega_sai超过 1 年前
Any forecasts that don&#x27;t involve probabilities or confidence intervals are useless. Also, if any forecasters were really serious, they would register their past forecasts and show how good they have been in the past. But I think most of forecasters are probably afraid of showing their true track record.
评论 #38824356 未加载
wslh超过 1 年前
I like the article but not the final conclusion &quot;We can’t predict much at all&quot;, I think we can predict more than we think, it is similar to asking if the glass is half empty and half full.<p>For example, sci-fi, scientists, and some economists have predicted a lot of things before but we don&#x27;t have an accurate time of happening: one thing is to predict an event for next year and another see a trend that will happen in the next 50 years. There is even a futurist science.<p>Regarding AI, and forgetting &quot;AI cults&quot; it is incredible that the neural networks that we are using now are similar to the ones studied decades ago but there was a breakthrough in other aspects such as computing capacity and techniques [1].<p>[1] <a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;323533a0" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;323533a0</a>
评论 #38826072 未加载
darkerside超过 1 年前
Language is important. We can predict anything and everything. There are some things we can&#x27;t _reliably_ predict.<p>This article ironically suffers from its own thesis. It assumes that because we haven&#x27;t provided some things successfully in the past, we will never predict anything in the future.<p>A simple counterexample should dispel this silly notion. We used to consider the weather completely unpredictable. Now we have elaborate systems and theories that allow us to predict the weather with at least some accuracy.<p>A more reasonable thesis might be, we can&#x27;t reliably predict human behavior, because much like the uncertainty principle, each prediction that is published, which it must be to be meaningful, affects the behavior it is trying to predict.
Zolde超过 1 年前
One should ask economists what a recession is, not how to predict one. Good modelers do not necessarily need (or want) to know what they are predicting and still beat &quot;domain experts&quot;.<p>Authority without clear track-record is a net negative to getting good results. It is better to stick to anonymity, and only let the track-record do the talking&#x2F;weighting. Without a clear track-record it does not even matter if the prediction-maker has skin in the game. If you do have skin in the game, there is no reason to sell your hide cheaply, or even give it away. You instead take the profit others say does and can not exist beyond &quot;luck&quot;: If you can&#x27;t even beat a random walk, you have no business evaluating the limitations of predictive modeling.<p>The big consultancy companies making bold predictions don&#x27;t even need to be right. Customers read the predictions these consultancy companies peddle, because these customers are not bold enough to make their own predictions. And nobody ever got fired for buying the predictions from big consultancy companies and incorporating them into a business strategy.
评论 #38824589 未加载
评论 #38824527 未加载
hardlianotion超过 1 年前
Everyone that provides a forecast that others depend on should really be on the hook to report on the outcome, and to provide the forecast error distribution, if the forecast is one they make regularly.
评论 #38829860 未加载
hamilyon2超过 1 年前
Isn&#x27;t inability to accurately predict some economic metrics consequence of efficient market hypothesis?<p>All available and some unavailable information is already reflected in market. So, sum of reasonable guesses of next year GDP more or less <i>is</i> today&#x27;s market index. Anything over that is some baseless speculation with no skin in the game.
评论 #38826354 未加载
评论 #38825580 未加载
评论 #38824970 未加载
lordnacho超过 1 年前
The critical distinction is IMO that the examples he gives are all examples of where the prediction forms part of its own outcome. Whether interest rate go up depends on whether people think interest rates will go up. Same with GDP and all the other examples.<p>Having this recursion is a massive problem, because there might not be a stable point where the prediction and its outcome are fixed.<p>This may be why it&#x27;s hard to steer cars automatically. I don&#x27;t know enough about the subject, but it would seem to fit in the category of &quot;fields where what you think affects what happens&quot;.<p>By contrast, a lot of other phenomena are complicated but do not have this prediction-feedback effect. The weather, astronomical observations, a lot of engineering systems.
philipswood超过 1 年前
What?!<p>The article ends with a Alan Kay quote attributed to Cindy Gallup:<p>&gt;Or as Cindy Gallup likes to say:<p>&gt;<p>&gt;“In order to predict the future, you have to invent it&quot;.<p>So she does like to say it (see quote below), but it seemed strange to end with a &quot;second hand&quot; quote.<p>From <a href="https:&#x2F;&#x2F;www.lbbonline.com&#x2F;news&#x2F;5-minutes-with-cindy-gallop" rel="nofollow">https:&#x2F;&#x2F;www.lbbonline.com&#x2F;news&#x2F;5-minutes-with-cindy-gallop</a><p>LBB &gt; ‘In order to predict the future, you have to invent it’, Alan Kay, is reportedly your favourite quote. Why?<p>CG &gt; Because I am all about inventing the future. Too many people feel that the future is something that happens without us, that we have no control over, that simply rolls us over in its wake. I believe in deciding what you want the future to be, and then inventing it.
skippyboxedhero超过 1 年前
You can predict what will occur quite easily, you can&#x27;t necessarily predict when it will occur.<p>A lot of the &quot;failed&quot; predictions relate to markets...the reason why you can&#x27;t predict this stuff is because humans are irrational and those irrational humans control outcomes in the short and medium term.<p>For example, you can see that a recession should have occurred. What people didn&#x27;t expect is fiscal stimulus worth about 50% of GDP, tens of trillions in monetary stimulus, etc. Yes, if the government just deposits hundreds of billions into people&#x27;s bank accounts then it is going to impact growth.<p>I remember back in 2007, Blackstone RE made insane leveraged bets at the very top of the market, it is very easy to point out rationally &quot;these are absolutely terrible investments, the price is awful, these aren&#x27;t economic&quot;...today, all these bets got bailed out by the government (after a short period of bankruptcy&#x2F;restructuring), the person responsible is probably going to be made head of Blackstone, that unit has hundreds of billions in AUM, etc.<p>The assumption that people make with forecasts has to be: the long-term is today. That is it. You will often be wrong but that does not mean that your model is wrong (indeed, the reason why this stuff is so predictable is because people believe that the models have stopped working repeatedly).<p>If you take something as apparently &quot;unpredictable&quot; as the market, you can predict returns to within 10bps very easily over the long-term because the fundamentals do not change (but, again, the current period has been the most unpredictable because of the level of government intervention, it is unprecedented...the government cannot hold back the waves forever though).<p>EDIT: referencing the 2005 interest rate prediction is quite humorous too, 2% against a predicted 5%...this was basically the start of it. Back then, no-one thought the Fed would cut rates to this level for, essentially, no reason...the Fed cut, the result was a financial crash. Turns out those predictions (which were essentially the long-term neutral rate) were right and the Fed was wrong...but the only account you hear about is: those damn forecasters, they failed to predict the Fed torching the economy, so stupid. Lol.
FergusArgyll超过 1 年前
Some questions from an economic analysis standpoint.<p>If someone can predict the future reliably, why don&#x27;t financial firms hire them? If a firm did hire them, how much money are they getting paid? why so little? Is the economic value of correct predictions lower than you&#x27;d think? Does the market believe &quot;past performance is no guarantee of future results&quot;?
jxf超过 1 年前
The author would also conclude:<p>* Collision avoidance systems are terrible at forecasting collisions because they almost never result in a collision. (The point of the system is to help you avoid an upcoming collision.)<p>* The prediction that Y2K would happen was a bad one since it didn&#x27;t happen. (We spent billions of dollars to make sure it didn&#x27;t.)<p>* The 1978 prediction that the ozone layer would be depleted by 2010 was a bad one since it didn&#x27;t happen. (Humans took action to reverse CFCs and the ozone layer began to regenerate.)<p>When you make a forecast about an event wherein agents can change the course of the event, the correct evaluation of the forecast is not &quot;did the event happen?&quot; but &quot;would the event have happened but for intervention?&quot;.<p>The author seems to miss this larger point.
评论 #38824843 未加载
评论 #38825026 未加载
评论 #38825690 未加载
dang超过 1 年前
Discussed at the time:<p><i>The Forecasting Fallacy</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24521279">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24521279</a> - Sept 2020 (9 comments)
jncfhnb超过 1 年前
Ugh. This annoys me.<p>Can we predict things super accurately? Often no. But you know what’s better than anecdotes about times predictions were bad? Training and testing sets to judge how good we expect models and predictions to be from the beginning. Because a lot of these are not high confidence predictions.<p>And no, it’s not “black swans”. Are those a thing? Sure. It’s ok that models can’t account for things that are not modeled or seen before. But if these things are common enough that they’re systemic and the mode is just not actually accommodating for the world of relevant factors, then it’s not going to have been a good model on the test set to begin with. And we would know that.
jaygray0919超过 1 年前
I just read Hari Seldon to find out what will happen in the future.
paulpauper超过 1 年前
does this matter if they cannot predict? Doctors cannot predict who gets heart attacks or cancer but they are still needed anyway.
hk__2超过 1 年前
(2020)