I have always been fascinated with how news and sentiment could drive markets. This has lead me to build an engine that attempts at predicting US stock market sell offs. So far it has been really successfully at doing so.<p>In our most recent example, on 16th July 2021 we predicted that there was going to be a major sell off on. The sell off actually happened on 19th July 2021.<p>In a more extreme example our engine successfully identified as early as 9th Jan 2020 that increased mentions of "coronavirus" in the world news was going to cause the sell off that occurred in 2020.
https://truesight.me/overview/2020-01-09/14<p>Unfortunately, that correlation panned really nicely for the next 365 days.
https://truesight.me/compare/SPY,452970-coronavirus/2021-06-25/720<p>Here is how we built this engine.
We pull in news headlines from around the world on an hourly basis.
Then we apply NLTK to extract the nouns that are associated with each news headline.<p>Next we pull in the stock prices of all the publicly listed companies on the US stock market to calculate the actual sell off levels for each day.<p>The final step is to apply Pearson's Coefficient at scale to identify nouns that have the highest correlation with sell off levels within defined periods we are examining.<p>These final set of nouns are then shortlisted as trends will be driving US market sell offs.<p>Here is the full story of how I went around the world curating that list of data sources to train my model.<p>https://www.ricemedia.co/features-the-singaporean-arrested-in-xinjiang/