The anatomy of an ML-powered stock picking engine

This was a very enjoyable read. I built a nearly (architecturally) identical system a few years back that also had to be scrapped for different reasons. This brought back a lot of memories. The sanity checks, the index reconstitution issues, dealing with the insanity of security identification and tracking through time.<p>The fun cases are the ones where it&#x27;s not even clear what the right answer truly is, e.g. company A spins out company B, and then 5 years later they re-merge. Who&#x27;s time series and associated data is &quot;the&quot; canonical one? The data vendors often try to give <i>their</i> answers to this question, but maybe their answers don&#x27;t make sense for your analysis.<p>Then there&#x27;s the fact that a lot of vendors don&#x27;t really do point in time correctly. They like to go back and helpfully revise data points for you that they or the company initially misreported. This is all well and good except that if you were trading for real, you wouldn&#x27;t have known the correct information at the time, and so any backtest based on the updated information will be invalid. Vendors are a bit better now about providing true point in time data sets, or at the very least accurately describing when they are&#x2F;aren&#x27;t doing this. But we had a few cases where they said they were, but they definitely weren&#x27;t.

Someone asked about how difficult it is to get outside investment....<p>It&#x27;s usually very difficult and it takes a lot of money to run a proper fund.<p>Let&#x27;s say you raise $50M. You can maybe charge 1 and 20,meaning you get 1% of assets each year for running the fund and 20% of profits.<p>1% of $50M( and keep in mind this is a large raise for someone without a track record on the sell side or inside another fund) give you $500,000 a year to pay:<p>- salaries( lets say you pay yourself $100,000 all in plus the same for a single analyst<p>- a Bloomberg terminal $30,000 including data feeds<p>- market data feeds you need $25,000&#x2F;year for basic market data and fundamental data that you are allowed to warehouse(you can&#x27;t store data you get from the Bloomberg terminal).<p>- rent $50,000&#x2F;year for office space<p>- outside lawyer fees and outside accounting fees $100,000&#x2F;year<p>- similar fees for someone to run your back office, roughly $100,000&#x2F;year.<p>And on the other side of expenses you have the money making side of things. Which as the OP pointed isn&#x27;t great. If you return 10% on the 50M you get to keep 20% of that so a 10% return gives $5M in profits and you keep $1M.<p>That allows you to bonus out yourself and analysts on good years. If you lose money one year then you get no bonus and have to bonus out the employees out of the retained earnings you kept from previous bonuses.<p>it usually gets worse as most funds have what&#x27;s called a high water mark. This means you don&#x27;t collect the performance fee until your fund gets back to the high water mark. So if you are down 10% one year you need to make that back before you start to make any performance fee, which is why most funds shut down if they go down more than 20%.<p>As to raising money.....Anyone can show a model that makes money. that doesn&#x27;t mean its easy to create a model, its just that there are alot of people capable of building such a model.<p>Its the risk management that people with money are really looking for and sadly that&#x27;s just really hard to show out of a model as part of the risk management is things like positions sizing and showing your model doesn&#x27;t pile into one asset class or trade correlated products.<p>it bodes well for the OP that they talk about market regimes as, IMHO, this is one of the biggest risk management tools that aspiring traders ignore.<p>And this risk management is why people ask for a track record of more than a year.

What&#x27;s the market beta? What&#x27;s the average turnover&#x2F;holding period? How are transaction costs modelled? What features explain most of the variance? How are they related to known factors? What&#x27;s the beta hedged performance?<p>These are all things I&#x27;d want to know before deploying something like this. (Perhaps some mentioned in the post, might have missed them.)<p>To first order, I&#x27;d forget about fat tails and similar popular concerns. They matter, but not as much as structurally understanding what this model is up to. Perhaps one feature is explicitly selling tails? That might answer it already.

I know a bit about this industry and I have worked on some profitable systems. Honestly not a bad effort for someone working on their own with low-cost data. Don’t let the haters get you down. I would recommend you to pick up a more recent textbook on portfolio construction like Isichenko’s recent book.

My heart goes out to this author, but you can tell even by his first table that he doesn&#x27;t quite understand the mathematics of financial markets, the purpose of a hedge fund, how they grow etc.<p>1) It&#x27;s plain by quickly looking at the allocation of capital in investment firms, that AUM is not made by performance; it&#x27;s marketing. At best people invest when they believe a person is connected to inside information. Saying you have an ML advisor is really just a pre-req to these people.<p>2) Is that allocation stupid? No, it&#x27;s not, because actually the powers of mathematics and by extension ML are intrinsically limited for investment returns because they are fat-tailed &lt;&#x2F;Taleb&gt;. For example this author quotes a realistic sharpe (0.8), but didn&#x27;t calculate the standard deviation in his sharpe, which I would bet a large sum was _at least_ 0.8. Ie: he doesn&#x27;t really know what his sharpe is. This is because equity assets behave like a student-t distributions with a degree-of-freedom parameter ~2 or less &lt;&#x2F;Mandlebrot, &#x2F;Bergomi, &#x2F;Gatheral etc.&gt;. Ie: higher moments such as uncertainty in sharpe, literally do not exist or converge and are unknowable. The only exception is if your strategy explicitly cuts off tails.<p>Once you understand 2) you begin to understand that there&#x27;s no such thing as a real quant fund (ie a fund which truly makes money predictably using models) which doesn&#x27;t trade a liquidity limited book that has quite advanced hedging. Wealthy people are aware of this, which is why the author can&#x27;t market this product.<p>If you&#x27;re doing something silly like holding equities without tail risk control, you literally cannot be quantitatively investing. You are just slowly rediscovering what Kelly, Bergomi, Mandlebrot, Bernay&#x27;s etc. realized with a little deep thought over pen and paper (while clumsily writing boilerplate software.) That markets are entropy machines rougher than a normal distribution, and any gains come directly from information. (see: Kelly: &quot;a novel interpretation of the information rate&quot;.)<p>For a high latency (ms) market data feed, the returns on information are very very small. Markets are efficient.

Nice writeup, thank you for sharing so openly!<p>The three things I always want to know from stock picking ML people:<p>1. Did you put your own money in it ?<p>2. How&#x27;d it go?<p>3. How well does your engine do vs a fixed stock allocation based on trend-statistics computed on the whole time window (i.e., compared to a fixed optimal portfolio computed with mean&#x2F;std values you don&#x27;t have access to, but which isn&#x27;t allowed to change its choice. what&#x27;s the regret if you are familiar with online learning)

Every gambler thinks they have a system, but often fails to recognize a game is unfair long before they arrived. lol =)

If you have a tool that can generate great returns, then why fall back to a newsletter?

<i>&gt; I have always kept in mind is that feature engineering is almost always the key difference between success and failure</i><p>I also developed an ML-powered service heavily relying on feature engineering<p><a href="https:&#x2F;&#x2F;github.com&#x2F;asavinov&#x2F;intelligent-trading-bot" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;asavinov&#x2F;intelligent-trading-bot</a> Intelligent Trading Bot<p>Its difference from Didact is that this intelligent trading bot is focused on trade signal generation with higher frequency of evaluation. It is more suitable for cryptocurrencies but also works for traditional stocks with daily frequencies so it could be adapted for stock picking. What I find interesting in your work is the general design of such kind of ML systems relying on feature engineering.

Hi, fellow HN&#x27;ers! Author here, please let me know if you have any questions or thoughts!

This was not only a very informative read but felt like an amazing achievement if everything described here was developed by one person (the author - @muggermuch).<p>The breadth of knowledge demonstrated by author from technology (bringing performance down to 14 minutes) to ML to deep understanding of financial markets is super-impressive.<p>Granted the author has an educational background in computer science and has been a trader which probably explains many of his abilities but to my small brain it feels next-level achievement.<p>Maybe I live in average circle of finance but I have never met nor heard of a person who could single-handedly conjuncture and implement such a system. To my knowledge, a typical hedge fund has several highly-paid people in different teams to build and maintain such a system.<p>I never thought one-person could do it. I genuinely wonder how he managed to wrap his head across this much knowledge. He seem to fall in 10x category. Kudos!

Nice report. How did you did risk management? Have you been leveraged? Have you paid for data? Kudos for a view from the trenches.

If you have a guaranteed compounding money machine that out perform the market by 20% just let it run, sooner or later you will be able to buy out those who did not invest in you. If its just a useful recommendation engine than there are indeed a lot of questions relating to personal finance or investment strategies that have nothing to do with machine learning that needs to be addressed for PMF. You don&#x27;t need better models you need to understand the needs of your customers.

Have you considered submitting your predictions to the Numerai Signals? It&#x27;s market neutral so as long as your models can generate some alpha you can still get good returns.

I&#x27;m curious what happens if you look at your returns and other metrics at different time scales, i.e. monthly and weekly, in addition to yearly. You can&#x27;t make any argument based on a sample size of 2.<p>As someone who used to work in the industry, I am 99.99% confident that you cannot have any alpha with a system like this, you are basically flipping coins, as some other commenters have pointed out.

If your predictions are good, I&#x27;d be happy to get you $100 million in assets to manage. It&#x27;s very unlikely that your predictions are good...

&quot;steadily beating the S&amp;P 500 for over a year on a weekly basis&quot;<p>Can be achieved by chance alone.<p>If not chance, I can give you a strategy that would be highly likely to achieve such a result: it would take a lot of risk though!<p>I love it when people post this stuff to HN. Naive people try it, loose a bundle to market makers, then go back to their day job.

Thanks for the article! What tool do you use to create the figures? I like the sketch style.<p>For anyone else, what tools do you recommend for generating pretty for system architectures, workflows, etc.?

OP, what are you using to draw the diagrams? They look nice and are very readable.

&quot;Predicting&quot; markets isn&#x27;t the challenge. Implementing real world strategies with associated frictions is. Show me a cash p&amp;l or it&#x27;s just a student project.

Do these frequent trading strategies ever account for taxation? Where I live, the returns compared to a buy and hold passive strategy would be cut by 50%.

Brilliantly written. As someone considering a move into the Quant field it is very informative.

this is very cool! where did you get your data from and how&#x27;s the transition to airflow?

Great post! Very informative! Thoroughly enjoyed it.

Lmao this engine is down 6.9% for the year, when literally it&#x27;s as simple as just buying some puts.

Excellent article