it seems people call these techniques machine learning/AI simply because it sounds cooler: a lot of the techniques have a 30+ year history in statistics. this is the <i>same</i> story as neural networks: neural networks are just nested logistic regression, a technique with much history in statistics.<p>a lot of high frequency trading is simply linear/logistic regression (on the right features of course). anything more complicated is too slow.
Market prediction is one of the hardest areas for machine learning/statistics/AI to prove their value. What the system <i>can</i> do is monitor more inputs than a single trader, so the screening process can be scaled up--provided you have relevant inputs. With more signals becoming available electronically it is possible to automatically incorporate non-market prices into trading decisions. All this will require lots of non-ML engineering to supply the data (just like Google search). When monthly-issued government economic statistic stop moving the market you'll know we are there.
Oh, great. AI will be over-hyped again, just like in the 80's, and we'll have to call it something else for a decade when our families ask us what we're doing lately.
Hypothetical: What if an AI driven investment fund significantly out performed other funds?<p>The focus on number crunching seems contrary to advice given by Warren Buffet and other successful investors about investing in what you know.<p>i.e. "I think the market is under-estimating the impact of Product X, I'll invest in that company" versus "Analyze market cap, yearly revenue...."
I have my hand at this now with <a href="http://www.algxchange.com" rel="nofollow">http://www.algxchange.com</a> - its really cool, but training is a nightmare. I have 4 situations that required training, and an actual decision model that needed training on what situation model to use.<p>So many factors, Bear market, Bull market, etc...
Thinking about it, if an investment fund has a strategy, maybe it should really be possible to put it into code after all. If not, then what is the basis of the investment fund - the gut feeling of it's managers?<p>You could also still add the gut feeling of the managers as one input variable for the system. Probably even then there would be lots of things left to automate. Just because it is AI doesn't mean it has to do everything by itself.
Isn't the underlying assumption behind all these machine learning models that an incomprehensibly complex system can be modeled with a far simpler approximation of that system? I only have limited experience with ML techniques but my impression was that they're all essentially statistical and the essence of a statistical approach is that you don't really model the underlying mechanics.<p>I'm not saying that this approach can't work, but just that an insanely complex macro-system like the market is never <i>really</i> going to be reducible to a tractable statistical model.
Having tried to do this sort of thing in the past, I'm fairly skeptical. Maybe if you're trading in large volumes over short spaces of time you can make money with AI, but at least in my simulated runs with small scale investment the results didn't look encouraging.<p>Also it should be borne in mind that this kind of activity isn't really making any constructive or lasting contribution to society. It's really just moving money around slightly more efficiently than previously. If you really want to change the world, do something else.
If human beings can’t understand the rationale that an AI-driven hedge fund is using to make its trades, then how can humans tell the difference between an AI that has actually figured out a strategy to beat the market and an AI that is just having a run of good luck?