Show HN: How I built a trading signal by scraping Nasdaq for short interest

My comment in the last thread opened with a post from this source:<p><i>Past performance does not guarantee future results" is still the operative principle here. Data-mining discovers patterns, but it doesn't lead to deep insight into causes, and markets are perturbed by many events that you don't put into your training algorithm. "The market can remain irrational longer than you can remain solvent" is still important investment advice.</i><p>You can never build a trading signal just by scraping historical data, unless you like losing your shirt.<p>Can you tell I'm reading <i>Antifragile: Things That Gain from Disorder</i> just now? I'm very sensitive to errors in statistical thinking today.

Back testing is a real bitch. I've been building my own app for back testing recently, my specific interest being how published insider buys (SEC Form 4 transactions) affect the prices of stocks in the short near and long term. You can get dividend data and stock splits easily enough from some public feeds. But where do you get a database of ticker changes, bankruptcy events, and spin-offs, especially on the OTC markets? You can't unless you're willing to shell out a lot of money. Back testing properly is probably out of the cost range of the individual investor.<p>Some examples:<p>* Lehhman's ticker changes on the way down<p>* GM going bankrupt and then coming back from the dead!<p>* Skye International used to trade under SKYY (at 0.35c/share), but now SKYY tracks a cloud SaaS ETF 20.60/share). Think you got a big win using that strategy that including buying SKYY? Think again!

I have for two years now been playing around with Algorithmic trading as a hobby and I am amazed by people who think wave riders or simple mathmatical transforms will get them profits in the market. I have found that the best method is still a good mix of modeling and trader input. I don't think a model exists that you can just turn on and have it print you money. So attempts like this to make one of those really are a waste of time. Your systems should be tuned to listen to you and then take what input you have and do what you cannot ( make decision in sub-second windows )

Can somebody explain to me why, if this really works, you would publish it in a blogpost? Shouldn't you be hunting down investments of $X to turn $1.093X?

When the broad market is rising by over 10% annually, it is very difficult to come up with a trading strategy that looses money.<p>For example, buying SPY and holding it for the same period would have outperformed your algorithm.

Serious question: does this meet "Show HN" criteria? I mean I value sharing the algorithm, but I thought that Show HN is reserved for entire projects (ie. sites, saas platforms, etc.), not using ones platform to put up a description of algorithm and some numeric data. I'm not trying to troll, just wanted to know how the community understands "Show HNs"? In this case it can be seen as more of a Quantopian show off (which is interesting service, but had already been showcased) than the algorithm or project itself?

While I'd love to take all the credit (blame?), the reformed academic in my feels compelled to admit that the idea to look for predictive value in stock loan data is not original to me. The finance literature has some fascinating articles on this dating back as far as the late 80s (look for Desai 2002, J of Finance, Asquith 2005 J Fin Econ, or most recently <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1570451" rel="nofollow">http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1570451</a>).<p>The intuition behind this signal as a market inefficiency, or 'anomaly' is that the market sees short sellers as informed investors, the so called 'smart money', and there is a herding effect to follow their trades which generates abnormal returns. The same logic can be applied to disclosed insider trades or institutional holdings filings made public via the SEC's EDGAR database.<p>Fawce's slick implementation of a 'Days to Cover' signal is a great way to highlight the power of aiming new tools like Quantopian at freely available public data stores (which exist expressly to increase market transparency). And sure, it doesn't go the whole way for you on execution details like borrow costs, liquidity etc. but those aspects tend to be unique to each trader.

You should put in some kind of protection for a max drawdown loss, like if you lose x%, you exit. Sometimes your algorithm messes up, or market conditions are bad. <a href="http://www.businessinsider.com/hedge-funds-smashed-worst-quarter-since-2008-collapse-2011-10" rel="nofollow">http://www.businessinsider.com/hedge-funds-smashed-worst-qua...</a> Long short equity funds did poorly in 2008 financial crisis, and also in 2011, when there was high volatility.

Very interesting stuff. "The Benchmark" is the SP500 I'm guessing? I couldn't find the answer after clicking around for a bit, sorry if I'm dumb. You might list the reference security in the chart, or do something like "SPY (benchmark)" in the key.

Did anyone actually try this with real money? Does the model include transaction costs and market impact effect?