HFT Quote Stuffing is a software bug

62 点作者 jpdus大约 11 年前

10 条评论

kasey_junk大约 11 年前

So again with all HFT conversations we have to be clear on what we mean with the language we are using.I've seen quote stuffing used to mean what Chris mentions here. The implication being that HFTs were using large volumes of low quantity orders for nefarious purposes such as DoS the exchange & SIP or confusing your competitors by making the order flow to heavy to process. These kinds of games did occur, but it was very briefly a long time ago because the exchanges themselves had a vested interest in stopping it and an easy mechanism which to do so (fill ratios). I prefer the term quote spamming for this, and I agree with Chris that most of the time when this happens it's just a bad order management bug.Quote stuffing can also refer to the practice of putting a few large orders into the order book with no intention of trading them in order to make the volume on the book look larger than it is, and then cancelling them quickly to take advantage of the "real" volume. This is a predatory activity designed explicitly to take advantage of market making strategies and is already illegal. Your mileage may vary when it comes to enforcement...

评论 #7628863 未加载

rsync大约 11 年前

I always assumed that step 2 was "DoS your competitors".This is crudely done with volume of traffic, but more elegantly done with frequency and amplitude (and so on) given some set of (possibly wrong) assumptions about how competing algorithms will respond to this rapid fire order placement.I know almost nothing (beyond ZH and chapters 1-5, so far, of _Dark Pools_) about HFT, but I can trivially think of ways you could be evil with quote stuffing as it is described in the OP.

评论 #7628725 未加载

评论 #7628584 未加载

评论 #7628601 未加载

hft_throwaway大约 11 年前

<a href="http://zacharydavid.com/2014/04/on-hft-part-ii-bugs-features-and-aggressive-incompetence/#A-Bug" rel="nofollow">http://zacharydavid.com/2014/04/on-hft-part-ii-bugs-features...</a>This blog post details similar issues that can lead to rapidly canceled orders. Generally, firms that are smart avoid moving their prices around "too much" since:a.) You lose your spot in the queue. If I am near the front of the price/time queue buying at 100.01 and move to 100.00 and back up again, now I'm behind everyone else and will either lose opportunity or face more negative selection (I'll only trade when someone wants to sell all the contracts bid at 100.01, whereas before I would get some trades where someone only wants to sell a few).b.) Sending orders is more computationally expensive than doing nothing. If you send enough orders you will slow down your own system.c.) Depending on market rules and behavior the messages you send may cause your gateway to queue up or block you from sending more messages which hurts you.I don't think quote flicker can provide any "evil" benefit to the person doing it. It doesn't make sense to do it to slow down competitors, since most feeds are multicasted to everyone and you still have to process all the orders you send, both from the gateway and over the data feed. If you are faster at handling this situation than your competition then you are going to beat them anyway so what's the benefit in slowing them down?I do think it can be disruptive in the sense that it places extra load on the exchange and can annoy participants who have trouble taking prices on their screen. Some markets have quote/trade rules to encourage more efficient messaging which is their right and not a bad idea if the limits are set reasonably. Rules like this make more sense than a minimum resting time, which just penalizes makers at the expense of fast takers. Imagine I am putting quotes out in SPY, an S&P 500 ETF, but they need to rest for half a second. Someone sees the S&P 500 futures or the individual stocks move and then my quote becomes a free option to trade at an unfair price. In a market like this, market makers would be forced to put up wider prices to compensate for these losses.A high quote/trade ratio in itself is not disruptive. A lot of the Nanex charts use thinly traded derivative products like ETFs as their whipping boy. The underlying value of these products will change whenever another stock or commodity changes, so market makers need to update their quotes quite rapidly to avoid trading at unfair prices, even if no trades have occurred in the product itself.

评论 #7628328 未加载

jpdus大约 11 年前

Nanex linked an interesting CS research piece[1] which also discusses other possible reasons for (malicious) quote stuffing including:- walking others into the book,- creating false midpoints and- trying to cause stale pricing and slowing others down.[1]<a href="http://de.scribd.com/doc/116761218/CS-HFT-DETECTION" rel="nofollow">http://de.scribd.com/doc/116761218/CS-HFT-DETECTION</a>

评论 #7628868 未加载

mikeash大约 11 年前

That sure is a lot of hand-wringing for what is a fairly simple and obvious problem. You're reacting to the extremely short-term derivative of a noisy signal. Of course things are going to go crazy from time to time.It's odd that he touches on the obvious solution of rate limiting, but doesn't actually discuss it as a way to stop this problem. If quote stuffing is a bug that harms the people carrying it out, then why don't HFT programs have a rate limiter on them that make them stop? He even goes so far as to say that you can get shut down for doing this, so why isn't there a tripwire to make sure you can never, ever do it?

评论 #7627978 未加载

评论 #7627964 未加载

anigbrowl大约 11 年前

Except, once in a blue moon this happens when the mean signal approaches the threshold and stays there for a while:I'm having some trouble with the contention that this is just such a rare thing. I do a lot of audio signal processing so I'm used to dealing with noisy signals at Khz rates. Threshold fluctiations are a fact of life if you're trying to do things like pitch detection, envelope following, or sidechain compression (all of which have obvious analogs in the trading context).Obviously you don't want to impose too brutal of a low-pass filter on your input because you would miss important signals, but what prevents you from running the input signal to an LPF in parallel and keying a noise gate when the amplitude of the low-frequency (trend) signal drops? This is such a common problem in dealing with audio that we use specific devices called de-essers for recording vocals, because sibilant sounds otherwise tend to throw off other components of the signal chain: <a href="http://en.wikipedia.org/wiki/De-esser" rel="nofollow">http://en.wikipedia.org/wiki/De-esser</a>Ironically, you mention using XKCD-type graphs because 'nobody reads' the explanations and caveats, but just looks at pictures. I wonder if you aren't missing a trick here by over-engineering your HFT systems for speedy response at the inevitable price of positional confusion. Is it possible that you're using Infinite Impulse Response filtering and running into denormalization problems as a result? If so, can you not rectify the problem by adding a small amount of noise?I get your point about the negative aspects (for the HFT trader) of quote stuffing, how it can easily lead to being on the wrong side of a trade or punishment by the broker for flooding the output channel with unintended orders. But, you want to make money from automated signal analysis, that's your problem in much the same way that a sound engineer is responsible for preventing transient pops and feedback howls in an audio network. Sure, these problematic signal conditions are statistically rare, but when you're dealing with a high volume of samples you'r going to run into statistically rare conditions on a regular basis - and indeed you mention that this is the sort thing you might expect to happen once a day.

评论 #7629343 未加载

lmg643大约 11 年前

this is an interesting way to look at the problem but in my reasonably informed opinion, flawed.it seems like this is working backwards from the assumption that quote stuffing must be benign activity, and then trying to rationalize it with signal/noise theories about finding the fair price level.it is totally inaccurate to say that there is more load on the sender than the general public, and also misses the mark on what the DOS theory alleges.the exchange bears considerably more load than the sender of orders. the exchange must (1) ack the order, or reject if any flags are populated out-of-spec, (2) add the order to the order book, (3) check for potential matches based on price of order, if so, cross and update feeds, (4) if non-marketable, publish out to (a) depth-of-book feed, (b) SIP feed, etc.the DOS theory is that excessive activity in a single partition can slow down the publishing of data in that symbol range, which would mess up the quoted price for the exchange in a symbol, which would affect orders priced on the exchange, or anyone using the price in a derived manner (at a dark pool etc).

评论 #7628398 未加载

评论 #7628414 未加载

评论 #7632928 未加载

austengary大约 11 年前

This is fun<a href="https://twitter.com/nanexllc/status/458663948800520192" rel="nofollow">https://twitter.com/nanexllc/status/458663948800520192</a>

gd1大约 11 年前

Obviously, well written HFTs avoid this by introducing some hysteresis to the signal. i.e. Insert an order when the signal crosses 0.5, but then only delete it if the signal falls below 0.4.Chris: You've also missed the most common cause of these issues in my experience - feedback from identifying your own orders in the public stream. HFTs need to filter their own orders out of the orderbook they are trading on, or a feedback loop can begin. This can be difficult, since some exchanges don't provide a clear way to identify your own order, or it may be the case that you receive the public UDP packet before the private TCP packet that contains the ID required to match the two.For example (a basic one), say I have an algo that calculates the midpoint based on the best bid and ask volumes. And can only insert within x distance of the midpoint. I add a bid to the top of the book for a 1 lot, get the UDP public depth update, don't know whether this is my order or someone elses. So I re-calculate the midpoint, which pushes the midpoint upwards (greater bid volume), which means I then pull my bid, which means I recalculate the mid again, then re-insert, etc. etc....

评论 #7628131 未加载

评论 #7628326 未加载

personZ大约 11 年前

The 2. in the series, if I understand it right, is that they overwhelming the derelict SIP feed between market locations, increasing the benefits that their own high speed connections brings. If true, that does make money because they can act "in advance" at other markets before the artificially slowed SIP data gets there.