Ask HN: Is it feasible to do high-frequency trading as an individual?

I'll share some experience a friend had with this stuff. I don't have the low-level details, but this should be enough to make an informed decision.<p>He was working on what he termed "high speed automated trading" -- basically all in this realm of picking pennies up in front of bulldozers.<p>He's a very smart and talented guy -- Harvard Physics/Comp. Sci double major, etc. So we're not talking about an amateur throwing around some code.<p>He spent about 6 months and over $50k setting up his system -- he had developed some algorithms to trade spreads between different securites (on the CBOE, the NYSE, and NASDAQ). He built a relationship with a clearing agent, a direct broker, etc. He got machines in colo facilities as close to the exchanges as possible (one data center in Chicago, one in Jersey City, etc.) SLAs on low-latency DS3 lines, etc. The whole nine yards.<p>I'd call it a "pro-sumer" level setup -- everything done by one guy, but done basically as well as a bigger firm would set it all up.<p>Here's what he discovered after about a week of trading: He didn't have even remotely a shot at competing. Like, not even close. If he had 10ms pings to the exchanges, someone else had 5ms. If he got down to 2ms, someone else who was physically at the exchange itself had 1ms. He got killed with commissions -- even if he did make a few $$ on some trades, it vanished with commissions.<p>Why? Because he was competing against guys who paid NO commissions. The broker-dealers and clearing companies themselves had internal automated trading setups. They had deeper relationships and deals that traded commissions for a cut of the profits, etc.<p>In other words, he figured out pretty quickly that this is not an algorithm game or speed game - it, like many things on Wall Street - is a 'who you know' game. No matter how smart, how fast, how sophisticated you are as an outsider, the likelihood is that someone on the inside has a similar trade idea and can do it faster and cheaper than you.<p>Now, the corollary to that: My friend is smart, but he also tends to give up too easily ;) Clearly if you've got an idea for a better/smarter/more innovative trade, then you'll make money on it. But that's a trade idea, not a speed advantage.<p>So just recognize that there's potential to make money, but "same idea, just faster" won't cut it.

Stay away from HFT, big players will eat your lunch. HFT isn't even a game of milliseconds anymore, it's a game of microseconds. You can still do well with algorithmic trading where speed of execution isn't an issue. Work on your strategy, test it on historical data, execute it manually, if it works then automate part or all of it.

I would ignore the "HFT" moniker and focus on the algorithmic part. HFT is a big-money game for companies who can afford to build a data center next to the exchange. But a longer-term strategy based on technical indicators can still do very well.<p>My main recommendation would be to take things slowly, browse through the COTS options in the field(which are plentiful), and make some of your own trades in order to learn and develop your strategy.

It's not a dumb idea, but you certainly should give up. The high frequency game is one you will lose, because many of the large investment banks have relationships with the stock exchanges allowing them to get information faster - and respond to that information faster - than other traders.<p>See this NYT article (including the graphic) for more: <a href="http://www.nytimes.com/2009/07/24/business/24trading.html" rel="nofollow">http://www.nytimes.com/2009/07/24/business/24trading.html</a>

Since everyone has said no so far, I'll go ahead and say it's possible, but very difficult.<p>HFT algo commission is much lower than $10/trade. IBKR offers 0.008/sh, and it can get much lower if you trade in volume.<p>Like others said, it is a game of milliseconds. So you need to own a server near the exchanges in new york (i.e. <a href="http://www.ubiquityservers.com/data-center/new-york.php" rel="nofollow">http://www.ubiquityservers.com/data-center/new-york.php</a>).<p>The best way to start learning is develop algos <i>not</i> for HFT. Instead of milliseconds think about 15 min - daily holds. If you're trading under 25k, you're only allowed to make 3 intraday trades a week.<p>So find a broker that has a FIX api, buy some data for backtesting, code some stuff out using R/python, and forward test your strat using papertrading for about a month-- then put real money to work.

Someone on reddit a few months back was succesful at this:<p>"I used to work as a software engineer and started developing and trading automated strategies in my spare time in 2006. I went full time in 2007 and have been profitable every quarter since. AMAA"<p><a href="http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automated_independent_retail_trader_i/" rel="nofollow">http://www.reddit.com/r/IAmA/comments/9s9d7/iama_100_automat...</a>

True high-frequency is very hard. Most of these guys got started in the 1999/2000 time frame and didn't have to deal with many of the start-up issues that new entrants face. For example, just consider the data you need to trade that quickly. Not only is the real-time feed expensive, but if you want to source the data from the execution venue then you have a lot of code to write. The amount of data is also quite large. For example, the US equities market executes approximately 45 million trades per day and there are about 700 million quotes per day, not including the non-top of book quote activity in the various ECNs. I collect between 2 and 4 GB of data per day, if you were to go direct to the execution venues you'd be looking at &#62;50 GB per day, probably.<p>So, anyway, the "true" high-frequency game is very tough today because we're already 10+ years into it. The markets have changed and the edge has gotten smaller, but there is still plently for a lowly individual automated trader to scratch away at.<p>Instead of looking for millisecond opportunities, look for second or minute opportunities. Go where the big guys can't because there isn't enough capacity. Can you find an edge that, on average, keeps you in a trade 30 seconds, for example? There's still plenty of alpha left, just don't step onto their playground and expect to get onto the swing set.<p>A few practical notes:<p>* You need to look into unbundled or cost-plus commission structures. These fee structures charge a per share commission and pass through all fees and rebates from the executing venue. This is required to do any sort of size with reasonable cost.<p>* Most "retail" brokers are not sufficient for any sort of high-volume algorithmic trading. Interactive Brokers is barely ok if you are in any way interested in limit order trading because they have fairly large cancel fees for direct routed orders. If you're model doesn't rely heavily on strictly offering liquidity or you're ok with letting IB route your order then IB is ok and offers an unbundled commission structure. Lightspeed Trading and Lime Brokerage are two that cater to active individual and institutions.<p>* Data storage is a big deal. Effective storage of regular (evenly spaced) and irregular time series will require you to engineer something. There are commercial solutions, but you can't afford them. When you're dealing with high-volume intraday trading this is one of the first issues you'll face. How do you store, query, and manipulate data that includes 45 million new rows per day? The relational DBs fall apart pretty quickly and even if they didn't they won't give you the time series operations you need/want.<p>* Data feeds are expensive, but required. Look at DTN NxCore. It is a full market feed that will give you the best you can get w/o going direct to the exchange. Some brokers will give you access to raw exchange feeds, but you'll need to engineer feed handlers and a ticker plant for them. This is a non-trivial task, but not impossible. Once you've done that, you'll need to figure out how to get all that lovely data off your co-located server and back to your home base for analysis. Network engineering will be required because your broker doesn't want you pushing 10-20GB per day through their network connection, so you'll need a circuit from an on-premise carrier.<p>* The banks are players in HFT, but not the original or best. Most of the guys that started it are still independent. Look at GETCO, RGM Advisors, etc.<p>* Flash orders, what most folks in this thread are refering to when they say the exchange gives the HFT firm a first look, are no more. That edge existed, and I'm sure HFT took advantage, but no HFT firm was built on flash orders. When they started flash orders didn't exist.<p>It is possible to be a successful, independent, automated trader. It is even possible to do it on a purely intraday, high-volume basis. Don't get caught up in the hype of needing to be high-frequency or not.<p>---<p>EDIT: The other thing I forgot to mention is that naked sponsored access is likely going away. This is a near-requirement for "true" HFT. Any future regulation won't affect any of the existing players because they're all grown up now and most have their own broker/dealers. Some form of sponsored access will likely still exist, but pre-trade risk checks will probably be required and will therefore still leave you're broker between you and the market.

Cheaper? How. 10/order makes no sense at all if you are ordering 10-20 shares at a time. It makes a lot of sense if your orders are 100,000-200,000 (or more) shares at a time. Individual traders (most often) only do the 10-20 shares or so... Check to see if your broker has limits on the number of shares/order.<p>High frequency algo trading, as presently committed to by the big banks, relies (as others have noted) on advanced notification (if only, at times, milliseconds in advance...) which, as an individual, you probably won't get. But that just means you can't make use of the same algorithms as the banks. You'll want to derive your own algorithms to exploit some specific pattern(s) that are different from those that rely on advanced word. It's possible, but unless you're already in possession of some wealth, unlikely that you'll be able to design, implement and tweak your algorithm to profitability in anything like a year or so. I certainly wouldn't try it as anything less than a full time occupation... certainly not something that can be done as an 'experiment'.

The only person likely to get rich off that is the broker...

No.<p>Your ping time will be &#62; 20ms. factor in a bit of latency in the data feed and call it 30ms. Nyquist says you'll get aliasing unless you sample 2xfreq. So The highest frequency you can possibly trade is 60ms. Not at all high frequency.<p>And I really doubt you'll get a 20ms ping time. I get 100ms to my broker. Can you do mid-frequency trading? Absolutely. But you don't have the money or access to resources required for hifi trading.<p>Also, $10/trade is VERY high. I'm using interactive brokers, which is $2/side+some costs which are small enough I don't bother accounting for them.

What about low frequency algorithmic trading? As in you run a program that is looking to make money holding a position for hours to days so the exact buy price isn't as big a deal. Or does it become to much of a game of chance then?

You should give up ... reason is that HFT costs for the big houses are just a fraction of what you are paying, like less than a penny per trade. You could not hope to gain back in margin, what you lose in transaction costs.

High Frequency trading, is really meant for people or companies that have their servers in the same network as the exchange they are HF'ing on. In fact, true HF'ers buy and sell in fractions of a second, some lasting seconds, or minutes.<p>I've researched this, and tried doing the same, but, in the end, trade fee's, and commissions kill the idea dead. In fact, HF Trading falls under Day Trading, which required that you must have the legal minimum $25,000 in equity on hand to day trade... that is the main thing that killed it for me... read for yourself: <a href="http://en.wikipedia.org/wiki/Day_trading" rel="nofollow">http://en.wikipedia.org/wiki/Day_trading</a><p>If you'd like to try it out before you trade live (here comes the shameless plug), you can use the trade simulation I made (buy/sell/sell short) and set your trade fee's and commissions totally free at <a href="https://algxchange.com" rel="nofollow">https://algxchange.com</a>

<a href="http://news.ycombinator.com/item?id=1431834" rel="nofollow">http://news.ycombinator.com/item?id=1431834</a>

"In other words, he figured out pretty quickly that this is not an algorithm game or speed game - it, like many things on Wall Street - is a 'who you know' game. No matter how smart, how fast, how sophisticated you are as an outsider, the likelihood is that someone on the inside has a similar trade idea and can do it faster and cheaper than you."<p>For every wall st question posted to HN this is the answer. It's a lot of money in somebody else's backyard. The game's played in that backyard are always rigged against the outsider.

Check out Interactive Brokers at <a href="http://www.interactivebrokers.com/" rel="nofollow">http://www.interactivebrokers.com/</a>. Their brokerage fees are substantially lower than that.

As others have said, I think focusing on HFT is a <i>bad</i> idea. You are going to be in competition with other players in the market. What are your weaknesses compared to them:<p>1) You are alone; you are competing with teams mixing 3-4 math/physics ph.d.'s and 3-4 programmers that might implement their algos on the latest GPUs (and this is <i>very</i> time consuming).<p>2) You have no experience. You are competing with <i>very</i> smart people that have been playing this game for years.<p>3) You (probably) have little money. So you want to minimize cost of historical data acquisition, data storage, data analysis, hardware, collocation …<p>IMH, these points pretty much rule out HF trading.<p>You also have some advantages compared to the competition. One of them is that you have little money, so you can invest in assets that are illiquid to someone that wants to move a lot of cash.<p>I think it is possible to trade as an individual; hard but possible. And if you are successful at some non-crazy frequency -- a few days, a few hours, a few minutes -- then, maybe, maybe have a look at HF.<p>And another thing, if you can, go shopping for a broker with lower fees. $10/trade? It looks as if you did not do basic homework -- but mabe you live in a country where that's the cheapest you can get.

As already said both here and in previous threads: HFT is for <i>big</i> players; requires:<p>- hosting on exchanges premises to cut roundtrip delay (µS speaking). Also, very often, you'll be throttled depending on how much you pay. For entry fees, you'll be limited to ridiculous rate such as 20 msgs/sec.<p>- man years of development. Basically, when starting from scratch, we develop Direct Market Access gateways (i.e, custom/proprietary access to exchanges) in 3 to 5 man-months (using an in house framework). And they sit on top of other products that have dedicated teams working fulltime for years.<p>- upkeep: exchanges update their systems once/twice a year. Migrations take somewhere from a few hours to weeks, or a from scratch when they rewrite their complete API. If you can't keep up with the updates, you won't be able to trade.<p>And here, we're only speaking about trading. As highlighted, you also need market data, both retrieval and processing. Then and only then you'll be able to seriously start trading. (and then you'll want to do some back office stuff)<p>&#62; The trading fee is close to $10/order<p>Deal breaker, especially for HFT.

HFT is a game of speed and low fees. Not a single individuals broker can offer substantial speed (microseconds count) and fees (cents count). It is a game for prime brokers and big institutionals. Moreover it is true that some exchanges (like NYSE) offer lower latencies for a serious monthly fee (not the one an individual can afford). Thus I don't think one person can solve all these complexities, however this topic is a good startup field with lots of ineffeciencies and unsolved problems.<p>Algorithmic trading is an option, though needs careful research on topics of broker setup (InteractiveBrokers should be a good starting point, ThinkOrSwim is another good one, both has rather tolerant programming APIs) and software for time series analysis and trading (among the most popular are Marketcetera and WealthLab + lots of libraries like Incanter and Weka). On some of these topics <a href="http://elitetrader.com" rel="nofollow">http://elitetrader.com</a> forum should be helpful.

Successful HFT desks are either large broker-dealers, or sophisticated hedge funds, both of which are well-capitalized, have expensive infrastructure, great location (close to exchanges to minimize lag), ultra-low trading commissions etc. Furthermore, only the hedge funds are pure prop groups. Most the broker-dealer desks act as market makers and aggregators for internal flows (almost free money).<p>So it's going to be very hard for you to compete with them, even if you come up with better algorithms.<p>I recommend looking at mid-frequency or low-frequency algorithms, which will offset the impact of your higher trading commissions and don't get affected much by infrastructure and location issues.<p>And yes, $10 per order is too much unless you talking really big money. Check out IB, FXCM etc.

As an individual it is unlikely. Your best bet would be to develop an algorithm with massive amounts of recent historical data that shows you could hypothetically be profitable. Shop that to trading firms or angel investors and use their capital to get a direct feed. You probably won't be able to do this to a major exchange, but there are lots of other options. If you find a smaller market you can also potentially get a market maker agreement. This will mean that you have to have certain positions open at all times, but you can place any orders you want in addition to those, and all of them will have $0 fees.<p>All that said, I would suggest you stick with vanilla algo trading to start, and leave HFT to the big boys (until you are one).

your internet-based API is nowhere near fast enough to compete with legit HFT firms. low to medium frequency with longer holding periods, sure, but real HFT via your broker's API? not gonna happen.

If you're not connecting directly to the exchanges, no.

I like lrm242's comments. I started my own high freq algo firm back in 2007 and crushed it for awhile. In the end though, the governing dynamics of the marketplace change and can change very rapidly, so any machine learning or neural networking you do perform can become obsolete fairly quickly. Unless you've developed HAL you're probably going to have a rough go of it alone...hope this helps!

High frequency low latency trading is an fascinating area. If you do decide to make a go of it and have the algorithm, the collocation site, the relationships, and the money, you might want to work with Maxeler Technologies, <a href="http://www.maxeler.com" rel="nofollow">http://www.maxeler.com</a>, which has the acceleration technology that is need to make it happen.

There is a talk being given by Andrew Sheppard on O'Reilly right now talking about using GPU programming to speed up number crunching, specifically in the financial arena.<p><a href="https://oreilly.connectsolutions.com/gpufinance/event/" rel="nofollow">https://oreilly.connectsolutions.com/gpufinance/event/</a><p>I believe the talk will be recorded and made available for later viewing.

A little OT: People here are always talking about trading stocks, anyone dabble in forex?

I can't even get real time (meaning not delayed by 15 minutes) stock quotes for my startup here in Brazil.

etrade is about to come out with an API and programmers could use it to do some really cool HF trading.