High Frequency Trading Tech For Non-Financial Problems.

The area is also called stream computing. I work on a project in this area, which made HN a bit ago: <a href="http://news.ycombinator.com/item?id=2442977" rel="nofollow">http://news.ycombinator.com/item?id=2442977</a><p>Note that the above article erroneously ties the Streams project with the Watson project - aside from coming from the same research lab, we have nothing to do with each other. My comment on Streams from that thread:<p>Why I find working on Streams exciting: we're designing a language and runtime system for a new programming model. Not just a new language (which we do have), but a new programming model. The way you write programs changes when you have essentially infinite streams of data coming through your system - you have to think in terms of operators that transform or filter individual pieces, and you have to keep in mind its inherent distributed nature. That is, you may write a chain of operators to process your data, but each operator can run in parallel with the others, even though you will probably design your application thinking of one particular piece of data marching through the system. The runtime system itself is, of course, distributed and fast. To get an overview of this programming model, take a look at this paper: "SPADE: The System S Declarative Stream Processing Engine": <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.160.." rel="nofollow">http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.160...</a>. Just keep in mind two things: the newest iteration of the language is called Streams Processing Language (SPL), and that it is a complete rewrite of Spade.<p>If you're interested in getting a feel for the programming model, here is a public getting started guide: <a href="http://publib.boulder.ibm.com/infocenter/streams/v2r0/index.jsp?nav=/3_1_3" rel="nofollow">http://publib.boulder.ibm.com/infocenter/streams/v2r0/index....</a><p>And the language spec: <a href="http://publib.boulder.ibm.com/infocenter/streams/v2r0/index.jsp?nav=/3_1_5" rel="nofollow">http://publib.boulder.ibm.com/infocenter/streams/v2r0/index....</a><p>I suppose I should put in the disclaimer that these are my views, and I do not represent IBM.

I'm all for burning less fuel and making shipping more efficient, but I think it is a bit of a stretch to say this particular use case is not related to finance. Ports becoming more efficient ultimately saves oil, natural gas, and other commodities companies money. Commodities traders are very tuned into port congestion and global movement of ships and make lots of money trading upon the flow of commodities (mostly oil/gas).<p>If this particular case is not related to finance, then how come the Bloomberg terminal tells me there are exactly 401 ships in Rotterdam port as of this morning, the 5 day moving average is creeping up, and 8 of the top 10 ships by tonnage are crude oil tankers?

Idly I've wondered if there was a way to turn the efficient markets model on it's head. Rather than gathering a ton of predictive information to try and get another 2% efficiency, instead use the already efficient market activity to predict other things. For instance if oil prices drop GM should start ordering more V8 engine parts instead of I4. I'm sure this already happens at some level.

Nice to see all the resources invested by the market makers in savage zero-sum competition having some actual use.