High velocity trading rendered visceral

The latest installment of Tim Harford's BBC/Open University podcast (RSS) More of Less has a fantastic and chilling look at the world of high-frequency automated stock trading, where warring algorithms execute millions of trades in an eyeblink. The story's jumping-off point is Knight Capital, whose faulty algorithm hemorrhaged $10,000,000 per minute, ultimately costing the company nearly half a billion dollars. But from there, Harford and co do a great series of examples trying to convey the sheer velocity of these markets. I've been following this stuff reasonably closely and had an abstract sense of it all, but this brought it home for me so firmly that it raised goosebumps.

Last week Knight Capital lost a lot of money very quickly. It was the latest chapter in the story of something called ‘high frequency trading’. Investors have always valued being the first with the news. But high frequency trading is different: algorithms execute automatic trades, conducted by computers, at astonishing speeds. We ask: is the rapid growth of high frequency trading progress, or – as some think – a threat to the stability of the entire financial system?

BBC - Podcasts - More or Less: Behind the Stats

MP3 link

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: Business • finance • ZOMGWEREALLGONNADIERUNHIDE

More at Boing Boing

Ants and Stars: Bruce Sterling and Jasmina Tesanovic visit the Sardinia Radio Telescope in Italy

The Snowden Principle

http://twitter.com/kpkpkp Kevin Pierce

And here I was worrying about costing the company $44 (the cost of a consumable) every time I tested a program I’d recently written.
http://storyspieler.net Roy Trumbull

A transaction tax might slow this down. A description I picked up in the book “When Genius Fails” which described the trillion dollar bet made by Long Term Capital Management before it was dismantled, it’s like picking up dimes in front of an advancing bulldozer.
- Yacko
  
  I agree, but I will go a step further. How about a transaction tax on everything? Both parties. Payment for services, buy at a big box, sell a car, receive salary, buy bonds, deposit money, stock transactions, sell property or a house – everything. Again both parties to the transaction. Get rid of every other tax. What would the rate be? Could it be as low as under 1%? Can anybody model this?
  - blurgh
    
    A flat transaction tax for everything is not very attractive in terms of encouraging the kinds of things economists like to encourage.
    
    A company making something by buying in parts and gluing them together would pay a lot more tax than someone manufacturing them themselves internally, and thus avoiding extra transactions. It ends up discouraging specialisation, competition, economies of scale etc.
    
    A reason value-added taxes are relatively popular is because they get around this, by taking the same amount of tax independent of how many people there are in the supply chain.
sean davidson

Zero hedge is the place to read about these algos, and actually see millisecond breakdowns of their crazy actions http://www.zerohedge.com/news/knights-berserk-algo-bought-26-million-worth-stock-every-second
- blurgh
  
  Zero hedge are pretty darn clueless about HFT.
  
  If you want a decent explanation, go to Nanex – e.g. http://www.nanex.net/aqck2/3522.html and http://www.nanex.net/aqck2/3525.html Zero hedge reference Nanex in that link, but add a layer of useless BS on top.
richard77

I remember reading a story (maybe on BB, not sure) that stated that even moving the servers nearer Wall Street made sense from the economical point of view, to lower the latency (I think they work under the millisecond)
HFT is a tax, like those you had to pay if you wanted to use a bridge or a windmill in the Middle Ages, or maybe is more similar to bullies that steal your lunch money because they are stronger
- Beanolini
  
  even moving the servers nearer Wall Street made sense from the economical point of view, to lower the latency
  
  The LSE actually offer co-location within their own datacentre to companies willing to pay for low latency.
Ryan Lenethen

It was my understaning that they canceled all the trades, so the company didn’t actually lose 400 Million and change.

Though perhaps they should, it would at least so the risk involved in this kind of trading. As it is, just like the bailout, there is no repercussions to bad behaviour. Since they can’t lose, why not just take on more risk for potential more profit?

That said from what I have heard this wasn’t a HFT algorthim gone berserk, but rather some dummy left the testing script still attched to some modified normal trading software, which was full of silly testing transactions, which of course the software did in real life when hooked up to production and the go button pressed. So it sounds like simple carelessness not programming or algorythemic error.

Still simple carelessness casuing almost 500M in bad trades is pretty scary in of itself.
- blurgh
  
  Only trades in 6 stocks got cancelled, so they did lose a shedload and a half.
  
  http://www.marketwatch.com/story/flash-crash-rules-made-knight-keep-bad-trades-2012-08-07
deadcanard

People, PLEASE, can we stop the madness about millions of trades in the blink of an eye? NASDAQ yesterday had 2 million trades total for the FULL day (see http://www.nasdaqtrader.com/trader.aspx?id=FullVolumeSummary#) . There are no algo trading millions of times during a second. Why people keep copy/pasting these made-up numbers?
- http://www.facebook.com/profile.php?id=513504196 Matthew Holloway
  
  Are the high frequency trades included in the daily volume statistics for the exchanges? I seem to remember reading somewhere (google search fails me) that certain classes of trades, or certain canceled trades, are not included. Also, on your link it seems to me that the total volume for the NASDAQ on 8/9/12 was 4.1 million, but I may be reading that wrong as that chart is fairly complex.
  - deadcanard
    
    Good point. 4.1M is the right number. That does not change my point though. All trades are displayed in these numbers, HFT ones are included (and anyway the exchange does not know which ones are HFT or not so could not suppress them). Canceled trades are fairly rare, they’re usually called busted trades. You might be thinking about canceled orders, not trades.
- blurgh
  
  It’s not really the number of trades that are the pain, but the number/lifetime of the quotes (e.g. http://www.nanex.net/Research/bloodbot/bloodbot.html ).
  - deadcanard
    
    I was just commenting on the summary that included the usual non-sense on million of trades per second which is wrong by several order of magnitudes.
    
    The issue with number/lifetime of quotes/orders is up for debate though. For example, market markers (which a lot of HFT are) need to provide liquidity 90% of the time (or so – dunno the actual number) which implies a large number of cancelled orders when the price moves.
deadcanard

Ryan, they did not cancel all the trades. They did cancel a few but it’s negligible compared to the total number of trades. I disagree that they should. The BEST way to ensure a working market is to make you pay if you screw up big time and they did.
http://www.cruft-private-janitorial.com/ Michael Johnson

Beware the Quants – if you ban the algo’s you’ll see the rise of the Mentats. I’ve met a few, some of scary smart – and I ain’t a shlub myself. I believe the tax would make for far more enlightened trading and a general increase in safety.
Alan Wexelblat

ETA: I was wrong about what Knight did – the Nanex explanation is clear. Removed my erroneous response as I hate showing my arse on the net.

In response to deadcanard, the amount of volume made up by HFT is the subject of hot debate. Generally accepted numbers are in the 30% of average daily volume across exchanges is HFT, but that’s a very loose number and fails to take into account the activity in dark pools and it misses certain activity that could be counted as HFT but is masked, for example the NYSE’s parity trading. It’s also worth noting that the traditional exchanges, particularly NASDAQ and NYSE make up less and less of the ADV in US trading.

Third, I think Zero Hedge are generally in it for the lulz and should be read as such. They’re kind of like Drudge, in that they can be wildly inaccurate in one story and then astonishingly spot-on the next.
Halloween_Jack

Didn’t Charles Stross use a sentient trading program as a plot device in Accelerando or one of his other books?
http://profile.yahoo.com/LUIN37FB2NMFDY6RI6DEMCB33Y Joey

I work for a company that dealt with Knight on a fairly regular basis.

Knight wasn’t involved in high frequency trading, the media has gotten this extremely wrong.

What we do know is that Knight was testing their trading software in a QA environment, once the testing was complete they pushed the update to production. Unfortunately Knight did not test the software update adequately.

After that things get fuzzy, what we think is that they pushed the updates for production to their test environments at the same time pushing the passwords to the NYSE also to the test environments.

Now since it is a test environment, all that it is supposed to do is simulate market traffic, as such it doesn’t contain a market volatility algorithm to analyze the risk. All it does is one thing, buy and sell as fast a possible since it isn’t (supposed to be) trading real money.

within 10 min they began receiving calls from other market makers asking why they were sending them two weeks worth of volume in 5 min. Essentially the tester was buying and selling literally everything it could get its hands on. An interesting thing to point out is that Knights own traders were also trading with the tester.

we are almost positive about what happened next, Knight rebooted their environments three times. We know this by analyzing the volume of the NYSE and you can see on three separate occasions dips in volume for the length of time it would take to reboot a production trading environment.

This wasnt trading gone wrong, this was a massive fuckup by everyone at that company. from the devs to the traders all the way to the CTO. This was literally a perfect storm of incredibly bad decision making, people not checking their work, and not having adequate safe guards in place.

People may think that this could happen with any trading firm, and while the paranoia is sound its incredibly doubtful. what makes the Knight mistakes so tragic is how absolutely massive that company was. Most of you who do not work in finance or trading probably haven’t heard of them before this little mishap. for those of us that do work in trading, Knight is quite well know because during trading hours 13-15% of all trades in the US Knight has its hands in. most trading firms don’t even come close to trading 1/1000th of the volume Knight does.

I apologize about the grammar of this rant but I’m extremely sleepy