I was having an e-mail conversation with a long-time friend in the business yesterday about the Matrix Market blog post last week.
A little background – my bud has been modeling deals for more than twenty years, and we often discuss the internals of the behavior we try to capture in choosing variables and basis functions to apply to those variables.
He came up with a wonderful image of “algo” trading systems I thought I would share:
I might have compared algorithmic trading to two dueling rube goldberg machines.
He has a point. I don’t think many outside the business, including the regulators, realize just how Wall Street systems come into being, or how they grow over time.
In a way, part of the problem is the way Wall Street manages itself.
Transactions drive “front office” software development. Basically, someone (or a small group) in trading, banking or research think an idea might make some money. A person or a handful of people work on modeling the idea to see whether it is saleable.
They use whatever tools or prior work they have that might give them results quickest. It could be spreadsheets, or “tweeks” to the production systems, or for some real dinosaurs, a quick APL program.
If the initial modeling looks promising, some shops will move the project into a development team in the systems or research areas. Specs are written, meetings are held, and the front office revenue generators quickly lose patience. That loss of patience is why most trading desks, corporate finance departments and research groups have their own “quants.”
I’ve lived through this a number of times at a number of firms. At one shop, we had an early PC (Compaq 25 MHz) running our “temporary” Agency pool factor data base while another group was developing the permanent version on an IBM mainframe. Visitors to our little mortgage finance team were stunned to see a gigantic $20,000 9-track tape machine hooked up to the PC, but that’s the way we got Agency pool data and non-Agency loan data.
Our temporary system was our only system for the entire time I was at that firm. As far as I could tell, the only thing we got from the mainframe people was $10 million a year in interdepartmental chargebacks.
Of course, any revenue-generating group can do what it needs to do to make money. When market conditions, by definition temporary, suggest a deal can make a profit, no one can stand in the way, insisting the software must be developed according to standards that were probably negotiated and memorialized in a painful multi-year process.
Once a deal gets done, even if it’s done with a prototype, that software model often takes on a life of its own. At this point, most major investment banks try to incorporate the features of the new deal’s model into their existing systems.
By definition, the new capabilities were not part of the original design, so even the production systems are now modified on the fly…. and the Rube Goldberg machine gets its next section. It’s only worse if the prototype is used to do a second or third deal. In that case, the former prototype becomes the production model, and the Goldberg Variation is to try to connect the new model to all the old models.
Perhaps the most significant historical accident relating to Wall Street systems is how they first got built, and who built them. In the early 80’s, Wall Street got its first big round of “rocket scientists.”
They tended to be mathematicians, physicists or economists by training. I can’t recall any of the early model-builders being trained systems developers.
What happened was that management looked around when they had a hard problem that the computer might help. They found the smartest guy in the area that might be able to give them the answers they needed. When they found somebody that could program the computer to give the answers, they kept going back to that person.
What management couldn’t know was the difference between programming and system development. I used to try to describe it to my management as follows:
Any smart person in business can write a decent memo. Some can even write excellent memos. Does that mean they can write a best-selling novel or a hit Broadway play? Of course not. But they’re both words on paper, right?
If you look back to the groups that produced the risk management and deal structuring models of the 80’s and 90’s, you’ll find very smart people who probably never tried to develop major sustainable software systems.
Just thought you’d like to know why the systems Wall Street relies on tend to be so poorly designed. The answer is that they weren’t designed per se, but grew over time, and the people put in charge of the projects weren’t system architects.
As we blow through a few more technical support levels in the markets today, we can be pretty sure that all the Rube Goldberg algo trading systems are coming to the same conclusion, yet again.
Good luck to us all. We’ll be needing it.