The Fundamental Problem With Financial Models

I know you’re in a real estate mood now that the election is done with… so let’s talk about it more, if only tangentially.

Hey DQYDJ readers, remember that whole real estate bubble popping thing which started a few years ago?  Now, there is plenty of blame for that little incident to go around, but one of the largest issues with the bubble and the ensuing collapse was the use of improper valuation techniques in securities related to real estate.

If you want me to be even more specific, I’ll gladly comply.  David X. Li introduced his gaussian copula to the world in 2000, meaning for it to be a model which could predict the performance of CDOs – collateralized debt obligations – and won a Nobel Prize for it.  Take note of that word, ‘predict’.

You’re 7’1″? According to this model that’s impossible!

Real Life vs. Predictions

Now, David Li isn’t the villain here.  Humble about the prospects of his own work, he even seems to have predicted the issues with the overuse of his model in a 2005 WSJ article where he stated, “The most dangerous part is when people believe everything coming out of it.”  Right he was, eerily enough.

In fact, high finance faces the same modeling limitations as does hard science – the truth is, models generally work out to be a very good approximation of actual behavior, instead of describing it perfectly.  Just as unified field theory is an area in constant search for hidden variables, high finance is a constant search for higher correlation to real life behavior.  The problem comes when people conflate a model’s predictions with truth, and ignore the corner cases (or the black swans and long tails.)  You see, for the uninitiated in the audience, a copula is a distribution function, while ‘Gaussian’ refers to a ‘normal’ distribution – think a bell curve in however many dimensions (variables) a formula has.  That’s right – a normal curve, by definition an approximation of a larger population.

The vicious cycle of collapse is pretty obvious at this point – and it will continue to repeat itself.  With no better model, it works something like this:

  1. Pre-model, a market works in a certain way – organically, depending on the whims of individual traders and investors operating worldwide
  2. A model is introduced, either academically or internally to a finance company, which attempts to predict the market in question
  3. If no competing models are developed, the singular model is disseminated
  4. The model itself, no longer the individual traders, starts to dictate the movements of the market.  This is because of algorithmic trading – at some point, the trades made predicting the individual traders represent more volume than the original trades.
  5. The model collapses in on itself – since every one of the common cases is already covered, the market itself is driven to the extremes – areas where the model is undefined.
  6. Inevitable collapse, as the behavior of the market is not hedged in the models used.

Feedback Mechanisms

I don’t think I’m breaking any new ground with this 6 step model, but if I am, contact me and we can co-author a paper and win a Nobel Prize.  In all seriousness, I’m just describing feedback – where previous information influences the present.  Think pointing a microphone at a monitor speaker – it quickly escalates to a point which neither the microphone or the speaker are designed to handle.  That’s right – the exact same inevitability which can rip speaker cones can collapse a market.

Why Doesn’t Collapse Happen Everywhere there is Algorithmic Trading?

Sometimes, the sheer volume of trades is enough to cover up negative feedback effects.  Consider a farmer who, for some reason, sells corn into the marketplace yet purchases corn at the store.  His purchase lowers the supply of corn at the same time as his growing increases it – he affects the price on both sides, yet he is such a small player he doesn’t noticeably move the market.  Another reason would be model improvements.  When some people recognize the weakness in a model it can be improved – either with an improvement to the model itself (known as heuristics to my computer science bretheren) or with a new model entirely.  This is the case in the options market, where Black Scholes is but one of a number of options models employed by finance firms.

That’s right – subtle differences in similar models between firms means that collapse is asymmetric.  Bear Sterns might collapse simply because there model didn’t flag a concern until one day after Goldman Sachs, for example.  Marginal improvements in processing power, latency, and yes, improved heuristics, might be all that separates a massive gain from a bankruptcy.  Just like Warren Buffett said, sometimes it really is like picking up nickels in front of a steamroller.

The other reason?  For whatever reason, perhaps the models haven’t imploded yet.  Otherwise, perhaps the spread is so low that an inefficiency in a proposed model is worthless to exploit (some inefficiencies are smaller than the transaction costs to correct them).  The truth is, any ‘perfect’ model would have to also predict its own effect on the market (and the proportion of the market affected!), a near impossible task… although conceptually possible.

The best way to prevent this sort of stuff?  Stop treating models and predictions as truth.  The fact remains – any model is a guess.  Just like you don’t know the value of your house until it sells, you can’t know the price of a security until it sells.

Beware of false precision – and take this to heart.  The best move you can do in your own portfolio is to build in error tolerance into your own predictions.  Maybe you think that $8 stock is worth $20?  Don’t be the CDO market – treat that second number as a range of $16-$20!


  1. greg says

    I think your analysis misses a very important part of the issue in your list of 6 steps: *mis-use* of the model — a special case of “garbage in, garbage out”. Saying that “the model collapses in on itself … [where] it is undefined”. I don’t think the model here was “undefined”, but rather that people grossly mis-judged the correlation of the assets being securitized (especially considering even the highest traunches were clobbered).

    From that sense, putting so much emphasis on a forumla seems a bit like blaming a car’s unintuitive lack of acceleration limit from stop lights for a driver hitting a pedestrian.

    • says

      Let me elaborate on that point: at some point I theorize that the model becomes the market. Picture a market with only individual traders which creates some efficiency described by a model. Soon, an algorithmic trader moves into the market and the spread starts to narrow.

      If the potential for profits is large enough, more algorithmic traders will enter the market – and more and more volume is attributable to computers as opposed to individual traders. The models are designed to guess how investors will react – and now they are playing against themselves. This is where the feedback comes in; when the computers start one upping each other eventually it will spiral to a place where the algorithm falls short (I know that feeling at my day job, ha).

      And yes, they misjudged it, and likely over-fit the model to the past. But a model is never reality, it just attempts to explain reality.

  2. says

    If you follow it you can find the mismatch in the insurance industry all the time. A company will come out with an annuity or product with certain riders that just sound amazing. All other companies will follow suit (I assume the actuaries are all depending on all the same wrong info). Advisors sell the hell out of the product. One company pulls it then they all pull it LOL

    • says

      Got any products offhand? In my head the best example I can come up with is hurricane insurance in Florida – when the Government stepped in it started a private hurricane insurance death spiral and the product pretty much doesn’t exist anymore, ha.

  3. JT says

    The models can seemingly run the show for a very long time. We’ve talked about this before, but CAPM has to be one of the longest running failed models the world has ever seen.

    I wonder when this train of thought will flip. I keep hearing that the markets are more and more efficient, but because of the way the math checks out, it leads me to believe that the meanings behind the words “efficient” and “correlated” are becoming one in the same. The model tracked correlation while dubbing the markets efficient. Now the markets are more correlated, so its perceived mathematically as efficiency. When will the craziness end?

    • says

      Efficient seems to me to mean ‘all is smooth until it blows up’.

      And yeah, you know my feelings on CAPM. Maybe we should write some super long CAPM rant posts to display our true feelings?

  4. 101 Centavos says

    Good thoughs, and fascinating article from the WSJ. Both should be required reading for any PF blogger. You’re correct in surmising that all this Gaussian copulation builds on itself, creating a shadow world of financial products increasingly decoupled from the grubby real world. What’s the next wunder-product beyond synthetic CDOs, Totally Artificial Promises to Default (TAPD)?

  5. says

    A model is only as good as the assumptions. Since we are all flawed, it’s impossible to rely on markets.

    What a sell down post QE3 huh? I’m jumping 100% back in. Why not. After a 7.5% sell-off, I like the S&P500 at 1,350. Gonna buy some structured notes.