Models Behaving Badly : Summary

Emanuel Derman’s book, “Models.Behaving.Badly”, gives a physicist and quant’s perspective of models Vs theories, their nature, what to expect of them, how to differentiate between them and how to cope with their inadequacies. Derman starts off by citing a few incidents from his childhood in Africa, where political models, social movement models failed badly. He then talks about the need to understand the crucial difference between theories and models, thus providing the necessary motivation for a reader to go over the book. The book is not as much as describing the precise faulty nature of all the financial models, which would be quite a stretch that it would not fit in a single book. The book is more about the kind of basic things that a financial modeler must always keep in mind and remain humble in his pursuit, always remembering that his model will always say “what something is like?”. It will never be able to answer “what something is”. Let me summarize the main chapters of this book.

Metaphors, Models and Theories

In this chapter, Derman makes the distinction between “theory” and “model”. He starts off by quoting Arthur Schopenhauer’s metaphor about sleep.

Sleep is the interest we have to pay on the capital which is called in at the death; and the higher the rate of interest and more regularly it is paid, the further the date of redemption is postponed.

In essence, this metaphor is saying that Life is temporary nonblackness. At birth you receive a loan, consciousness and light borrowed from the void, leaving a hole in the emptiness. The hole will grow bigger each day. Nightly, by yielding temporarily to the darkness of sleep, you restore some of the emptiness and keep the hole from growing limitlessly.

What is the relevance of metaphors in a book that aims to talk about models? Well, the connection is this: A language is nothing but a tower of metaphors, each “higher” one resting on a “lower” one and all of them resting on non-metaphorical words and concepts. Metaphors by default say that X is something like Y. This is similar to a Model that tells us “what something is like”. Derman then cites Dirac, who came up with a similar metaphor in physics where he pictured positron, as a brief fluctuation in the vacuum. However what grounds Dirac’s work is the Dirac’s equation. This fundamental equation when combined with the metaphor successfully predicted the existence of a particle no one had seen before. Metaphor by itself seldom qualifies as a theory.

Why do we need Models at all? Because the world is filled with quasi-regularities that hint at deeper causes. We need models to explain what we see and to predict what will occur. So, the important thing to note is that Models are a kind of proxy to the deeper causes. Whatever be the model, i.e. a Model T from Ford, a fashion Models, an artists’ Models, a weather Model, an economic Models, Black-Scholes Model; they are all a proxy to the real world which is too complex for us to understand. A model thus is a metaphor of limited applicability, not the thing itself. A model is a caricature that overemphasizes some features at the expense of others. The world is impossible to grasp in it’s entirely. We focus on only a small part of its vast confusion and here is where models come in, They reduce the number of dimensions and allow us to make a little extrapolation in that proxy world.

**_What is a theory?
_**Weather model’s equations are a model, but Dirac equation is a theory. Similarly you build an econometric model for interest rate prediction, the equations are a model, but the Newton’s equation is a theory. What’s the difference? Derman goes on to give a nice explanation to explain this difference.

Models are analogies; they always describe one thing relative to something else. Models need a defense or an explanation. Theories, in contrast are the real thing. They need confirmation than explanation. A theory describes essence. A successful theory becomes a fact. So, basically what he saying is “Dirac equation IS the electron”, “Maxwell’s’ equations ARE electricity and magnetism”. A theory becomes virtually indistinguishable from the object itself. This is not the case with Models. By default, there is always a gap between models and real thing. A Theory is deep, A Model is shallow. A Theory doesn’t simplify. It observes the world and tries to describe the principles by which the world operates. A theory can be right or wrong but it is characterized by its intent: the discovery of essence. You can layer metaphors on the top of the equations of a theory but the equation is the essence.

The chapter ends with Derman narrating his experience with the doctors. About 25 years ago, Derman went through a retina surgery and since then a peculiar problem haunts him, Monocular Diplopia (seeing double in one eye). This problem keeps recurring at random times. In 2008 he gets an acute problem in his eye and visits n number of doctors for a remedy. None of them seem to get a handle on the problem which appears like a problem relating to retina. Finally a technician cracks the problem by looking at the problem by a simple examination of the symptoms and not bringing in too many assumptions about the problem. Derman then says that this expert problem persists in many of us, who do not step back and question the assumptions from time to time. If you ask an active fund manager who has beaten the market for x number of consecutive years, he starts attributing his success to skill rather than luck/ extraneous factors. Similarly if you go and ask a passive fund manager, he might scorn at arbitrageurs, who breathe day-in day-out “the inefficient market hypothesis”. I remember something from Twyla Tharp’s book, “The creative habit”, that goes something like this: one should always consider oneself to be inexperienced in whatever they pursue. Putting X number of years, managing X number of people, executing X number of ideas, doing X number of surgeries etc should never make one feel that he/she is an expert. Because once you think you are expert, you will definitely develop some amount of fear, be it financial/psychological/ social / intellectual/ etc… however Inexperience erases fear.. One must constantly question the assumptions in a model so that one does not risk of becoming an expert in one specific area / specific type of modeling. This also means that if you are P type quant(buy side), it might make sense to develop a Q type quant model(sell side) and vice-versa, the reason being the kind of modeling that happens on the two sides are different and you never know which model might becomes useful in a particular time frame.

The Absolute

Derman talks about the nature of theories by illustrating Baruch Spinoza’s analysis of human emotions. Spinoza theory is similar to Euclid’s geometry. Euclid starts off with a few axioms and develops entire structure of geometry. Spinoza does the same with respect to human emotions. The following diagram summarizes Spinoza’s logical structure behind the theory:

This shows that Spinoza’s theory basically has three primitives, pain, pleasure and desire. There are derivative emotions like pity, cruelty etc.

In one of the blog entries, Derman gives a beautiful summary of the Spinoza theory:

That thing is called free which exists from the necessity of its nature alone, and is determined to act by itself alone. But a thing is called necessary, or rather compelled, which is determined by another to exist and to produce an effect in a certain and determinate nature.

Underlyers <==> free. Derivatives <==> compelled. Derivatives suffer passions, Underlyers have actions. Try to be an underlyer, not a derivative

What’s the relevance to the context of the book? Well, as one can see, Spinoza’s theory is not saying “something is like something”. It basically states things and shows the way world IS, or in the case the emotions ARE. Also there is also something that one can learn from Spinoza, the way to extend our inadequate knowledge. Spinoza was of the view that some of the ways to extend our inadequate knowledge was a) via particulars, b) via generalities and c) via intuition. The author gives a set of examples to show that most of breakthrough theories in science occurred via intuition that various scientists had about their inventions. Once intuition was firmly in place, ONLY then the mathematical framework was used to show it to the world.

The Sublime

Derman gives a whirlwind tour of electromagnetism and the importance of Maxwell’s intuition in development of the entire theory. Along the way Derman drives home the point that Maxwell’s equations IS electromagnetism. The theory becomes indistinguishable from the object itself. Derman also mentions about “renormalization” in physics and “recalibration” in quant’s world and says both are completely different though they seem to be connoting the same meaning. In Physics the normal and abnormal are governed by the same laws, whereas in markets the normal is normal only while people behave conventionally. In crisis the behavior of people changes and normal model fails. While Quantum electrodynamics is a genuine theory of reality, financial models are only mediocre metaphors for a part of it.

The Inadequate

This chapter is the core of the book where Derman relegates almost all the financial models to imperfect models and says financial modeling is not the physics of the markets. Like there are theorems in mathematics, laws in physics, one tends to use phrases such as “fundamental theorem of finance” etc, which actually makes no sense. Derman raises a critical question, “How can a field whose focus is the management of money and assets possess a theorem? and physics a field which deals with real life lack a fundamental theorem “.

The basic thing that every financial modeler tries to have a grip on is “value” of a security. Unlike physics where fundamental particle’s value / charge is absolute, there is nothing absolute about the value of a financial security. Value is determined by people and people change their minds. So, it does appear that whatever value that equity-research people, technical analysts, quants come up with, it is all a big fraud.

Derman rips apart the EMM / CAPM model and raises the critical question: Is EMM a theory or a model and concludes that EMM is not a theory, it is not even a good model, it is an ineffectual model. No financial theory can dictate what return an investor should expect in exchange for taking risk that too depends on appetite and varies with time. CAPM is a useful way of thinking about a model world that is quite often far from the world we live in. Derman , on the other hand, does not criticize Black-Scholes model and says that it is the closest to robustness that any model has been in finance.Why? the replication argument says that the risk premium for a stock and risk-free bond should be same as the option, an argument which is robust even though one can debate about the computation of risk premium.

The chapter concludes with a reference to the movie Bedazzled where the protagonist fails to woo the waitress in all the seven scenarios he seeks a wish for. Devil outwits in all the seven scenarios. Derman concludes saying

The difficulty of the hopeful would-be lover is the same difficult we face when specifying the future scenarios in the financial models; as does the devil, markets eventually outwit us. The devil is indeed in the details. Even if the markets are not strictly random, their vagaries are too rich to capture in a few short sentences or equations.

Breaking the cycle

In the final chapter of the book, Derman comes to the rescue of models and says that they are useful in finance because

• Models facilitate interpolation, whatever be its rudimentary form
• Models transform intuition in dollar value( example – implied vol of the option)
• Models are used to rank securities by value

And the right way to use models are by following a few rules like

• Avoid Axiomatization
• Good Models are vulgar in sophisticated way
• Sweep Dirt under the Rug, but Let Users Know about it
• Use imagination
• Think of Models as Gedankenexperiments
• Beware of Idolatry

The book ends with Financial Modeler’s Manifesto, an ethical declaration for scientists applying their skills to finance.

The Modeler’s Hippocratic Oath

• I will remember that I didn’t make the world, and it doesn’t satisfy my equations.

• Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.

• I will never sacrifice reality for elegance without explaining why I have done so.

• Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.

• I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

Takeaway :

This book is a must read for financial modelers as it shows that , whatever model one comes up, it is just a metaphor. If one keeps this in mind while developing models, it is likely that at least the end result will be a vulgar but useable & practical model instead of a mathematically elegant but a totally useless model.