Mathematician Benoit Mandelbrot died earlier this week (the news just came out today). I got to know him while editing a piece that he and Nassim Nicholas Taleb wrote for Fortune in 2005, then interviewed him a couple times for Myth of the Rational Market. For a while after that we kept in touch—I remember getting an out-of-the-blue call from him once while riding a crosstown bus through Central Park—but I hadn't spoken to him in a couple of years. To commemorate his passing, here's an excerpt from the book dealing with his complicated (and, in light of the recent financial crisis, somewhat tragic) relationship with academic finance.
Mandelbrot was a Polish Jew who had emigrated to France in 1936, spent what would have been his high school years hiding from the Nazis, and then got a doctorate in mathematics at the Sorbonne. It was a 1949 book by Harvard linguist George Zipf that first piqued his interest in strange statistical distributions. Pick a text and rank the words in it by how often each appears, then graph the result, as Zipf did, and you get a fascinating pattern. “The curve does not fall smoothly from most common to least common word,” Mandelbrot observed. “It plunges vertiginously at first, then declines more slowly—like the profile of a ski jumper leaping into space, to land and coast down the gentler slope below.”
Such statistical distributions have become known as “power laws,” because one variable is exponentially related to the other. These patterns, which allow far more room for outliers than the standard bell curve, had first been observed around the turn of the nineteenth century in the distribution of wealth, and it was the statistics of wealth and income that Mandelbrot studied. Then he visited Hendrik Houthakker’s Harvard classroom and saw that cotton futures prices
fell into the same pattern as incomes and words. It wasn’t just the ski jump line; the data was also “self-similar”—that is, charts of small snippets looked just like those of large swaths. Mandelbrot was later to find similar patterns in historical climate data along the Nile, the coast of Britain, and the ins and outs of tree bark. After he dubbed them “fractals” in 1982, he was hailed as a visionary, one of the progenitors of the new science of chaos and complexity that was transforming physics and other fields. By then, though, Mandelbrot had long abandoned finance. At the beginning he had been warmly welcomed into the small but growing fellowship of random walkers. Gene Fama became his informal student. Harvard invited him to spend the 1964–65 academic year as a visiting professor of economics. He authored a paper that appeared not long after Samuelson’s in 1965 showing mathematically that a random market would be a rational one. “The first period was very nice,” Mandelbrot recalled. “They were receptive, but with an ominous cloud.”
The “cloud” was the frustration that developed among economists as they discovered how hard it was to work with Mandelbrot’s power laws. In his depiction of security price movements, variance—the measure of how widely scattered the different data points are—was infinite. For scholars who were just getting acquainted with Markowitz’s depiction of portfolio selection as a tradeoff between mean and variance, infinity was not helpful.
“Mandelbrot, like Prime Minister Churchill before him, promises us not utopia but blood, sweat, toil and tears,” wrote random walk ringleader Paul Cootner in 1964. “If he is right, almost all of our statistical tools are obsolete . . . Surely, before consigning centuries of work to the ash pile, we should like to have some assurance that all our work is truly useless.” Such assurances were not forthcoming, and before long, finance scholars had ceased paying attention to Mandelbrot at all. “The reason people didn’t latch on to that stuff is it’s not that tractable,” said Eugene Fama, who went from Mandelbrot disciple to Mandelbrot ignorer in a few short years in the 1960s. “It’s not that easy to deal with those predictions in a systematic way.”
Physicist M. F. M. Osborne, who visited UC–Berkeley’s Business School in 1972 to teach two finance courses, told his students that Mandelbrot’s ideas about infinite variance were “a stew of red herring and baloney.” Sure, there were jumps and dips in stock prices that couldn’t be shoehorned into a normal distribution, Osborne acknowledged. But for most purposes, it was OK to ignore them. The important thing was to figure out what you were measuring probability for:
You ask what is probable and what is improbable, but definitely not impossible. For rainfall you take 99% of the occasions (days) when you average less than two inches of rain . . . That kind of information is significant for grazing or agriculture, for what kind of vegetation is likely to grow. The improbable situation, which may give much more than 1% of the total rain which may fall, is really concerned with a different caliber of problems. Are the roads going to be washed away, is it safe to build a house in certain locations if you want to live there for twenty or thirty years?
The improbable-but-not-impossible was not something that bell curve statistics could address. But Osborne didn’t see any point in reinventing statistics to handle it. When it came to rare events, he argued, one had to look outside the statistics of randomness and identify actual causes for the anomalies. This required judgment and experience, two areas in which finance scholars possessed no comparative advantage. They focused instead on the probable.
