Over the weekend, there was an interesting article in the Times Magazine by Joe Nocera, examining the role of Value at Risk (VaR), a risk assessment model used by Wall Street that, he suggests, was meant to prevent something like the financial crisis we now have. It's a fascinating piece, and I recommend it, but I think the conclusion to draw is slightly different than the one Nocera suggests.
Part of the problem is that Nocera clearly isn't a "quant jock" or anything approaching the kind of in depth data analyst like the ones he interviews for the piece. His frankness in admitting that he doesn't understand all of the complicated statistical terms is wonderfully charming in a writer... but not knowing is not knowing, and his lack of understanding tends to show. I've never had a stats course, but thanks to my Mom and several former co-workers, I was clearer on the kinds of things Nocera brings up than he is. It's not hard, really, to learn a little something.
But Nocera's reticence indicates the larger problem here - the reliance of those who don't necessarily trust math on complicated math formulas to provide the data they need. The problem with risk analysis, as I mentioned over at Ezra's (where he plays Captain Obvious in reaching the same conclusions Nocera did), isn't that the risk analysis got some things wrong; it's that when signs pointed towards too much risk, people ignored or denied them.
The problem with numbers is that they are cold. Cold calculations can tell you, often, to be more ruthless and less romantic in your assumptions, to abandon hope and make tough, expedient decisions. Nocera opens his article by pointing to an anecdote about Goldman Sachs having identified bad bets in 2007, and being able to bet against the market after careful analysis. Nocera uses this example to suggest that the "human factor" - that looks beyond the numbers, or looks more carefully - is what's needed... but I think the point is the opposite: the Goldman folks show that, when you follow the numbers, wherever they lead, you can make the hard choices.
To be sure, part of the problem with VaR is that it is not a perfect model - there isn't one, really. It's certainly true that part of the problem leading up to this crisis is that excessive reliance on VaR to predict the future was part of the problem. But just as problematic is the fact that, using data analysis, some people were able to see the housing bubble, the subprime mortgage mess, and other aspects of the crisis (not to mention, BTW, identifying the Madoff scandal years before he confessed). Numbers and data analysis don't get the blame for where we are; the failure was a human one, the "irrational" part of Irrational Exuberance, the part where a rational, dispassionate examination of the facts would take you somewhere less certain, less comfortable.
This reality has all sorts of implications - for one thing, it's worth keeping in mind as we discuss the health care crisis and the need to institute "Best Practices" that standardize treatment for some basic conditions: doctors resist Best Practices much the way traders deny risk assessments; that is, they often go with "feelings" and anecdotal evidence rather than hard scientific conclusions, because they... are human. That's why what can seem like cold, unfeeling "insurance analysis" about treatment benefits can seem the problem, rather than the doctor's advice. We trust a person. We don't trust the numbers.
In the end, Risk Assessment, like other data analysis, is only as good as the person analyzing the result; someone who trusts the numbers, but knows the limits of their value (i.e., understands that no model is perfect or anticipates every scenario). VaR didn't miss the bubble... it just wasn't asked to find it. And the indicators that suggested a bubble were too often ignored or dismissed as "bad numbers" by people who should have listened. And in the end that's why we're unlikely to prevent every future crisis, no matter how we try to regulate markets going forward; our human capacity to believe in fantasy and ignore reality - loike the hard numbers - is remarkably resilient. It's risky business, because we dare to indulge our fantasies. The alternative of living without them, I suspect... is not an option.
Comments