Major Breakdowns and the Alignment of Errors

Who is responsible for the housing crisis? Some candidates are borrowers, lenders, rating agencies, and securities investors.  However, attempts to blame one party or another fail, because the crisis is the result of a combination of errors by different parties which all aligned. Think of a wedge of Swiss cheese; to see through it, all the holes must line up. This approach is explained in James Reason’s Human Error, and illustrated in a diagram from that book:

In the housing crisis, here are some errors which had to align to get to where we are today:

1) Borrowers took out loans they couldn’t afford

2) Lenders made loans to borrowers which the borrowers couldn’t afford

3) Ratings agencies rated securities comprised of these loans as safe

4) Security purchasers relied on the erroneous ratings and bought the securities

Any of these parties could have averted the crisis had they avoided their respective error.

In any complex system, it’s often more likely that a major breakdown is the result of an alignment of errors, rather than the failure of a single component.

People Remember Your Worst Moments

I’ve previously posted on why people tend to remember your worst reasons for your decisions (see here). This is true of all kinds of behavior – Tom Cruise spends a lot of time in the public eye, but what most people remember is his bizarre behavior in this video of an interview with Oprah.

Via Newmark’s Door, here are two entertaining collections of famous worst moments:

Seven Great Talk Show Train Wrecks

The Twenty Weirdest TV Interviews of All Time

It’s obvious, but if you want to be an effective manager, don’t say or do stupid things.

The Inevitability of Errors

Errors are inevitable – no matter what the stakes, no matter how much you practice, things are going to go wrong a certain percentage of the time in any complex task or decision. The New York Times has an article with an excellent example: basketball free throws.

There is nothing in sports as straightforward as a free throw; the equipment is always the same, the geometry is constant, and there is no defense interfering. The only variables are the player’s concentration and control over his or her body. And yet, at the highest level of the game, it goes wrong 25% of the time, year after year after year:

In the National Basketball Association, the average has been roughly 75 percent for more than 50 years. Players in college women’s basketball and the W.N.B.A. reached similar plateaus — about equal to the men — and stuck there.

The general expectation in sports is that performance improves over time. Future athletes will surely be faster, throw farther, jump higher. But free-throw shooting represents a stubbornly peculiar athletic endeavor. As a group, players have not gotten better. Nor have they become worse.

“It’s unbelievable,” Larry Wright, an adjunct professor of statistics at Columbia, said as he studied the year-by-year averages. “There’s almost no difference. Fifty years. This is mind-boggling.”

And it’s not like the stakes aren’t high:

Last season, Memphis was 38-2 despite making only 61 percent of its free throws, missing an average of nearly 10 a game. The Tigers lost the national championship game after missing 4 of 5 free throws in the final 72 seconds against Kansas, which had made a late 3-point shot to tie the game and won in overtime…About two-thirds of a winning team’s points in the final minute typically come from the free-throw line…

Obviously, we need to work to eliminate mistakes and design systems to minimize the chance of them occurring. But, a certain percentage of the time errors will happen. Learn what you can from them and move on.

Keep the Bonuses, Change the Criteria

Thomas Gehrig and Lukas Menkhoff at VOX survey the research on bonuses and suggest we keep them, with some changes. An excerpt:

In fact, banks themselves are trying to correct their internal incentive schemes in order to re-adjust incentives on longer horizons. They seem to largely agree that, prior to the crisis, their systems may have been excessively short-sighted, and they are now trying to base rewards on more sustainable performance criteria such as average growth rates and volumes across longer sampling periods.

My suggestion (posted here) is measuring shareholder equity over a five year period.

Management, Feedback, and US Air Flight 1549

Via The Big Picture, an amazing animation with audio of the US Air Flight 1549 takeoff and landing.

It’s striking how the flight controllers’ understanding of the situation lags actual conditions. I think there’s a parallel with management and regulator understanding of what’s happening on the ground (or in the air, in this case) during rapidly changing conditions.

A Modest Proposal to Reform Management Compensation

Nassim Nicholas Taleb has a post on incentive compensation in Financial Times which describes the problem with the typical bonus plan. Here’s an excerpt:

Take two bankers. The first is conservative. He produces one annual dollar of sound returns, with no risk of blow-up. The second looks no less conservative, but makes $2 by making complicated transactions that make a steady income, but are bound to blow up on occasion, losing everything made and more. So while the first banker might end up out of business, under competitive strains, the second is going to do a lot better for himself. Why? Because banking is not about true risks but perceived volatility of returns: you earn a stream of steady bonuses for seven or eight years, then when the losses take place, you are not asked to disburse anything. You might even start again, after blaming a “systemic crisis” or a “black swan” for your losses. As you do not disgorge previous compensation, the incentive is to engage in trades that explode rarely, after a period of steady gains.

Taleb’s solution is radical:

We trust military and homeland security people with our lives, yet they do not get a bonus. They get promotions, the honour of a job well done and the disincentive of shame if they fail. Roman soldiers signed a sacramentum accepting punishment in the event of failure. This is prompting me to call for the nationalisation of the utility part of banking as the only solution in which society does not grant individuals free options to look after its risks.

I like the military analogy, but I think Taleb goes further than needed in eliminating bonuses entirely, and I think he doesn’t go far enough when he limits the proposal only to banking. Here is what I propose for all managers of public companies (or private ones which rely on public support, for example, a privately held bank).

1) Your base pay is limited to it’s military equivalent:

2) The rest of your compensation is in the form of whatever bonus your company thinks is appropriate – no mandated limits or performance criteria. However, the bonus must be paid in cash, and it gets paid into a federal trust fund.

3) After five years, the trustees compare the shareholder equity for the year the bonus was paid with current shareholder equity. If equity is the same or has increased, the bonus is paid. If shareholder equity has declined the bonus is forfeited and used to offset costs of administering the trust and then for some good purpose (education, or unemployment benefit funding). You don’t have to stay at the company to get the bonus.

This approach will cause managers to be thinking about values five years out, which should be a long enough horizon to avoid the problem Taleb describes. It also encourages managers to control employees engaging in risky actions (e.g., traders), and to move on if they think the company is taking excessive risks.

There are plenty of potential objections, but I think the most serious one is that some risk taking often produces real long term rewards, and this approach will dampen productive risk taking in public companies. That’s true, but I think it’s mitigated by the fact that private companies and partnerships will still be around to take big risks, and to provide opportunities for those who can’t wait five years for their reward.

Lumpers, Splitters, and Decisionmaking

There are two kinds of people; lumpers, and splitters. Lumpers look for commonalities, and lump things into groups. Splitters look for differences between things, and treat each individually.

For example, you, Ernst, and Ralph are looking at a toy poodle, a labrador, a newfoundland, a sharpei, and a mastiff playing in a dog park. You ask them each to break the dogs into groups.

Ernst says there is just one group – they’re all dogs. Ernst is a lumper. You press him, and he comes up with this group:

Color	Dogs
White	Toy Poodle
Black	Labrador
	Newfoundland
Brown	Mastiff
	Sharpei

You press him some more, and Ernst identifies this group:

Size	Dogs
Large	Newfoundland
	Mastiff
Medium	Labrador
Small	Toy Poodle
	Sharpei

Now it’s Ralph’s turn. Ralph says there are no groups, all the dogs are different. You press him, and Ralph comes up with this:

Color, Size	Dogs
White, Small	Toy Poodle
Black, Medium	Labrador
Black, Large	Newfoundland
Brown, Small	Sharpei
Brown, Large	Mastiff

Note that, although Ralph has identified categories, each dog has his own.

Neither of these approaches is wrong – you need both approaches to make effective decisions. For example, an extreme lumper will not be effective at working out loans, because he or she will offer the same modification to every borrower, and the same modification will not work for every borrower. An extreme splitter will also be ineffective, because every modification will be customized for each borrower, and the time that takes will result in few modifications being completed.

The error lumpers make is the failure to identify relevant differences. The error splitters make is to identify differences which are not salient, encumbering the decisionmaking process.

Ideally, what you want is to identify the salient differences between borrowers (a splitter skill), and group the borrowers according to their salient differences (a lumper skill) so similar modifications can be offered to the each group.

Obviously, people can put on different hats and function as a splitter some of the time and as a lumper on other occasions. I do believe, however, that people have a preferred mode.

These ideas are drawn from Scott Page’s excellent book, The Difference, which I highly recommend.

Give Me a Reason

Everyone who has raised a child has gone through the “Why?” stage. Humans are wired to figure things out, and asking why is often the most direct route to understanding.

In his book Why? Charles Tilly outlines a framework for how this question is answered. There are four approaches: Conventions, stories, codes, and technical accounts.

1) Conventions are standard explanations of an event which don’t attempt to establish causality (e.g., you explain to a friend you have not responded to an email because you are swamped at work). You and your friend both know this is not literally the reason (after all, you’re not at work 24 hours a day), but your friend accepts the convention and does not make you account for every minute since you received the email.

2) Stories are more detailed narratives which provide more details and present a simplified causality model. If you are a 20 minutes late to work, a conventional comment about traffic being bad will probably suffice. If you keep your spouse waiting 20 minutes in a restaurant for an anniversary dinner, you will probably have a very detailed story establishing you left with plenty of time, the nature of the accident causing the traffic delay, etc.

3) Codes explain actions by referring to existing rules. For example, an employee is not reimbursed for five 40 mile business related trips but is reimbursed for one 60 mile trip because the policy is no reimbursements for trips less than 50 miles.

4) Technical Accounts are detailed explanations of events which attempt to prove causality (or at correlation) with evidence. An example is an engineering study explaining the failure of a dam.

Here is how the rejection of a loan might be explained:

• The borrower had bad credit (convention)
• I approved a loan to someone who had multiple late credit cards just like this applicant and the prior loan defaulted…(story)
• The investor will not buy the loan if the borrower’s FICO score is below 600 (code)
• Studies have shown that borrowers with FICO scores below 600 default 40% more frequently…(technical account).

Each approach has drawbacks. Conventions are by definition incomplete and don’t fully establish causality or educate. They’re fine as a kind of shorthand when nobody really cares much about the answer, but are usually inappropriate when someone really wants to know why.

Similarly, codes are a shorthand explanation which don’t educate (except about the code itself) or explain. Saying an investor won’t buy a loan if the FICO score is below 600 doesn’t explain why that’s the investor’s policy any more than “Because I said so!” explains to a child why they have to go to bed at 9:00 PM. Codes work best when everyone understands the reasons underlying the code already.

Stories are extremely effective at conveying values and information. Like asking “Why?”, narrative explanation and understanding seem to be hard wired into humans (picture cave dwellers huddled around the fire listening to stories of the day’s hunt). However, if you’re using a story to convey a truth, you need to make sure your story represents a truth, and not an isolated instance. For example, your story about how dangerous unleashed dogs are because you were once bitten by one will not resonate with someone who spends a lot of time around unleashed dogs at a dog park. Also. stories are seductive, and as discussed in in a previous post people are all too willing to accept them uncritically.

Technical accounts use evidence and establish causality to answer the question. They are the nuclear weapons of explanation – after a good technical account you really know why. The problem with technical accounts is they tend to be boring, and more information than the audience is looking for. They also tend to be limited in scope – it’s hard to establish the why of big events conclusively.

Given each has drawbacks, which is the best form to use? They can each be appropriate in certain contexts. Tilly introduces the concept of “superior stories”; stories which draw on the strengths of a narrative format, but which are underpinned with the evidence and causal links found in a technical account.

Give Me a Plausible Story and I'll Believe You

There is a great, illuminating mind game in Robert Burton's On Being Certain: Believing You're Right Even When You're Not. Read this paragraph, and as you read think about what you're feeling:

A newspaper is better than a magazine. A seashore is a better place than the street. At first it is better to run than to walk. You may have to try several times. It takes some skill, but it is easy to learn. Even young children can enjoy it. Once successful, complications are minimal. Birds seldom get too close. Rain, however, soaks in very fast. Too many people doing the same thing can also cause problems. One needs lots of room. If there are no complications it can be very peaceul. A rock will serve as an anchor. If things break loose from it, however, you will not get a second chance.

Irritating? Frustrating? I can fix that with one word - kite. Reread the paragraph, and notice how everything fits. Feels better, doesn't it?

Humans are problem solving machines. Give them a puzzle and they're frustrated if they can't solve it, satisfied if they can. This is deeply wired in us at a physical and emotional level, and the drive to solve problems is what keeps us moving forward.

But, there is a potential problem with our wiring. Try rereading the paragraph and recapture the feeling of not knowing what it is about. You can't; your brain says "kite" with every sentence. What if I told you the paragraph is not about kites, and to reread it to find a different solution? You almost certainly will not be able to do that either. Once your brain "knows" an answer it is very difficult to dislodge that answer and consider alternatives.

Bank executives took excessive risks out of greed - plausible, but true? American car makers are on the verge of bankruptcy as a result of capitulating to unions in the past - plausible, but true? Teaser rates were an important contributor to subsequent mortgage defaults - plausible, but true? Honestly, how much actual evidence have we seen supporting these propositions? It's frightening to realize how much we accept as true simply because it's plausible.

Daniel Mudd, Richard Syron, and Kerry Killinger are not Stupid, Greedy, or Crooks

An article in today's Seattle Times says "Washington Mutual suffered an ugly death, leaving thousands without jobs, homeowners facing foreclosure, a civic crater in Seattle and a 100 year old institution flushed away by miscalculation and greed...Shareholders are also appalled by what they see as incompetence, and worse, by executives in their failure to protect the company...The Ontario Teachers Pension Plan Board of Canada, a major shareholder, has filed a securities class action complaint against Washington Mutual and some officers, including former Chief Executive Officer Kerry Killinger." Daniel Mudd, former CEO of Fannie Mae, and Richard Syron, former CEO of Freddie Mac, have been similarly lambasted (see, for example, His Name is Mudd) and the subject of calls for criminal investigations.

First, let me make clear that I don't know these men; they actually could be stupid, greedy and crooks. But, I think that's unlikely; my guess is they're probably really smart guys, and as honest and ethical as the rest of us (here are a couple of interesting posts arguing the elite really are elite, and the difficulty of assessing the ability of those at levels above our own). I think there is a much simpler explanation for the decisions which blew up their companies:

They tried to earn what they were being paid.

It's admirable, of course, to earn what you're given -if they didn't try to do that, they would be justly criticized. In 2007 Mr. Killinger's compensation was $14,364,883, Mr. Mudd's was $14,231,650, and Mr. Syron's was $14,497,981. What should they have done in 2008 to earn that money?

Lenders compete on price (interest rates, fees, processing costs), execution (speed and certainty of delivery of the promised transaction), and terms (leverage, documentation, covenants). By all accounts, all three companies were very competitive on price and execution. That leaves terms. As long as there are lenders willing to lend more aggressively (higher LTV loans, lower income ratios, less documentation, fewer reserves and covenants) conservative lenders will lose market share. You do not get paid $14M to lose market share.

In the old days (1970's and '80s), savings and loans were called 3-6-3 businesses; pay depositors 3% interest, extend mortgages at 6%, hit the golf course by 3PM. WAMU, Fannie, and Freddie were all stable, well run companies that could have made a good return making/buying secure loans, and their CEOs could have been on the golf course by 3. But, that would not be worth $14M. So, they tried to earn it by competing for riskier business, and they failed.