Trial Lawyers and the Winner’s Curse

Chances are, you don’t enjoy movies as much as you think you will. That’s because movie tickets present us consumers with a tricky problem of incomplete knowledge. We want to buy tickets to movies we’ll enjoy, but the only sure way to know how much we’ll enjoy a movie is to watch it, and we can’t do that unless we’ve already bought a ticket.

So we try to make an educated guess. We watch trailers, read reviews, and talk to friends who’ve already seen it, hoping to get enough information to make a good estimate of how much we’d enjoy it if we bought a ticket. Then we compare our estimate of our enjoyment to the cost of the movie — including the price of the ticket, the cost of snacks, and the value of our time — and decide whether or not to give the movie a chance.

We’ll never be able to estimate exactly how much we’ll enjoy a movie we haven’t seen — we just don’t have enough information — so there will always be an error factor. That is, our estimate of our enjoyment is conceptually equal to our actual enjoyment plus a random error factor (which can be positive or negative).

If the error factor is random, you’d think that would mean we would find ourselves enjoying movies more than we expected or less than we expected with equal probability. But it turns out that that only happens if we ignore our estimates and just go see every movie we can. We get a different result if we only go see movies for which we estimate a high enjoyment value, because the movies we overestimate will tend to have a higher estimated enjoyment value than the movies we underestimate.

For example, if you estimate your enjoyment of 100 movies and split them into two groups by rank — the highest 50 and the lowest 50 — the higher group is likely to have more overestimations, and the lower group is likely to have more underestimations. So if you go and see all the movies in the top half, you will see more movies which you overestimated. In other words, you will see movies which you enjoy less than your estimate predicted you would. As I said at the top of this post, you don’t enjoy movies as much as you think you will.

This is a variation on a phenomenon called the winner’s curse. The name comes from the observation that in an auction, the person who bids highest is likely to be the person who overestimates the value of the auctioned item the most.

I was reminded of all this a few days ago by a post I first saw at Gideon’s a public defender blog about a new American Psychological Association paper which shows that lawyers overestimate their chances at trial which has made the rounds of the legal blogosphere. Here’s a bit from the abstract of the paper:

The study investigated the realism in predictions by a sample of attorneys (n=481) across the United States who specified a minimum goal to achieve in a case set for trial. They estimated their chances of meeting this goal by providing a confidence estimate. After the cases were resolved, case outcomes were compared with the predictions. Overall, lawyers were overconfident in their predictions

This makes sense as another example of the winner’s curse. Lawyers and their clients are more likely to take a case to trial when their estimate of the outcome is high, but because overestimations result in higher estimates, those are precisely the cases which they are most likely to be overestimating. Whereas the cases they underestimate are more likely to look like losers, so they’re more likely to settle before trial.

In other words, the cases lawyers take to trial are those in which they have the most confidence, but those are also likely to be the cases in which they are most overconfident.

Unfortunately for my nice little story, the winner’s curse is also something of a paradox. The problem is that lawyers presumably know about the winner’s curse — maybe not by that name, but the concept isn’t exactly a dark secret. More importantly, the lawyers will have experienced the winner’s curse. They will see the outcomes of the cases the take to trial, and they should be able to use that knowledge to adjust their estimates to eliminate the curse. And yet this recent study shows that doesn’t happen.

It’s not just lawyers. Economists have studied the winner’s curse in artificial simulations and in the real world for years, and there is some evidence that people consistently fail to compensate for it. It’s a bit of a mystery. The most obvious non-economic explanation is that people are a bit stupid, but that’s unsatisfying because it accomplishes too much: Once you assume that people are stupid, you can use that to explain any behavior you don’t understand.

The APA paper doesn’t mention the winner’s curse, but a section called Metacognitive Realism does discuss several possible causes for overconfidence: It could be a side effect of zealous representation, a result of the need to display confidence to attract clients, distortion due to the perception of control, or simple wishful thinking.

I don’t know enough about psychology to begin to guess whether any of those theories are correct, but I know enough about economics to notice something that’s missing. Lawyers are supposed to represent the interests of the client, but they wouldn’t be human if they didn’t also consider their own interests. Depending on the fee arrangements, many lawyers get paid more when a case goes to trial, which gives them an incentive to go to trial when it wouldn’t be best for the client.

So if lawyers are following the path of higher earnings, we’d expect the rate of overestimation to be higher for lawyers who make more by going to trial.

The APA study doesn’t report fee arrangements directly, but we can try to make a few guesses from the data in the paper’s Table 2, which I used to prepare this data:

Success rate	Criminal prosecution	Criminal defense	Civil plaintiff	Civil defense
Estimated	72.8	50.1	65.1	65.1
Actual	67.1	43.7	51.6	62.6
% Overestimated	8	15	26	4

Some criminal defense attorneys have a fee structure that brings in more money when they take a case to trial, but as far as I know all prosecutors work for a fixed salary, so the overestimation rate should be higher for criminal defense attorneys than for prosecutors, and indeed the data seems to confirm this, with defense attorneys overestimating their chances of achieving their goals by 15% to the prosecutors’ 8%.

For the civil bar, I’d expect to see a similar result, since plaintiffs’ attorneys often work on a contingency basis (which aligns their interests with their clients) but civil defense attorneys usually charge by the hour and therefore make more by going to trial. However, the data goes the other way, with plaintiff’s lawyers wildly overestimating their chances at 26%, while civil defense lawyers only miss by 4%.

Whatever’s going on, I guess it’s more complicated than my simple economic theory. Or maybe I’m misunderstand the data. Or else I’m misunderstanding how lawyers earn money from legal work. In any case, it’s getting late and I’m going to bed.

Heck, maybe plaintiff’s lawyers and criminal defense lawyers are all just romantic dreamers…