July 19, 2005 · Cass Sunstein
An initial thanks for the many excellent comments and emails, which I’m trying to absorb. We’ve been discussing several methods for aggregating views: markets a la Hayek, group deliberation, and wikis (with a brief mention of open source software). One emphasis has been on problems with group deliberation, because like-minded people usually end up thinking a more extreme version of what they thought before.
In his fun and illuminating book, The Wisdom of Crowds, James Surowiecki emphasizes another method of aggregating opinions: ask a lot of people and take the average answer. In many cases, this method seems to work magically well. If you put a bunch of jelly beans in a jar, and ask 200 people how many beans are in the jar, the average answer is likely to be eerily good. Often the average answer of a large group is right on the mark.
Surowiecki doesn’t explain why this happens, but the answer lies in the Condorcet Jury Theorem. If you have a group of people, and if each person is more than 50% likely to be right, the likelihood that the average answer will be right approaches 100% as the size of the group increases. (The math here is so simple that even we lawyers can almost understand it. For nonbinary choices with plurality voting, the math isn’t so simple, and this lawyer can’t even almost understand it, but there’s a result that explains why Condorcet’s basic insight applies there too.) Condorcet’s result has implications for many practices; it hasn’t been adequately exploited by people in business, law, and politics.
Here’s a problem, though. If group members are less than 50% likely to be right, the likelihood that the average will be right approaches ZERO as the size of the group increases. (I asked members of the faculty at the University of Chicago Law School to estimate the weight of the horse who won the Kentucky Derby, the number of lines in Antigone, and the number of Supreme Court invalidations of state and federal law. The group average did really well with the first question, pretty badly with the second, and horrendously with the third!) Condorcet was well aware of this point, and hence he emphasized that we can’t rely on the wisdom of group averages when most group members are likely to be biased or wrong.
Are group averages likely to be worse than what emerges from group deliberation? The answer is mixed. Sometimes deliberation does help to correct errors (especially when people are considering a “eureka” problem, where the answer is clearly right once identified). But sometimes deliberating groups do little better, and sometimes even worse, than predeliberation averages.
Are markets likely to do better than group averages? The simplest answer is yes, because participants have strong incentives to be right, and won’t participate unless they think they have something to gain.