 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|
 |
 |
 |
 |
 |
« Back to Lessig News
« Back to the Lessig Blog
Lessig Blog Archives: Cass Sunstein Guest Entries
Here’s my little Bob Dylan story: I took my 15 year old daughter to a Dylan concert a short while ago, and it was wonderful throughout, but the best part was the encore, when he sang Like A Rolling Stone. In the original version, it’s an angry, mean, sneering, contemptuous, and hateful song (great too, of course). Dylan himself described the song with the words “hate” and “revenge.” Thus the chorus:
How does it feel
To be on your own
With no direction home
Like a complete unknown
Like a rolling stone?In the concert the other night, the song had no hate, and there was no sense of revenge. As it was performed, it was happy, joyful, exuberant, inclusive — the chorus above all. The audience got it, almost immediately, and so the song turned out to be a celebration — more or less, a celebration of freedom and human equality. In short, the performance turned the song inside out.
Thanks much to Larry for inviting me to post, and thanks much to you all for the terrific comments and the helpful and generous emails. I’ve learned a lot.
There’s another form of information aggregation that we haven’t discussed: traditionalism. Conservatives who like traditions often build on the work of Edmund Burke, who emphasized that each of us has a small stock of wisdom, and that traditions embody the wisdom of the many. In this way, there’s a link between Burke on the one hand and Condorcet on the other — and a less direct link between Burke and Hayek. Here’s a passage from Burke’s essay on the French Revolution:
“The science of government being therefore so practical in itself, and intended for such practical purposes, a matter which requires experience, and even more experience than any person can gain in his whole life, however sagacious and observing he may be, it is with infinite caution than any man ought to venture upon pulling down an edifice which has answered in any tolerable degree, for ages the common purposes of society, or on building it up again, without having models and patterns of approved utility before his eyes… . We are afraid to put men to live and trade each on his own private stock of reason; because we suspect that this stock in each man is small, and that the individuals would do better to avail themselves of the general bank and capital of nations, and of ages. Many of our men of speculation, instead of exploding general prejudices, employ their sagacity to discover the latent wisdom which prevails in them.”
Burke’s claims have a lot of power; the problem is that some traditions may be a result of a cascade or group polarization (or worse).
In the meantime I’m wondering whether I’ll have the courage to say something about the Bob Dylan concert I recently attended.
Markets, Prediction Markets, and OSS
First things first: A grateful thanks for the incredibly helpful comments on my ignorant queries about OSS. (More comments on those queries are more than welcome.) The comments prompt the following thought. For OSS, there’s a lot of dispersed knowledge and also creativity, and that’s a big reason for the success. Something similar is true of Wikipedia (though as some people have suggested, the aggregation process is less reliable there). With ordinary product markets, there’s also a lot of dispersed knowledge, both about product performance and about individual tastes. If a watch doesn’t keep time very well, the market will respond, at least in the aggregate. If most people like the looks of a new car, the market will respond too.
Here we have a clue about why prediction markets sometimes fail, as in the cases of the Rehnquist resignation and the Roberts nomination. There just isn’t a lot of dispersed information out there about what the Chief Justice is likely to do or about the President’s particular choice. It follows that we shouldn’t think (as many do) that prediction markets will be able to foresee terrorist attacks. Investors, taken as a whole, probably lack the information that would make for judgments that improve on the conventional wisdom. If this is right, then prediction markets are unlikely to do well in foreseeing a range of international events.
I have a feeling that this is just a rough cut at the question — and some of the comments to an earlier post might do better — but maybe we’re making some progress.
On information aggregation, I haven’t yet said anything about open source software (though some comments refer to it). But to an outsider, OSS does exceptionally well in incorporating the ideas of numerous people. It’s analogous to the most optimistic understanding of Wikipedia (yes?). Here are the ridiculously ignorant outsiders’ queries, with apologies for the ignorance: Does OSS do as well as it seems in aggregating dispersed information (and dispersed creativity)? If so, why? If not, why not? It’s hard to have an adequate understanding of how information aggregation can go well, or badly, without having some answers. (I’ve read and learned a ton from Eric Raymond, Larry L., and several others, but the questions are not entirely answered.)
An empirical note on group polarization and outrage: A few years ago I was involved in a series of experiments (with Daniel Kahneman and David Schkade), trying to figure out why juries (and others) get outraged, and why they end up imposing high or low punitive damage awards.
Testing about 1000 jury-eligible people, we found that on a bounded scale (1-6 or 1-8, where 1 means not at all outrageous, or no punishment, and 6 or 8 means extremely outrageous, or severe punishment), Americans agree on the appropriate level of outrage and punishment. At least in personal injuries cases, a “5” is thought, by most people, to be a “5.” Whites agree with African-Americans, old people with young people, poor with rich, well-educated with not well-educated.
The dollar metric produces a lot more variety. People don’t agree on whether a “5” should be punished with a $1,000,000 award, or a %5,000,000 award, or a $50,000 award. (The study can be found in the Yale Law Journal circa 1998 and also in Cass R. Sunstein et al., Punitive Damages: How Juries Decide, circa 2002.)
But this study didn’t involve deliberating juries. With deliberation, we found some surprises. (a) If the median juror started low on the bounded scale, at say 2, the jury ended up at 1 — a leniency shift. (b) If the median juror started high on the bounded scale, say at 6, the jury ended up at 7 — a severity shift. (c) The jury’s dollar awards were much higher than the median juror’s dollar awards — a BIG severity shift for dollars. In 27% of cases, the jury’s award was as high as, or even higher than, that of the highest individual juror’s awards before jurors started to talk! (This study can be found in the Columbia Law Review circa 2000 and also in Punitive Damages: How Juries Decide.)
A general lesson is this: If people are outraged, and are surrounded by other people who are outraged too, they end up getting more outraged still. There’s a severity shift, often a big one. I speculate that the point bears on political polarization in general — and that it has implications for the blogosphere too.
Here’s a passage from the first entry on Judge Richard Posner’s blog (which he runs with Gary Becker): “Blogging is … a fresh and striking exemplification of Friedrich Hayek’s thesis that knowledge is widely distributed among people and that the challenge to society is to create mechanisms for pooling that knowledge. The powerful mechanism that was the focus of Hayek’s work, as of economists generally, is the price system (the market). The newest mechanism is the ‘blogosphere.’ There are 4 million blogs. The internet enables the instantaneous pooling (and hence correction, refinement, and amplification) of the ideas and opinions, facts and images, reportage and scholarship, generated by bloggers.”
I think that Posner is wrong to see the blogosphere as a Hayekian mechanism akin to the price system. The blogosphere does not produce prices. It doesn’t even produce a giant wiki, aggregating dispersed information. Instead it offers an amazingly diverse range of claims, perspectives, rants, insights, lies, facts, non-facts, sense, and nonsense. In his recent book, Blog, Hugh Hewitt celebrates the power of blogs to hold powerful actors, including the mass media, to account. He’s right to celebrate that power. And of course it’s true that the blogosphere makes it more likely that dispersed knowledge will get out. But the analogy to the price system is badly strained.
My little book, Republic.com, was written before blogs had anything like the prominence they now have. But it would be easy to apply the argument there to the blogosphere — to suggest that too much of the time, like-minded people are speaking (or at least listening) mostly to one another, ensuring that they end up thinking a more extreme version of what they thought before they started to talk. I believe that this view would be much too pessimistic (see the diverse comments on this blog, for example, or at The Volokh Conspiracy), but the question is really an empirical one on which we don’t yet have a lot of data.
The blogosphere is exposing people to lots of new topics, perspectives, and information. But Posner’s invocation of Hayek is a big stretch.
In his treatment of democracy, Jurgen Habermas emphasizes the importance and internal morality of deliberation. He thinks that under ideal conditions, “the forceless force of the better argument” will prevail. His account of deliberative democracy lies at the heart of his treatment of constitutional theory. Of course democracy can be seen as a mechanism for aggregating diverse views about both facts and values; and Habermas offers a distinctive account of democracy.
But here’s a serious problem. Even under ideal conditions, the better argument may not prevail. Careful experiments have shown that groups often amplify, and do not merely propagate, individual errors. Group polarization, as discussed previously, brings about extremism, even if extremism is unjustified. Information held by a few people, or just by one, often plays little or no role in a group’s ultimate decision. Informational cascades can lead deliberation in unfortunate directions. And because people care about their reputations, they may silence themselves even if they know something that is both important and true.
True, Habermas’ conditions include a principle of equality and a ban on strategic behavior. But even if these conditions are fulfilled, every one of these problems may infect deliberation.
Hayek, of course, stressed the qualities of the price system, which in his view serves as an excellent method of aggregating dispersed information. If we underline Hayek’s emphasis on economic incentives, we have the core of a challenge to Habermas: In deliberation, those incentives might well be absent, and hence people sometimes fail to say what they know.
We’ve also seen some reasons why the price system might fall victim to the same problems that beset deliberation. (Cf. the Clement cascade, affecting prediction markets.) And it wouldn’t be too hard to sketch a Habermasian critique of Hayek. But at least it can be said that group deliberation often goes badly wrong, and for identifiable reasons, even under ideal conditions.
Prediction markets, springing up at a rapid rate, provide another way of aggregating private information. Far more Hayekian than simply polling people, these markets have had some terrific success in predicting the outcomes of presidential elections (see the Iowa Electronic Markets) and also in predicting the Oscars and general box office results (see the Hollywood Stock Exchange). For Hayek’s reasons, it’s easy to see why prediction markets might work well. They aggregate private judgments, and dispersed bits of information, in a way that is backed by economic incentives. They have big advantages over the Condorcet method (poll and average) and what we might call the Habermasian method (deliberate and exchange reasons).
Here’s the but: The prediction markets apparently did very badly with the Supreme Court nomination. Roberts was way behind for a long time on tradesports.com, and during the Clement cascade, Clement started to dominate everyone else. (Also Rehnquist was strongly predicted to resign; investors got that one wrong too.) Can we develop a general account of when prediction markets will work well, and when they won’t? (And if so, should we eventually test that account in a prediction market?)
Odd: Some people have objected to my little post about Judge Clement, but apparently its substance was right. People were indeed participating in an informational cascade. Unfortunately, I ended up joining that cascade (tentatively). The confident view that the President had chosen Judge Clement, like the confident view that the Chief Justice was about to retire, was clearly a process in which many people were confidently relying on unreliable people, to the point where the number of (confident) people was misleadingly high. That’s a (bad) cascade. With respect to the confirmation hearings, I predict we’ll see at least one other bad cascade in the next two months. Let’s watch for it.
Opinions Are Aggregating Even Now
In an extremely short time, everyone seems to have concluded that Judge Edith Brown Clement will be nominated to the Supreme Court. This is clearly an informational cascade, in which almost everyone is responding to the statements of others, who are responding in turn to the statements of others, etc. (Compare the frenzy over the supposedly definite resignation of Chief Justice Rehnquist — also an informational cascade, including many people in high positions in the media and government.) A tentative hunch, though, is that everyone is right on this one.
If the President does choose Judge Clement, the most obvious point is that he’s chosen someone without much of a record. Other candidates include, for example, Michael McConnell, Michael Luttig, Frank Easterbrook, Mary Ann Glendon, and Edith Jones, all of whose views are much easier to find. The choice of someone without much of a paper trail — if that’s what we’re going to see — would be extremely interesting.
An initial thanks for the many excellent comments and emails, which I’m trying to absorb. We’ve been discussing several methods for aggregating views: markets a la Hayek, group deliberation, and wikis (with a brief mention of open source software). One emphasis has been on problems with group deliberation, because like-minded people usually end up thinking a more extreme version of what they thought before.
In his fun and illuminating book, The Wisdom of Crowds, James Surowiecki emphasizes another method of aggregating opinions: ask a lot of people and take the average answer. In many cases, this method seems to work magically well. If you put a bunch of jelly beans in a jar, and ask 200 people how many beans are in the jar, the average answer is likely to be eerily good. Often the average answer of a large group is right on the mark.
Surowiecki doesn’t explain why this happens, but the answer lies in the Condorcet Jury Theorem. If you have a group of people, and if each person is more than 50% likely to be right, the likelihood that the average answer will be right approaches 100% as the size of the group increases. (The math here is so simple that even we lawyers can almost understand it. For nonbinary choices with plurality voting, the math isn’t so simple, and this lawyer can’t even almost understand it, but there’s a result that explains why Condorcet’s basic insight applies there too.) Condorcet’s result has implications for many practices; it hasn’t been adequately exploited by people in business, law, and politics.
Here’s a problem, though. If group members are less than 50% likely to be right, the likelihood that the average will be right approaches ZERO as the size of the group increases. (I asked members of the faculty at the University of Chicago Law School to estimate the weight of the horse who won the Kentucky Derby, the number of lines in Antigone, and the number of Supreme Court invalidations of state and federal law. The group average did really well with the first question, pretty badly with the second, and horrendously with the third!) Condorcet was well aware of this point, and hence he emphasized that we can’t rely on the wisdom of group averages when most group members are likely to be biased or wrong.
Are group averages likely to be worse than what emerges from group deliberation? The answer is mixed. Sometimes deliberation does help to correct errors (especially when people are considering a “eureka” problem, where the answer is clearly right once identified). But sometimes deliberating groups do little better, and sometimes even worse, than predeliberation averages.
Are markets likely to do better than group averages? The simplest answer is yes, because participants have strong incentives to be right, and won’t participate unless they think they have something to gain.
In case anyone hasn’t heard by now: The President plans to announce his nominee to the Supreme Court tonight.
Hayek’s big claim about the price system was that it aggregates widely dispersed information and tastes. For this reason, he said that it was a “marvel.” We’ve been discussing other ways of aggregating information, and it might be useful to start with Wikipedia, if only because Jimbo Wales refers to Hayek in his comments.
Wikipedia does aggregate dispersed information — amazingly so. In a general way, it’s definitely a Hayekian process. But there are at least two differences between Wikipedia and the price system. First, Wikipedia doesn’t rest on economic incentives. People aren’t participating because they’re getting a commodity or money. There are no trades. Second, Wikipedia generally works by a “last in time” rule. The last editor. and hence a single person, can do a lot. But in the price system, the last purchaser usually can’t have a huge effect. (Even if you buy 10,000 copies of each of Larry’s books, you won’t affect the price.) The upshot is that Wikipedia is different from the price system; it aggregates dispersed information in a distinctive and less reliable way.
Two qualifications. 1) Wikipedia nonetheless works, at least for the most part. 2) The price system doesn’t always work, in the sense that bad information, sometimes spreading like wildfire, can produce inflated and deflated prices. (So Hayek was too optimistic, as behavioral economists have shown.)
What is this discussion missing?
As recently reported, Republican-appointed court of appeals judges get significantly more conservative, and Democratic-appointed court of appeals judges get significantly more liberal, when they are sitting with judges appointed by a president of the same political party. But there are two areas where this does NOT happen — where Republican appointees differ from Democratic appointees, but where judges’ voting patterns are unaffected by the composition of the panel. Any guesses?
Affirmative action? No. Environmental protection? No. Gay rights? No. Campaign finance or commercial advertising or obscenity? No. Race and sex discrimination? No. The two areas are: Abortion and capital punishment. In those areas, Republican appointees differ a lot from Democratic appointees, but the rest of the panel doesn’t much matter.
Extremism and Information Aggregation
Here at the University of Chicago, we have something called the Chicago Judges Project, by which we tabulate and analyze thousands of votes of judges on federal courts of appeals. One of our key findings thus far is this: In many controversial areas (eg, affirmative action, campaign finance, sex discrimination, disability discrimination, environmental regulation, and more), Republican appointees show especially conservative voting patterns when they’re sitting on 3-judge panels that consist only of Republican appointees. So too for Democratic appointees: They’re far more liberal, in their voting patterns, when sitting with two fellow Democratic appointees, than when sitting on a panel with at least one Republican appointee. In other words, Republican appointees look more conservative when they sit only with fellow Republican appointees, and Democratic appointees look more liberal when they sit only with fellow Democratic appointees.
This is real-world evidence, we think, of group polarization: the process by which like-minded people, engaged in deliberation with one another, typically end up thinking a more extreme version of what they thought before they started to talk. (So, for example, French people who distrust the US distrust the US even more after talking with one another.) Group polarization reflects a form of information aggregation, or at least opinion aggregation, that sometimes leads in unfortunate directions. It’s a big contrast to the price system, Wikipedia, and open source software.
Here’s a related phenomenon: hidden profiles. In a deliberating group, shared information (information held by many or most) usually has a much bigger effect than unshared information (information held by a few or just one). The result is that groups have “hidden profiles,” in the form of information that doesn’t get out, or that has less impact than it deserves. Some big mistakes by private and public organizations (including faculties!) are a result of hidden profiles.
Here again, the price system, Wikipedia, and open source software do a lot better. (Of course there are important differences among the three, as the terrific comments so far suggest, and on which more soon.) All this raises many puzzles. Here’s one: Are there ways to incorporate what we’ve been learning, from those three, so as to make group deliberation go better?
(originally posted 7/17/05)
On Aggregating Information: Hayek, Blogging, and Beyond
This is from Cass Sunstein; I’m most grateful to Larry for inviting me to post on his blog for a bit. His kind invitation is a result of a naive and ignorant inquiry I sent him in the recent past, about information aggregation and its possible limits.
Background: A few years ago, a book of mine, Republic.com, emphasized the risks associated with echo chambers and self-insulation. I’m doing a new book, still inchoate, that continues to explore those risks, but that also stresses the excitingly general possibility that the Internet can allow widely dispersed “bits” of human information to be properly aggregated — as, for example, through open source software, wikis, prediction markets, and even blogging.
All this is pretty abstract, so let me try to focus it. One of the greatest arguments of the twentieth century is Hayek’s about the price system. In particular, Hayek claimed that any “price” is capturing the information and tastes of many people, in a way that will outperform the judgments of even the best experts. Hence prices do a lot better than any central planners. Here’s a puzzle: How close is the analogy between the price system on the one hand and wikis, open source software, and even the blogosphere on the other? Where does the analogy break down? When, in particular, will wikis and the blogosphere fail as mechanisms for aggregating dispersed information? I’ll venture some thoughts before long, but for the moment I’d just like to pose the question. I know that there’s a lot of information out there about all this; any help would be appreciated.
(originally posted 7/16/05)
« Back to Lessig News
« Back to the Lessig Blog
This work is licensed under a
Creative Commons License.