July 20, 2005  ·  Cass Sunstein

Prediction markets, springing up at a rapid rate, provide another way of aggregating private information. Far more Hayekian than simply polling people, these markets have had some terrific success in predicting the outcomes of presidential elections (see the Iowa Electronic Markets) and also in predicting the Oscars and general box office results (see the Hollywood Stock Exchange). For Hayek’s reasons, it’s easy to see why prediction markets might work well. They aggregate private judgments, and dispersed bits of information, in a way that is backed by economic incentives. They have big advantages over the Condorcet method (poll and average) and what we might call the Habermasian method (deliberate and exchange reasons).

Here’s the but: The prediction markets apparently did very badly with the Supreme Court nomination. Roberts was way behind for a long time on tradesports.com, and during the Clement cascade, Clement started to dominate everyone else. (Also Rehnquist was strongly predicted to resign; investors got that one wrong too.) Can we develop a general account of when prediction markets will work well, and when they won’t? (And if so, should we eventually test that account in a prediction market?)

  • Will “scifantasy” Frank

    I heard on the radio, but cannot cite links, that a group of scientists is trying to set up a prediction market for hurricanes.

    I think the general rule might be similar to Asimov’s psychohistory…The more people that are involved in making a decision, the more that decision is predictable. The Oscars, for example, are voted on by a lot of people, and box office totals are an aggregate of individual people’s decisions. Rehnquist’s decision was made by one person; and I’m not going to comment on how many people may or may not have factored into Roberts’ nomination.

    What that means for the hurricanes, I’ll never know.

  • John

    An interesting paradox:

    “A person is smart, but people are stupid”
    reference: riots/mobs

    yet,

    “The Wisdom of Crowds” by James Surowiecki

    Where and when does each of these paradigms come true and why? Are we moving towards a sophist attitude (which I think many here will disagree with)?

  • Gregorus

    It’s probably an Asimov “psycho-history” effect.

    The President is free to choose whoever he likes, and is free to look at who the markets think he is going to pick.

    So, a strategy for making the markets wrong (perhaps to get more ‘buzz’), when you are the sole decision-maker, is to narrow down your selection to a few choices, all optimum or close to optimum, until the last second, and then at that last second, choose someone who is not doing well in the markets.

    I wonder: if the people making the decision aren’t allowed or able to see the market prediction, is the market more likely to be accurate?

  • http://sethf.com/ Seth Finkelstein

    This is very relevant (none of the below is mine, it’s a quote of the post):

    http://crookedtimber.org/2005/07/06/even-breaks/

    “Bainbridge’s argument here reminds me a little of this old post of Dan’s, which argues among other things that Hayek’s notion of the market as a knowledge-creating entity sits rather uneasily with more standard economic arguments such as efficient-market theory. But Bainbridge’s argument is somewhat different and points to a different tradeoff. If you want to use markets to make the best predictions possible on the basis of available information, you’ll want to allow insider trading, which is, by definition, trading by those with valuable hidden information. But this means that you’re likely to lose liquidity by driving out ordinary punters who don’t want to be fleeced by those in the know. And without ordinary punters, insider traders have no incentive to transact (the only reason that they would want to transact is to fleece suckers who know less than they do). The only way in which this contradiction can really be resolved is if there’s a supply of suckers out there, who are willing to make bets against people who are better informed than they are. As Bainbridge points out, this is a condition that can be satisfied. But by and large, it’s only satisfied when people have extraneous reasons to make a bet (they enjoy a flutter). Bets that aren’t “fun” or otherwise attractive in some way aren’t likely to attract suckers. Thus, they’ll have low liquidity, and not be very useful as a source of information (this seems to be borne out by the empirics; as the authors of this paper note, “as the wonkishness of the contract rises, however, volume and liquidity falls rapidly.”) Thus, even apart from the objections that Dan and John Q. have raised in past posts, prediction markets aren’t likely to be very useful for a very wide variety of important policy issues.”

  • http://law.shu.edu/faculty/fulltime_faculty/pasquafa/pasquale.html Frank

    I’m interested in how your interest in aggregation processes jibes with your earlier work questioning the unthinking acceptance of preferences as they stand.

    In The Partial Constitution, you warned that existing consumer preferences (particularly for entertainment) may be misshapen because, say, media conglomerates dumb them down.

    Yet in your more recent work, including your inquiries here, you appear to assume the legitimacy of preferences as given and just look for better ways to aggregate them.

    I think it would be good to balance the influence of the later, democatic-theory work of Habermas with his earlier, critical theoretical inquiries. In other words, why not emphasize that the legitimacy of preference-aggregation varies with the certainty of the result we want to obtain? I don’t mind if markets dictate what styles of tables are on sale at the local furniture store; I do care if they tend to be creating vile or mindless entertainment as the dominant option in the media.

  • http://law.shu.edu/faculty/fulltime_faculty/pasquafa/pasquale.html Frank

    I think these types of prediction markets are very dangerous because they can become self-fulfilling prophecies. I recall that Bush was way ahead of Kerry in Tradesports in September and August. Could this have led to more donors (especially those seeking help from the government) tipping to Bush over Kerry? And thereby funding his victory?

    Roger Shattuck’s Forbidden Knowledge suggests that this is a type of information that is best kept private.

    In the final analysis, if the preferences to be aggregated are entirely unjustified, aren’t they merely variations on Keynes’ old characterization of the stock market: i.e., it’s like a contest of “who’s the most beautiful face” out of a hundred, where the winner is the person who picks the face that the most other people also pick?

    The wikipedia is different because people’s writing on a topic has to be justified by, say, citations, a compelling writing style, or some modicum of accuracy, lest a fan of the page knock them off.

  • geoff

    I think the contention that wikis are more likely accurate than markets is absurd. It may be true, in a best of all possible worlds, that the only ones who bump previous writers on a wiki entry are those with better citations, a more compelling writing style and/or more accuracy, but ours is not that world. While there is a corrective available to abuse (subsequent authors), I hardly think it is only the “fans of the page” who knock prior writers off — it is also, of course, those with ulterior motives.

    As to the prediction markets’ success or failure: It is not enough to note that the market was imperfect. The relevant question is: did it do better than other sources of information. On this score, I think the answer must be yes: The market did predict the winner before, say, the AP announced it, and the Roberts trend began several hours — not seconds — before the nomination was made public. Did any other source do consistently better?

  • http://www.chrisfmasse.com/ Chris. F. Masse

    Cass Sunstein asked:

    “Can we develop a general account of when prediction markets will work well, and when they won’t?”

    Since I follow prediction markets on a daily basis (my website is a reference in the field), I have developed this viewpoint:

    - Prediction markets work when there’s openess. Typical example is the business of trying to predict the outcomes of elections. It’s public, and information (polls, experts’ pronouncements) is everywhere in the media for the traders to aggregate.

    - Prediction markets usually don’t work when the decision-making generating the outcome of an event is made in secrecy —either by a lone decision maker who won’t tell or by a kind of sequestrated conclave that is told not to talk to the outside. Examples of recent prediction markets failures:

    * SCOTUS nomination futures markets (note that the SCOTUS confirmation futures markets will certainly work finely);
    * 2012 Olympic city futures markets (the markets saw Paris as the winner);
    * papacy futures markets (the Pope would come from Europe, said the markets, but they failed to divine Ratzinger and Germany as country of origin);
    * Michael Jackson futures markets (like the commentariat, the markets had him behind bars);
    * Purcell resignation futures market (the market said he would not resign);
    * George Tenet resignation futures market (idem).

    Prediction markets floated by IEM are chosen carefully. Elections, interest rate, a bunch of stocks, the flu, the hurricanes, etc.

    However, the British and Irish event-driven futures exchanges will float just about anything that will bring in traders. I’m not blaming them. I’m just underlining that they don’t pick carefully with in mind the goal of maintaining a good track record when it comes to predictions.

    You’ll hear soon about my proposal for a think tank regarding prediction markets.

    Best regards,

    Chris. F. Masse

  • http://lawquad.squarespace.com tom f

    It’s terribly intriguing that we’re citing Asimov’s psycho-history theory. I remember reading The Foundation as a young teenager and being entranced by the possibility that one could predict events based on the cumulative actions of billions of choices.

    Whether that’s a verifiable theory or not, I think it’s important to distinguish between the two types of events Professor Sunstein hopes to be able to predict. One is (for lack of better terminology) a known event — the number of lines in Antigone, for instance. The answer to a known event is static — it will never change. The second is an unknown event — whether Karl Rove will resign, or whether Judge Roberts will be confirmed, etc.

    If the Condorcet Jury Theorem applies well to known events, it’s unclear (at least to me) why it would apply to unknown events — or, if it does apply, it’s unclear that it would apply for the same reasons. For known events, I would surmise that the theorem results in good approximations because random errors in large data sets tend to cancel themselves out. Does the same apply to unknown events? I’m not sure.

    But here’s a (possibly) interesting idea. Professor Sunstein believes we’re likely to see another cascade in the next few months. It wouldn’t be too hard to write a small software program that would look for keywords (like “Judge”, “Roberts”, etc.) as they appear on blog postings. The program would follow hyperlinks (and trackbacks) from blog posting to blog posting, creating a map of all postings relevant to the cascade and ordering them by whom they linked to and who linked to them, and ending when coming to a blog posting with no relevant link. That information could be used to identify the source(s) of the cascade, which, i suspect, is (are) probably more important in identifying the overall prediction of the blogosphere than any aggregation of posted beliefs.

  • http://www.technoutopia.blogspot.com Will Curtis

    The value of a prediction market is a function of how much information is distributed (in the aggregate) among the participants of said market. I could set up a prediction market to decide what I’ve got in my pocket, but how much information does the market have about this to aggregate?
    The Bush team’s thinking process is unobservable to the market — only a few key players are privy to wha’t going on in their mind. Thus, I suspect that there’s little real information present in the market that could be aggregate to inform a good estimate of Bush’s choice. What we saw was the market trying to do a pattern recognition thing, where it looked at past bush administration policies and decisions … but the market didn’t have the crucial information gained by Bush through his interviews of the candidates etc.

  • Paul Gowder

    Pah on prediction markets working better than discourse (a.k.a. Habermasian deliberate-and-exchange-reasons). (Yes, I’m leaping to the defense of my beloved Habermas.)

    Is there any evidence for this comparative statement? Has anyone actually studied the predictive power of markets versus the predictive power of discussions?

    It seems to me that a predictive market must overcome the natural incentive to fool the other betters. In concrete terms: if I predict that a company’s share price will go down because I believe it is overvalued, my incentive is to sell the stock. And sure, that sell order will add information to the market and will become available to others.

    But suppose I decide instead to float a rumor that I’m planning a takeover of that company? And suppose I leave secret instructions with my broker to sell as soon as the stock price inflates beyond a certain point, said inflation being caused by the expected bidding war that would accompany a takeover attempt?

    Sure, in the long term, my information will go into the market: everyone will see that I was lying about the takeover attempt when I dump my stock, and the price will veer back down. But in the short term — which is the relevant time period for making our predictions useful, the market has been a very poor predictor indeed.

    How do we know that Tom Goldstein, for example, didn’t know about the Roberts thing days in advance and push other potential nominees on his blog so that he could clean up on the internet betting sites?

    If we strip the economic incentive away and rely on discourse, we might lose some participation — those who would have had an incentive to “predict” by the possibilit of profiting don’t do so — but we gain some honesty.

    I incorporate Seth’s comments by reference here. And also Gregorus’s. What are we to do about the participation of decisionmakers in the prediction market? In a discourse-predictive world, we can incorpoate the decisionmakers or exclude them, without fear, because nobody stands to gain or lose by the decision. On the other hand, in a prediction market, we have the insider trader problem. If my company is the object of predictions, I can strategically fire a bunch of employees to manipulate the stock price.

    I still think we need to do some empirical testing of discourse prediction against market prediction…

  • ACS

    With respect to forseeing an event by analysis of a range of statistical data I think there is some success even in respect of things based on a purely guess work basis. However, in the complex field of law the probability is not dependant on the sample but rather the question posed. Hayek theory should be placed within the boundaries of ‘How far do you have to go to find the legal answer?’.

    For example, if someone is caught on video tape stealing a loaf of bread, admits to the theft and doesnt give any excuse there is a good chance that the legal community will reach agreement and arrive at the outcome.

    However, if there was a more complex problem such as whether Peer to Peer (sorry to go back here but it is a good example given the nature of this blog usually) software infringes copyright there is a greater chance of new or alternative reasons and rationales being provided that could not be foreseen or could have been foreseen in effect but not in rational details.

    There is also issues such as abortion or flag burning which muddy the waters further.

    All of a sudden the precise mathematical nature of statistics clash with the (oft un) logical world of human group interaction and dispute resolution. Despite the generalisation of plaintiff wins or defendant wins the law is not a binary equation. The variables at law skyrocket and the likelyhood of a correct predicition plumet. A single word out of place may destroy an entire prediciton.

    I have no doubt only an infrinite (and impossible) field from which to select a prediciton could render a perfectly accurate result in the reasons for a decision. However, the probability of such a result within a finite field with respect to a complex and unseen legal problem is impossible to define.

    In any case, you dont know whether your prediciton will be right until afterwards anyway. (I know it is a dichotomy of mammoth proportion – but I believe in time travel – just like Doctor Who).

    However, this has always been the field of jurists to speculate on the next big issue and I do agree that publication of a wide range of jurists is an important development however I doubt that it will increase the value of success of prediction for this reason:- Some lawyers are better than others – I dont know whether it is better reading, a better mind or the fact that everyone else was getting laid during college but some are just sharper quicker and better.

    I wouldnt put my theories up against lessig, dworkin or hayek unless it was on a thoroughly defined ground.

    On that basis the concept of prescience of the law is not the same as counting jelly beans. Some guessers already know about how many jelly beans there are and some of us are way off. This is the market disparity which flaws “Jurisprediciton”.

    Sorry you had to read all that to see my new word

  • http://www.kmcluster.com/sfo/PM John T. Maloney

    The SF/Silicon Valley Summit on Prediction Markets is Friday, October, 21, 2005, 8:00am – 5:00pm in downtown SF. Registration is open.

    http://www.kmcluster.com/sfo/PM