Tradesports predictions for 2006 elections: Difference between revisions

From ChanceWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(38 intermediate revisions by the same user not shown)
Line 5: Line 5:
Robert Forsythe<br>
Robert Forsythe<br>


We have discussed the use of betting markets to predict the outcomes of elections several times in  Chance News.  See for example [http://www.dartmouth.edu/~chance/chance_news/recent_news/chance_news_12.02.html#item5 Chance News 12.02]
We have discussed in Chance News several times  the use of betting markets such as [http://www.tradesports.com/ Tradesports], the [http://www.biz.uiowa.edu/iem/ Iowa Political Markets] and other such markets to predict the outcomes of elections, sports events, Oscar winners etc.  See for example [http://www.dartmouth.edu/~chance/chance_news/recent_news/chance_news_12.02.html#item5 Chance News 12.02]


[http://people.cs.uchicago.edu/~fortnow/ Lance Fornow] Computer Scientist at the University of Chicago, [http://dpennock.com/ David Pennock] and  
[http://people.cs.uchicago.edu/~fortnow/ Lance Fornow] Computer Scientist at the University of Chicago, [http://dpennock.com/ David Pennock] and  
[http://research.yahoo.com/ Chen Yiling]  Research Scientists at Yahoo have carried out research to evaluate the ability of markets such as [http://www.tradesports.com/ Tradesports], the [http://www.biz.uiowa.edu/iem/ Iowa Political Markets] and other such markets to predict the outcomes of elections, sports events, Oscar winners etc.  
[http://research.yahoo.com/ Chen Yiling]  Research Scientists at Yahoo have been interested in how to evaluate the ability of betting markets to make predictions.


In their discussion of the predictions relating to the 2006 Senate races, they provided the following map showing results as of about 9 AM CST election day.  
In his [http://blog.oddhead.com/Oddhead Blog] David Pennock has a nice article [http://blog.oddhead.com/category/prediction-markets/ Evaluating probabilistic predictions].  He also recommends an article in the Online New Republic [http://www.tnr.com/doc.mhtml?i=w061120&s=cowgillsunstein112106 "Can predictions markets forecast elections? Good Bet"], Cass R. Sunstein & Bo Cowgill, 11.21.06.
 
In their discussion of the predictions relating to the 2006 Senate races, Fornow, Pennock and Yiling provided the following map showing results as of about 9 AM CST election day.  
<center>http://weblog.fortnow.com/media/election-day-2006-map.JPG </center>
<center>http://weblog.fortnow.com/media/election-day-2006-map.JPG </center>
In his analysis, Fornow wrote:
In his analysis, Fornow wrote:
<blockquote>
<blockquote>
Every state colored blue was won by a democrat and every state colored
Every state colored blue was won by a Democrat and every state colored
red went to a republican. But also note the 69% given to GOP
red went to a Republican. But also note the 69% given to GOP
(Republican) Senate control although this election will give control
(Republican) Senate control although this election will give control
to the democrats. No outcome would have made
to the Democrats. No outcome would have made
all the states and senate control agree with the 9 AM map.
all the states and Senate control agree with the 9 AM map.
<br><br>
<br><br>
Were the markets inconsistent? No, because the markets predict not
Were the markets inconsistent? No, because the markets predict not
absolutely but probabilistically. For example, the markets gave a
absolutely but probabilistically. For example, the markets gave a
probability of winning 60% for each of Virginia and Missouri and the
probability of winning 60% for each of Virginia and Missouri, and the
democrats needed both to take the senate. If these races were
Democrats needed both to take the Senate. If these races were
independent events, the probability that the democrats take both is
independent events, the probability that the Democrats take both is
36% or a 64% chance of GOP senate control assuming no other surprises.
36% or a 64% chance of GOP Senate control assuming no other surprises.
<br><br>
<br><br>
Of course the races were not independent events and there are other
Of course the races were not independent events and there are other
states involved making it more difficult to compare the
states involved making it more difficult to compare the
probabilities of the individual races with that of senate control.
probabilities of the individual races with that of Senate control.
<br><br>
<br><br>
So how did the markets do as predictors? Quite well as the outcome
So how did the markets do as predictors? Quite well as the outcome
seems quite reasonable given the markets. Other outcomes would have also
seems quite reasonable given the markets. Other outcomes would have also
been reasonable such as the Democrats losing Virginia and the senate
been reasonable such as the Democrats losing Virginia and the Senate
remaining in republican hands, a possibility that came very close to
remaining in Republican hands, a possibility that came very close to
happening.  
happening.  
</blockquote>
</blockquote>
While it seems likely that the outcomes of individual state elections are not independent it is interesting to check this by looking at the data.


To do this we need to  describe how the Tradesports betting works. We illustrate how this in terms of betting that Hillery Clinton will be nominated as the democratic candidate for the 2008 presidential election.
We thought it would be interesting look in more detail at how Tradesports did in predicting the outcomes of the 2006 election. To do this we need to  describe how the Tradesports betting works. As explained by WikiPedia:


Betting at Tradesports is made through contracts. You can buy or sell contracts. In this example the a contract has a price between 0 and 100.  A contract with price 0 is worthless. A contract with price 100 is worth $10.
<blockquote>Tradesports markets trades in the binary 0-100 format. If the event specified in a given contract occurs, the contract settles at 100 points, or $10 per contract or share; otherwise, the contract settles at 0. Thus, the current price of the contract can be imputed as the probability that the specified event will occur.</blockquote>
 
We will illustrate this in terms of a current event: will Hillary Clinton be nominated as the Democratic candidate for the 2008 presidential election?
   
   
Until the nomination is determined you can offer to buy or cell contracts at specific prices. If your offer to buy matches another persons offer to sell you can bid for a contract.
Tradesports makes the initial price for a contract and after this the price is determined by offers to buy and sell the contracts.  Thus Tradesports is simply a match maker, though there are some fees attached to this which are explained [http://www.tradesports.com/ here]
Tradesports makes the initial price for a contract and after this it is determined by offers to buy and sell them.  Thus Tradesports is simply a match maker though there are some fees attached to this which are explained [http://www.tradesports.com/ here]


As of December 23 Hillery contracts were given in the following table.  <center>http://www.dartmouth.edu/~chance/wikivideos/clintonbidask.jpg</center>.
On December 23 Senator Clinton contracts were given in the following table.  <center>http://www.dartmouth.edu/~chance/wikivideos/clintonbidask.jpg</center>.


At  this is written the price of a contract is 50.3. This is interpreted to mean that Hillery has a 53% chance of being nomiated.
A price of 53.0 is interpreted to mean that Senator Clinton has a 53% chance of being nominated. It would cost you $5.30 for this contract. Then you should win $10 with probability .53 and 0 with probability .47, giving you an expected winning of $5.30 and making it a fair bet.
   
   
The closing price of a contract varies through time as illustrated by the following graphic:
For each day there is a closing price for a contract which varies through time as illustrated by the following graphic:


<center>http://www.dartmouth.edu/~chance/wikivideos/clintongraph.jpg</center>
<center>http://www.dartmouth.edu/~chance/wikivideos/clintongraph.jpg</center>


Here are similar graphs for other leading cadidates for democratic and republican nominations as of Dec. 5, 2006 that appeared in a New York Times article," '08 Candidates in a Virtual Market", Dec. 5, 2006.
Here are similar graphs for other leading candidates for Democratic and Republican nominations as appeared in a New York Times article," '08 Candidates in a Virtual Market", Dec. 5, 2006.
 
<center>http://graphics8.nytimes.com/images/2006/12/05/us/politics/1205-nat-webACTION.gif </center>
 
Now we return to the 2006 election. There were a number of contracts you could bid for.  We will concentrate on the contracts for the Democrats winning individual states.
 
We should first see if the contract prices really act like probabilities.  Here is an elementary test.  When you buy a contract for the outcome of a particular state you have three choices for the contracts: The Democratic candidate will win, the Republican candidate will win, and some other candidate will win.  if the prices are really probabilities the prices for these three choices should add up to about 1.  Of course these prices are themselves random variables so we cannot expect them to add up to exactly 100.
 
Here are the sums for some of the states that were considered important for the Democrats to win to get control of the  Senate.
 
<center>http://www.dartmouth.edu/~chance/forwiki/tradesportstable.jpg </center>
 
The sums are close enough to 100 that we can say that the Tradesportts prices pass this test.
 
Note that all the prices except Tennessee are greater than 1/2 so Tradesports would predict that, in all these states except Tennessee, the Democratic candidate would win the Senate race which is what actually happened.
 
Next we would like to see if the prices for contracts that the Democrats will win two different states are independent.  One way to look at this is to compare plots of the prices of two different states.  We do this for three pairs of states chosen from the six states widely publicized as states the Democrats would have to win to win control of the Senate.
 
<center>http://www.dartmouth.edu/~chance/forwiki/VirginiaMissouri.jpg</center>
 
<center>Virginia prices (bottom) and Missouri prices (top)</center>
 
<center>http://www.dartmouth.edu/~chance/forwiki/RhodeIslandOhio.jpg</center>
 
<center>Rhode Island prices (bottom) and Ohio prices (top)</center>
 
<center>http://www.dartmouth.edu/~chance/forwiki/MontannaPennsylvania.jpg</center>
 
<center>Montana prices (bottom) and Pennsylvania prices (top)</center>
 
Well, they do seem to follow each other and so we can say that they are not independent but this may not be completely convincing.
 
We look next at how the Tradesports might predict who would control the senate. There were 33 Senate races.  The Democrats had to win 24 or more of these races to win control the Senate.  It was pretty clear that the Independent candidates Joe Lieberman in  Connecticut and Bernie Sanders in Vermont would win their races and vote as Democrats, so the Democrats would have to win 22 or more of the remaining 31 races.
 
If the prices of individual states are not independent we cannot estimate the probability that the Democrats win 22 or more races. But, we can estimate the expected number of states the Democrats will win by simply adding the probabilities for states with contract price greater than 1/2.
Doing this, using the closing price for each day from January 15 to November 6, we obtain the following graph:
 
<center>http://www.dartmouth.edu/~chance/forwiki/demleads.jpg</center>
 
This suggests that only in the last days of the race did the Democrats appear to have a reasonable chance of winning the required 22 states.


http://graphics8.nytimes.com/images/2006/12/05/us/politics/1205-nat-webACTION.gif
Tradesports did not have a contract that the Democrats would win control of the Senate, but they did have a contract that the Republicans would, so we can use this to determine the probability that the Democrats win control of the Senate.  Here is Tradesports graph of the closing prices for the contract that the Republicans would win control of the Senate:


Now we return to the 2006 election and the question raised earlier: Assuming that the price of a contract the democrats will win a particuar state estimates the probability that they win this state, are the events that the they win two states independent. On might think that events such as the Folley scandal and his resignation in late september could influence the prices of both contracts. One way to look at this is to compare plots of the prices of two different states.  We do this for three pairs of states chosen from the six states widely publized as states the democrats would have to win to win control of the senate.
<center>http://www.dartmouth.edu/~chance/forwiki/repSenatecontrole.jpg </center>


While the probability that the Republicans would win control of the Senate decreases as the election gets closer, even the day before the election we would still predict that it is unlikely that the Democrats would win control of the Senate.  This is not consistent with our observation that, in terms of the estimates for the expected number of states they win, the Democrats  did have a reasonable chance of winning control of the Senate.


and consider the 33 Senate races.  The democrats had to win 24 or more of these races to win control the Senate.  Assuming that the indpendent candidates Joe Lieberman in  Connecticut and Bernie Sanders in Vermont will win their races and vote as democrates.  the democrates would have to win 22 or more of the remaining 31 races.  
For more serious attempts to measure the success of Markets in Predicting the Future we recomend the articles on this topic available at the [http://www.biz.uiowa.edu/iem/ Iowa Electronics Markets.]


We assume that the price of a contract for the democrats winning a particular state is an estimate for the probability they will win this state. If we could assume that the outcome of different states are independent the probability that the win a specified number of states would be the product of the probabilities for which each state. Then we coulc calculate probability that they would win 22 or more of the states. However, they are probability not independent,  But, we can estimate the expected number of states the democrats will by simply adding the probabilities for states with contract price greater than 1/2.
---DISCUSSION QUESTIONS---


Doing this using the closing price for each day from Jan 15 to November 6 we obtain the following graph:
(1) What other methods might be used to test the claim that Tradesports prices can be considered to be probabilities?


<center>http://www.dartmouth.edu/~chance/forwiki/demleads.jpg<\center>
(2) What other methods might be used to test the independence or dependence of the prices of contracts for different states.

Latest revision as of 19:36, 3 January 2007

Can predictions markets be right too often?
David Pennock

Prediction map post mortem.
Robert Forsythe

We have discussed in Chance News several times the use of betting markets such as Tradesports, the Iowa Political Markets and other such markets to predict the outcomes of elections, sports events, Oscar winners etc. See for example Chance News 12.02

Lance Fornow Computer Scientist at the University of Chicago, David Pennock and Chen Yiling Research Scientists at Yahoo have been interested in how to evaluate the ability of betting markets to make predictions.

In his Blog David Pennock has a nice article Evaluating probabilistic predictions. He also recommends an article in the Online New Republic "Can predictions markets forecast elections? Good Bet", Cass R. Sunstein & Bo Cowgill, 11.21.06.

In their discussion of the predictions relating to the 2006 Senate races, Fornow, Pennock and Yiling provided the following map showing results as of about 9 AM CST election day.

http://weblog.fortnow.com/media/election-day-2006-map.JPG

In his analysis, Fornow wrote:

Every state colored blue was won by a Democrat and every state colored red went to a Republican. But also note the 69% given to GOP (Republican) Senate control although this election will give control to the Democrats. No outcome would have made all the states and Senate control agree with the 9 AM map.

Were the markets inconsistent? No, because the markets predict not absolutely but probabilistically. For example, the markets gave a probability of winning 60% for each of Virginia and Missouri, and the Democrats needed both to take the Senate. If these races were independent events, the probability that the Democrats take both is 36% or a 64% chance of GOP Senate control assuming no other surprises.

Of course the races were not independent events and there are other states involved making it more difficult to compare the probabilities of the individual races with that of Senate control.

So how did the markets do as predictors? Quite well as the outcome seems quite reasonable given the markets. Other outcomes would have also been reasonable such as the Democrats losing Virginia and the Senate remaining in Republican hands, a possibility that came very close to happening.

We thought it would be interesting look in more detail at how Tradesports did in predicting the outcomes of the 2006 election. To do this we need to describe how the Tradesports betting works. As explained by WikiPedia:

Tradesports markets trades in the binary 0-100 format. If the event specified in a given contract occurs, the contract settles at 100 points, or $10 per contract or share; otherwise, the contract settles at 0. Thus, the current price of the contract can be imputed as the probability that the specified event will occur.

We will illustrate this in terms of a current event: will Hillary Clinton be nominated as the Democratic candidate for the 2008 presidential election?

Tradesports makes the initial price for a contract and after this the price is determined by offers to buy and sell the contracts. Thus Tradesports is simply a match maker, though there are some fees attached to this which are explained here

On December 23 Senator Clinton contracts were given in the following table.

http://www.dartmouth.edu/~chance/wikivideos/clintonbidask.jpg

.

A price of 53.0 is interpreted to mean that Senator Clinton has a 53% chance of being nominated. It would cost you $5.30 for this contract. Then you should win $10 with probability .53 and 0 with probability .47, giving you an expected winning of $5.30 and making it a fair bet.

For each day there is a closing price for a contract which varies through time as illustrated by the following graphic:

http://www.dartmouth.edu/~chance/wikivideos/clintongraph.jpg

Here are similar graphs for other leading candidates for Democratic and Republican nominations as appeared in a New York Times article," '08 Candidates in a Virtual Market", Dec. 5, 2006.

http://graphics8.nytimes.com/images/2006/12/05/us/politics/1205-nat-webACTION.gif

Now we return to the 2006 election. There were a number of contracts you could bid for. We will concentrate on the contracts for the Democrats winning individual states.

We should first see if the contract prices really act like probabilities. Here is an elementary test. When you buy a contract for the outcome of a particular state you have three choices for the contracts: The Democratic candidate will win, the Republican candidate will win, and some other candidate will win. if the prices are really probabilities the prices for these three choices should add up to about 1. Of course these prices are themselves random variables so we cannot expect them to add up to exactly 100.

Here are the sums for some of the states that were considered important for the Democrats to win to get control of the Senate.

http://www.dartmouth.edu/~chance/forwiki/tradesportstable.jpg

The sums are close enough to 100 that we can say that the Tradesportts prices pass this test.

Note that all the prices except Tennessee are greater than 1/2 so Tradesports would predict that, in all these states except Tennessee, the Democratic candidate would win the Senate race which is what actually happened.

Next we would like to see if the prices for contracts that the Democrats will win two different states are independent. One way to look at this is to compare plots of the prices of two different states. We do this for three pairs of states chosen from the six states widely publicized as states the Democrats would have to win to win control of the Senate.

http://www.dartmouth.edu/~chance/forwiki/VirginiaMissouri.jpg
Virginia prices (bottom) and Missouri prices (top)
http://www.dartmouth.edu/~chance/forwiki/RhodeIslandOhio.jpg
Rhode Island prices (bottom) and Ohio prices (top)
http://www.dartmouth.edu/~chance/forwiki/MontannaPennsylvania.jpg
Montana prices (bottom) and Pennsylvania prices (top)

Well, they do seem to follow each other and so we can say that they are not independent but this may not be completely convincing.

We look next at how the Tradesports might predict who would control the senate. There were 33 Senate races. The Democrats had to win 24 or more of these races to win control the Senate. It was pretty clear that the Independent candidates Joe Lieberman in Connecticut and Bernie Sanders in Vermont would win their races and vote as Democrats, so the Democrats would have to win 22 or more of the remaining 31 races.

If the prices of individual states are not independent we cannot estimate the probability that the Democrats win 22 or more races. But, we can estimate the expected number of states the Democrats will win by simply adding the probabilities for states with contract price greater than 1/2. Doing this, using the closing price for each day from January 15 to November 6, we obtain the following graph:

http://www.dartmouth.edu/~chance/forwiki/demleads.jpg

This suggests that only in the last days of the race did the Democrats appear to have a reasonable chance of winning the required 22 states.

Tradesports did not have a contract that the Democrats would win control of the Senate, but they did have a contract that the Republicans would, so we can use this to determine the probability that the Democrats win control of the Senate. Here is Tradesports graph of the closing prices for the contract that the Republicans would win control of the Senate:

http://www.dartmouth.edu/~chance/forwiki/repSenatecontrole.jpg

While the probability that the Republicans would win control of the Senate decreases as the election gets closer, even the day before the election we would still predict that it is unlikely that the Democrats would win control of the Senate. This is not consistent with our observation that, in terms of the estimates for the expected number of states they win, the Democrats did have a reasonable chance of winning control of the Senate.

For more serious attempts to measure the success of Markets in Predicting the Future we recomend the articles on this topic available at the Iowa Electronics Markets.

---DISCUSSION QUESTIONS---

(1) What other methods might be used to test the claim that Tradesports prices can be considered to be probabilities?

(2) What other methods might be used to test the independence or dependence of the prices of contracts for different states.