I was quoting the statistics, I wasn't pretending to be a statisitcian: Difference between revisions

From ChanceWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(22 intermediate revisions by 2 users not shown)
Line 5: Line 5:
Plus Magazine, 2002<br>
Plus Magazine, 2002<br>
Helen Joyce
Helen Joyce
[http://www.sciencemag.org/cgi/content/summary/309/5734/543a Flawed Statistics in Murder Trial May Cost Expert His Medical License]<br>
Science, 22 July 2005<br>


Multiple sudden infant deaths--coincidence or beyond coincidence<br>
Multiple sudden infant deaths--coincidence or beyond coincidence<br>
Line 12: Line 15:
___________________________________________________________________________________
___________________________________________________________________________________


Sir Roy Meadow is a pediatrician well known for his research in child abuse.  The BBC article reports that the UK General Medical Council (GMC) has found Sir Roy guilty of serious professional misconduct has "struck him off" the medical registry.  If upheld under appeal, this will prevent Meadow from practicing medicine in the UK.
Sir Roy Meadow is a pediatrician, well known for his research in child abuse.  The BBC article reports that the UK General Medical Council (GMC) has found Sir Roy guilty of serious professional misconduct and has "struck him off" the medical registry.  If upheld under appeal, this will prevent Meadow from practicing medicine in the UK.
 
This decision was based on flawed statistical estimate that Meadow made while testifying as an expert witness in a 1999 court case in which a Sally Clark was found guilty of murdering her two baby boys and given a life sentence.  The title of this page is taken from Meadow's defence at the hearing.


The death of Sally Clark's first child was recorded as a natural cot death.  Å "cot death" is other names for SIDS (sudden infant death syndrome).   
This decision was based on flawed statistical estimate that Meadow made while testifying as an expert witness in a 1999 trial in which a Sally Clark was found guilty of murdering her two baby boys and given a life sentence.   


The name SIDS was proposed by the pathologist Bruce Beckwih at a conference in 1969 and the definition, which is still current, was formulated at the conference by Beckwith and others as follows:  
To understand his testimony we need to know what a SIDS (sudden infant death syndrome) is.  The name SIDS was proposed by the pathologist Bruce Beckwih at a conference in 1969 and the definition, which is still current, along with many others,was formulated at the conference by Beckwith and others as follows:  


<blockquote>The sudden death of a baby that is unexpected by history and in whom a<br> thorough post-mortem examination fails to demonstrate an adequate cause of death. </blockquote>
<blockquote>The sudden death of a baby that is unexpected by history and in whom a<br> thorough post-mortem examination fails to demonstrate an adequate cause of death. </blockquote>


We were not able to find a transcript for the original trial but we were able to find from Lexis Nexis transcripts of two appeals that the Clarks made one in October 2000 in which the appeal was lost and the other in 2003 in which was successful and Sally Clark was releiced from jail. (See the reference for how to find these transcripts.) Since it is important to know just what statistical statements Meadow said we will given the discussion of this from the 2003 appeal.  
The death of Sally Clark's first baby was reported as a cot death, which is another name for SIDS.  Then when her second baby died she was arrested and tried for murdering both her children.
 
THE STATISTICAL EVIDENCE


Before turning to consider the evidence which it is alleged was not disclosed to the defence and the evidence now
We were not able to find a transcript for the original trial but from Lexis Nexis we found transcripts of two appeals that the Clarks made, one in October 2000 that they lost, and the other in April 2003 which they won and Sally Clark was released from jail. The 2003 transcript reported on the statistical testimony in the original trial as follows:
available that flows from it, we must record the other feature of the evidence at trial which it is contended may well have
had an unfair impact upon the jury's considerations. That evidence was statistical evidence given by Professor Meadow.


Professor Meadow was asked about some statistical information as to the happening of two cot deaths within the  
<blockquote> Professor Meadow was asked about some statistical information as to the happening of two cot deaths within the  
same family, which at that time was about to be published in a report of a government funded multi--disciplinary research  
same family, which at that time was about to be published in a report of a government funded multi--disciplinary research  
team, the Confidential Enquiry into Sudden Death in Infancy (CESDI) entitled 'Sudden Unexpected Deaths in Infancy' to  
team, the Confidential Enquiry into Sudden Death in Infancy (CESDI) entitled 'Sudden Unexpected Deaths in Infancy' to  
which the professor was then writing a Preface. Professor Meadow said that it was 'the most reliable study and easily the  
which the professor was then writing a Preface. Professor Meadow said that it was 'the most reliable study and easily the  
largest and in that sense the latest and the best' ever done in this country.  
largest and in that sense the latest and the best' ever done in this country. <br><br>
 
It was explained to the jury that there were factors that were suggested as relevant to the chances of a SIDS death within a given family; namely the age of the mother, whether there was a smoker in the household and the absence of a wage-earner in the family.<br><br>


It was explained to the jury that there were factors that werewage earner in the family. None of these factors had relevance to the Clark family and Professor Meadow was asked if a  
None of these factors had relevance to the Clark family and Professor Meadow was asked if a  
figure of 1 in 8,543 reflected the risk of there being a single SIDS within such a family. He agreed that it was. A table  
figure of 1 in 8,543 reflected the risk of there being a single SIDS within such a family. He agreed that it was. A table  
from the CESDI report was placed before the jury. He was then asked if the report calculated the risk of two infants dying  
from the CESDI report was placed before the jury. He was then asked if the report calculated the risk of two infants dying  
of SIDS in that family by chance. His reply was: 'Yes, you have to multiply 1 in 8,543 times 1 in 8,543 and I think it  
of SIDS in that family by chance. His reply was: 'Yes, you have to multiply 1 in 8,543 times 1 in 8,543 and I think it  
gives that in the penultimate paragraph. It points out that it's approximately a chance of 1 in 73 million.'  
gives that in the penultimate paragraph. It points out that it's approximately a chance of 1 in 73 million.' <br><br>


It seems that at this point Professor Meadow's voice was dropping and so the figure was repeated and then  
It seems that at this point Professor Meadow's voice was dropping and so the figure was repeated and then  
Professor Meadow added: '1 in England, Wales and Scotland there are about say 700,000 live births a year, so it is saying  
Professor Meadow added: 'In England, Wales and Scotland there are about say 700,000 live births a year, so it is saying  
by chance that happening will occur about once every hundred years.'  
by chance that happening will occur about once every hundred years.' <br><br>


Mr Spencer then pointed to the suspicious features alleged by the Crown in this present case and asked: 'So is  
Mr. Spencer [for the prosecution] then pointed to the suspicious features alleged by the Crown in this present case and asked: 'So is  
this right, not only would the chance be 1 in 73 million but in addition in these two deaths there are features which would
this right, not only would the chance be 1 in 73 million but in addition in these two deaths there are features, which wuld
be regarded as suspicious in any event?' He elicited the reply 'I believe so'.  
be regarded as suspicious in any event?' He elicited the reply 'I believe so'. <br><br>


All of this evidence was given without objection from the defence but Mr Bevan QC (who represented the  
All of this evidence was given without objection from the defence but Mr. Bevan (who represented the  
appellant at trial and at the first appeal but not before us) cross--examined the doctor. He put to him figures from other  
Appellant at trial and at the first appeal but not before us) cross--examined the doctor. He put to him figures from other  
research that suggested that the figure of 1 in 8,543 for a single cot death might be much too high. He then dealt with the  
research that suggested that the figure of 1 in 8,543 for a single cot death might be much too high. He then dealt with the  
chance of two cot deaths and Professor Meadow responded:  
chance of two cot deaths and Professor Meadow responded:  
'This is why you take what's happened to all the children into account, and that is why you end up saying the chance  
'This is why you take what's happened to all the children into account, and that is why you end up saying the chance  
of the children dying naturally in these circumstances is very, very long odds indeed one in 73 million.'  
of the children dying naturally in these circumstances is very, very long odds indeed one in 73 million.'  
He then added:  
He then added: <br><br>


'. . . it's the chance of backing that long odds outsider at the Grand National, you know; let's say it's a 80 to 1 chance,  
'. . . it's the chance of backing that long odds outsider at the Grand National, yu know; let's say it's a 80 to 1 chance,  
you back the winner last year, then the next year there's another horse at 80 to 1 and it is still 80 to 1 and you back it again  
you back the winner last year, then the next year there's another horse at 80 to 1 and it is still 80 to 1 and you back it again  
and it wins. Now here we're in a situation that, you know, to get to these odds of 73 million you've got to back that 1 in 80  
and it wins. Now here we're in a situation that, you know, to get to these odds of 73 million you've got to back that 1 in 80  
Line 64: Line 63:
know, you've happened to have won it, but the chance of it happening four years running we all know is extraordinarily  
know, you've happened to have won it, but the chance of it happening four years running we all know is extraordinarily  
unlikely. So it's the same with these deaths. You have to say two unlikely events have happened and together it's very,  
unlikely. So it's the same with these deaths. You have to say two unlikely events have happened and together it's very,  
very, very unlikely.'  
very, very unlikely.' <br><br>
 
The table that was produced to the jury gave just the figures for probability of a SIDS death in families where
one or more factors thought to be relevant were present together with the figures when there was no such factor. In the
CESDI report the table was accompanied by explanatory text but although this was available to the prosecution and the
defence, it was not before the jury. It made clear the purpose of the information saying: 'The identification of families
at higher risk of SIDS is of importance in allowing the appropriate deployment of scarce health care resources and in
attempting to achieve changes in lifestyle or patterns of child care that might reduce this risk.' It did not in any way suggest
that it provided statistical information that would enable diagnosis of an unnatural death in an individual case.
The report also made clear that the figures did not 'take account of possible familial incidence of factors other
than those included' in the table. It ended with the warning: 'When a second SIDS death occurs in the same family, in
addition to careful search for inherited disorder, there must always be a very thorough investigation of the circumstances--
though it would be inappropriate to assume maltreatment was always the cause.'
 
None of these qualifications were referred to by Professor Meadow in his evidence to the jury and thus it was the
headline figures of 1 in 73 million that would be uppermost in the jury's minds with the evidence equated to the chances
of backing four 80 to 1 winners of the Grand National in successive years.
 
Professor Berry was one of the four editors of the CESDI study. He made the point that simply squaring the
figure was an illegitimate over simplification and he drew attention to the qualifications to which we have referred.


The trial judge clearly tried to divert the jury away from reliance on this statistical evidence. He said:  
The trial judge clearly tried to divert the jury away from reliance on this statistical evidence. He said:  
'I should, I think, members of the jury just sound a word of caution about the statistics. However compelling you may  
'I should, I think, members of the jury just sound a word of caution about the statistics. However compelling you may  
find them to be, we do not convict people in these courts on statistics. It would be a terrible day if that were so. If there is  
find them to be, we do not convict people in these courts on statistics. It would be a terrible day if that were so. If there is  
one SIDS death in a family, it does not mean that there cannot be another one in the same family.'  
one SIDS death in a family, it does not mean that there cannot be another one in the same family.' </blockquote>
 
Note that Meadow obtained the odds of 73 million to one from the CESDI report so there is some truth to the statement "I was quoting the statistics, I wasn't pretending to be a statistician” that Meadow made to the General Medical Council. Note also that both Meadow and the Judge took this statistic seriously and must have felt that it was evidence that Sally Clark was guilty.  This was also true of the press.  The Sunday Mail (Queenstand, Australia) had an article titled "Mum killed her babies" in which we read:


<BLOCKQUOTE>Medical experts gave damning evidence that the odds of both children dying from cot death were 73 million to one.</blockquote>


There are two obvious problems with this 1 in 73 million statistic: (1) Because of environmental and genetics effects it seems very unlikely that the a SIDS death for a families the first baby and for their second baby are independent and (2) These odds might suggest to the jury that there is a 1 in 74 million chance that Sally Clark is innocent. This is, of course,  the well-known prosecuter's paradox.  In addition, the medical experts testimonies were very technical and some were contradictory. The 1 in 73 million odds were something the jury would at least understand.


Of course these odds  of 73 million to 1 for SIDS teaths are useless to the jury in assessing guilt unless they are also given the corresponding odds that the deaths were the result of murders.  In an article on the Sally Clark trial. Professor Roy Hill estimated that, for a randomly chosen family with two baby deaths, the probability that the deaths are the result of SIDS is about 10 times more likely than that probability that they are the result of murders.  Thus, if this kind of statistical evidence means anything, it suggests that Sally Clark is innocent.


The Clarks had their first appeal in 2 October 2000.  By this time the Clarks realized that they had to have there own statisticians as expert witnesses.  They chose Ian Everett from the Forensic Science Service and Phillip Dawid, Professor of Statistics at the Department of Statistical Science, University College London.  Both of these statisticians have specialized in statistical evidence in the courts.  In his report Dawid gave a very clear description of what would be required to obtain a reasonable  estimate of the probability of two SIDS deaths in a randomly chosen family with two babies.  He emphasized that it would be important also to have some estimate of the variability of this estimate.  Then he gave an equally clear discussion on the relevance of this probability, emphasizing the need for the corresponding probability of two murders in a family with two children.  His conclusion was:


There are three obvious problems with this 1 in 73 million statistic: (1) Because of environmental and genetics effects it seems very unlikely that the a SIDS death for a families the first baby and for their second baby are independent and (2) giving such odds with no further explanation might suggest to the jury that their is a 1 in 74 million chance that Sally Clark is innocent and (3) neither the defense or the prosecution suggested that the deaths were SIDS deaths--the prosecution claiming that they were murders, and the defense that they were natural deaths explained by the medical evidenceAs far as we can tell only problem (1) was seriously discussed in the newsThe first article to discuss this issue appears to be the article  "False statistics may have led to solicitor's murder conviction" by Sherry Norton in The Independent (London), December 31, 1999. This article reports on and article by Dr Stephen Watkins, director of public health at Stockport health entitled "Conviction by mathematical error? (BMJ, 2000, 320, 2-3). Norton writes:
<blockquote>The figure “1 in 73 million” quoted in Sir Roy Meadow's testimony at trial, as the probability of two babies both dying of SIDS in a family like Sally Clark's, was highly misleading and prejudicial.  The value of this probability has not been estimated with anything like the precision suggested, and could well be very much higher.  But, more important, the figure was presented with no explanation of the logically correct use of such information - which is very different from what a simple intuitive reaction might suggestIn particular, such a figure could only be useful if compared with a similar figure calculated under the alternative hypothesis that both babies were murderedEven though assessment of the relevant probabilities may be difficult, there is a clear and well-established statistical logic for combining them and making appropriate inferences from them, which was not appreciated by the court. </blockquote>


<blockquote>In Britain, there are 344 cot deaths each year, with the risk for the whole population being one cot death in 2.75 million. There are 378,000 second or subsequent births in England each year, and Dr Watkins has calculated that, if cot death is random, two such deaths will occur in the same family somewhere in England once every seven years.<br><br>
These two statisticians were not allowed to appear in the court proceedings but only to have their reports read.


However, because cot deaths are not random and can be linked to genetic defects, social class and lifestyle, recurrence rates have been found to be about five times the general rate. Therefore Dr Watkins has concluded that two cot deaths in the same family will, on average, occur at least once every one and a half years. </blockquote>
The Clarks Ground 3a for the appeal dealt with the statistical issues including Meadow's incorrect calculation and the Judge failing to warn the jury against the "prosecutor's fallacy".


Problem (2) was addressed by  Wai-Chin Leung, Senior Registrar in Public Health Medicine in his response  to Watkins article: "Conditional probability should be applied" to Watkins's article. Wai-Ching writes:
Concerning the miscalulation of the odds for two SIDS in a family of two children he remarks that this was already known and all that really mattered was that appearance of two SIDS deaths is unusual.


<blockquote>It is vital to state the precise conditional and outcome events as well as the underlying assumptions in quoting probability figures. All the figures quoted by Watkins(1) in his discussion about the use of probability in the conviction of Sally Clark could be correct:- <br><br>
He dismisses the prosecutors fallacy with the remark:


1) The probability that two consecutive infants died of cot death in a given social class I family is 1 in 73 million, assuming no increase in recurrence rate.<br><br>
He [Everett] makes the obvious point that the evidential material in Table 3.58 tell us nothing whatsoever as to the guilt or innocence of the appellant.  


2) The probability that two consecutive infants died of cot death in a given social class I family, given that the first infant had already died, is 1 in 8,500, again assuming no increase in recurrence rate.<br><br>
He concludes:


3) The probability that two consecutive infants died of cot death in a family is 1 in 2.75 million, again assuming no increase in recurrence rate. <br><br>
<blockquote> Thus we do not think that the matters raised under Ground 3(a) are capable of affecting the safety of the
convictions. They do not undermine what was put before the jury or cast a fundamentally different light on it. Even if
they had been raised at trial, the most that could be expected to have resulted would be a direction to the jury that the issue
was the broad one of rarity, to which the precise degree of probability was unnecessary. </blockquote>


However, none of these are relevant to the jury. The relevant question is: What is the probability that the causes of deaths of the infants in this family were cot deaths, given that these deaths have occurred? Surely, the answer would depend on the comparative likelihood of the two possible causes of death - cot death and homicide. For example, if we can rule out homicide completely (e.g. by several other witnesses), the probability that the causes were cot death is 1 (certainty). </blockquote>
   


Thus Wai-Ching is talking about the "Prosecutor's Paradox."  This is usually analyzed using Bayes Theorem which is probably too much for the jury.  However, as he points out, there is a much simpler way to explain this.  We need only estimate the probability that the deaths are murders  given that are murders or SIDS deaths.  We shall see later that one estimate shows that deaths are 9 times more likely than cot deaths.  Thus with no other information, if the deaths are cot deaths or murders the probability that Sally Clark is innocent is 9/10 and faced with this situation the jury would certainly not convict Sally Clark. This may not be so convincing if we compared murders with any natural deaths but it would probably still say that murder was more likely that natural death. 


These were nicely spelled out by a letter from the Royal Statistical Society to the Lord Chanceller dated 23 January 2002. This letter included the following discussion of the statistical issues involved:


<blockquote>You will be aware of the considerable public attention aroused by the recent conviction, confirmed on appeal, of Sally Clark for the murder of her two infants. One focus of the public attention was the statistical evidence given by a medical expert witness, who drew on a published study to obtain an estimate of the frequency of sudden infant death syndrome (SIDS, or "cot death") in families having some of the characteristics of the defendant's family. The witness went on to square this estimate to obtain a value of 1 in 73 million for the frequency of two cases of SIDS in such a family. This figure had an immediate and dramatic impact on all media reports of the trial, and it is difficult to believe that it did not also influence jurors.<br><br>
The calculation leading to 1 in 73 million is invalid. It would only be valid if SIDS cases arose independently within families, an assumption that would need to be justified empirically. Not only was no such empirical justification provided in the case, but there are very strong reasons for supposing that the assumption is false. There may well be unknown genetic or environmental factors that predispose families to SIDS, so that a second case within the family becomes much more likely than would be a case in another, apparently similar, family.<br><br>
A separate concern is that the characteristics used to classify the Clark family were chosen on the basis of the same data as was used to evaluate the frequency for that classification. This double use of data is well recognize by statisticians as perilous, since it can lead to subtle yet important biases.<br><br>
For these reasons, the 1 in 73 million figure cannot be regarded as statistically valid. The Court of Appeal recognized flaws in its calculation, but seemed to accept it as establishing "... a very broad point, namely the rarity of double SIDS" [AC judgment, para 138]. However, not only is the error in the 1 in 73 million figure likely to be very large, it is almost certainly in one particular direction - against the defendant. Moreover, following from the 1 in 73 million figure at the original trial, the expert used a figure of about 700,000 UK births per year to conclude that "... by chance that happening will occur every 100 years". This conclusion is fallacious, not only because of the invalidity of the 1 in 73 million figure, but also because the 1 in 73 million figure relates only to families having some characteristics matching that of the defendant. This error seems not to have been recognized by the Appeal Court, who cited it without critical comment [AC judgment para 115]. Leaving aside the matter of validity, figures such as the 1 in 73 million are very easily misinterpreted. Some press reports at the time stated that this was the chance that the deaths of Sally Clark's two children were accidental. This mis-interpretation is a serious error of logic known as the Prosecutor's Fallacy (1). The jury needs to weigh up two competing explanations for the babies' deaths: SIDS or murder. The fact that two deaths by SIDS are quite unlikely is, taken alone, of little value. Two deaths by murder may well be even more unlikely. What matters is the relative likelihood of the deaths under each explanation, not just how unlikely they are under one explanation. </blockquote>
References
References


Transcriipt for the 2000 and 2003 appeals can be obtained from Lexis Nexis following the following route:
Transcript for the 2000 and 2003 appeals can be obtained from Lexis Nexis following the following route:


Open Lexis Nexis
Open Lexis Nexis
Line 136: Line 115:
Choose" Commonwealth and Foreign Nations" from the sidebar
Choose" Commonwealth and Foreign Nations" from the sidebar


Choose  "Sally Clark"  for the " Keyword"
Choose  "Sally Clark” for the " Keyword"


Choose "UK Cases" for the "Source"
Choose "UK Cases" for the "Source"


Choose "Previous five years"  for the "Date."
Choose "Previous five years” for the "Date."


The two " r v Clarks" are the appeals.
The two " r v Clarks" are the appeals.


To be continued
To be continued
Retrieved from "http://chance.dartmouth.edu/chancewiki/index.php/I_was_quoting_the_statistics%2C_I_wasn%27t_pretending_to_be_a_statisitcian"
The two " r v Clarks" are the appeals.
To be continued
Retrieved from "http://chance.dartmouth.edu/chancewiki/index.php/I_was_quoting_the_statistics%2C_I_wasn%27t_pretending_to_be_a_statisitcian"

Latest revision as of 03:08, 30 July 2005

Sir Roy Meadow struck off by GMC
BBC News, 15 July 2005

Beyond reasonable doubt
Plus Magazine, 2002
Helen Joyce

Flawed Statistics in Murder Trial May Cost Expert His Medical License
Science, 22 July 2005

Multiple sudden infant deaths--coincidence or beyond coincidence
Paediatric and Perinatal Epidemiology 2004, 18, 320-326
Roy Hill

___________________________________________________________________________________

Sir Roy Meadow is a pediatrician, well known for his research in child abuse. The BBC article reports that the UK General Medical Council (GMC) has found Sir Roy guilty of serious professional misconduct and has "struck him off" the medical registry. If upheld under appeal, this will prevent Meadow from practicing medicine in the UK.

This decision was based on flawed statistical estimate that Meadow made while testifying as an expert witness in a 1999 trial in which a Sally Clark was found guilty of murdering her two baby boys and given a life sentence.

To understand his testimony we need to know what a SIDS (sudden infant death syndrome) is. The name SIDS was proposed by the pathologist Bruce Beckwih at a conference in 1969 and the definition, which is still current, along with many others,was formulated at the conference by Beckwith and others as follows:

The sudden death of a baby that is unexpected by history and in whom a
thorough post-mortem examination fails to demonstrate an adequate cause of death.

The death of Sally Clark's first baby was reported as a cot death, which is another name for SIDS. Then when her second baby died she was arrested and tried for murdering both her children.

We were not able to find a transcript for the original trial but from Lexis Nexis we found transcripts of two appeals that the Clarks made, one in October 2000 that they lost, and the other in April 2003 which they won and Sally Clark was released from jail. The 2003 transcript reported on the statistical testimony in the original trial as follows:

Professor Meadow was asked about some statistical information as to the happening of two cot deaths within the

same family, which at that time was about to be published in a report of a government funded multi--disciplinary research team, the Confidential Enquiry into Sudden Death in Infancy (CESDI) entitled 'Sudden Unexpected Deaths in Infancy' to which the professor was then writing a Preface. Professor Meadow said that it was 'the most reliable study and easily the largest and in that sense the latest and the best' ever done in this country.

It was explained to the jury that there were factors that were suggested as relevant to the chances of a SIDS death within a given family; namely the age of the mother, whether there was a smoker in the household and the absence of a wage-earner in the family.

None of these factors had relevance to the Clark family and Professor Meadow was asked if a figure of 1 in 8,543 reflected the risk of there being a single SIDS within such a family. He agreed that it was. A table from the CESDI report was placed before the jury. He was then asked if the report calculated the risk of two infants dying of SIDS in that family by chance. His reply was: 'Yes, you have to multiply 1 in 8,543 times 1 in 8,543 and I think it gives that in the penultimate paragraph. It points out that it's approximately a chance of 1 in 73 million.'

It seems that at this point Professor Meadow's voice was dropping and so the figure was repeated and then Professor Meadow added: 'In England, Wales and Scotland there are about say 700,000 live births a year, so it is saying by chance that happening will occur about once every hundred years.'

Mr. Spencer [for the prosecution] then pointed to the suspicious features alleged by the Crown in this present case and asked: 'So is this right, not only would the chance be 1 in 73 million but in addition in these two deaths there are features, which wuld be regarded as suspicious in any event?' He elicited the reply 'I believe so'.

All of this evidence was given without objection from the defence but Mr. Bevan (who represented the Appellant at trial and at the first appeal but not before us) cross--examined the doctor. He put to him figures from other research that suggested that the figure of 1 in 8,543 for a single cot death might be much too high. He then dealt with the chance of two cot deaths and Professor Meadow responded: 'This is why you take what's happened to all the children into account, and that is why you end up saying the chance of the children dying naturally in these circumstances is very, very long odds indeed one in 73 million.' He then added:

'. . . it's the chance of backing that long odds outsider at the Grand National, yu know; let's say it's a 80 to 1 chance, you back the winner last year, then the next year there's another horse at 80 to 1 and it is still 80 to 1 and you back it again and it wins. Now here we're in a situation that, you know, to get to these odds of 73 million you've got to back that 1 in 80 chance four years running, so yes, you might be very, very lucky because each time it's just been a 1 in 80 chance and you know, you've happened to have won it, but the chance of it happening four years running we all know is extraordinarily unlikely. So it's the same with these deaths. You have to say two unlikely events have happened and together it's very, very, very unlikely.'

The trial judge clearly tried to divert the jury away from reliance on this statistical evidence. He said: 'I should, I think, members of the jury just sound a word of caution about the statistics. However compelling you may find them to be, we do not convict people in these courts on statistics. It would be a terrible day if that were so. If there is

one SIDS death in a family, it does not mean that there cannot be another one in the same family.'

Note that Meadow obtained the odds of 73 million to one from the CESDI report so there is some truth to the statement "I was quoting the statistics, I wasn't pretending to be a statistician” that Meadow made to the General Medical Council. Note also that both Meadow and the Judge took this statistic seriously and must have felt that it was evidence that Sally Clark was guilty. This was also true of the press. The Sunday Mail (Queenstand, Australia) had an article titled "Mum killed her babies" in which we read:

Medical experts gave damning evidence that the odds of both children dying from cot death were 73 million to one.

There are two obvious problems with this 1 in 73 million statistic: (1) Because of environmental and genetics effects it seems very unlikely that the a SIDS death for a families the first baby and for their second baby are independent and (2) These odds might suggest to the jury that there is a 1 in 74 million chance that Sally Clark is innocent. This is, of course, the well-known prosecuter's paradox. In addition, the medical experts testimonies were very technical and some were contradictory. The 1 in 73 million odds were something the jury would at least understand.

Of course these odds of 73 million to 1 for SIDS teaths are useless to the jury in assessing guilt unless they are also given the corresponding odds that the deaths were the result of murders. In an article on the Sally Clark trial. Professor Roy Hill estimated that, for a randomly chosen family with two baby deaths, the probability that the deaths are the result of SIDS is about 10 times more likely than that probability that they are the result of murders. Thus, if this kind of statistical evidence means anything, it suggests that Sally Clark is innocent.

The Clarks had their first appeal in 2 October 2000. By this time the Clarks realized that they had to have there own statisticians as expert witnesses. They chose Ian Everett from the Forensic Science Service and Phillip Dawid, Professor of Statistics at the Department of Statistical Science, University College London. Both of these statisticians have specialized in statistical evidence in the courts. In his report Dawid gave a very clear description of what would be required to obtain a reasonable estimate of the probability of two SIDS deaths in a randomly chosen family with two babies. He emphasized that it would be important also to have some estimate of the variability of this estimate. Then he gave an equally clear discussion on the relevance of this probability, emphasizing the need for the corresponding probability of two murders in a family with two children. His conclusion was:

The figure “1 in 73 million” quoted in Sir Roy Meadow's testimony at trial, as the probability of two babies both dying of SIDS in a family like Sally Clark's, was highly misleading and prejudicial. The value of this probability has not been estimated with anything like the precision suggested, and could well be very much higher. But, more important, the figure was presented with no explanation of the logically correct use of such information - which is very different from what a simple intuitive reaction might suggest. In particular, such a figure could only be useful if compared with a similar figure calculated under the alternative hypothesis that both babies were murdered. Even though assessment of the relevant probabilities may be difficult, there is a clear and well-established statistical logic for combining them and making appropriate inferences from them, which was not appreciated by the court.

These two statisticians were not allowed to appear in the court proceedings but only to have their reports read.

The Clarks Ground 3a for the appeal dealt with the statistical issues including Meadow's incorrect calculation and the Judge failing to warn the jury against the "prosecutor's fallacy".

Concerning the miscalulation of the odds for two SIDS in a family of two children he remarks that this was already known and all that really mattered was that appearance of two SIDS deaths is unusual.

He dismisses the prosecutors fallacy with the remark:

He [Everett] makes the obvious point that the evidential material in Table 3.58 tell us nothing whatsoever as to the guilt or innocence of the appellant.

He concludes:

Thus we do not think that the matters raised under Ground 3(a) are capable of affecting the safety of the

convictions. They do not undermine what was put before the jury or cast a fundamentally different light on it. Even if they had been raised at trial, the most that could be expected to have resulted would be a direction to the jury that the issue

was the broad one of rarity, to which the precise degree of probability was unnecessary.



References

Transcript for the 2000 and 2003 appeals can be obtained from Lexis Nexis following the following route:

Open Lexis Nexis

Choose "Legal Research" from the sidebar

From "Case Law" choose "Get a Case"

Choose" Commonwealth and Foreign Nations" from the sidebar

Choose "Sally Clark” for the " Keyword"

Choose "UK Cases" for the "Source"

Choose "Previous five years” for the "Date."

The two " r v Clarks" are the appeals.

To be continued

Retrieved from "http://chance.dartmouth.edu/chancewiki/index.php/I_was_quoting_the_statistics%2C_I_wasn%27t_pretending_to_be_a_statisitcian" The two " r v Clarks" are the appeals. To be continued

Retrieved from "http://chance.dartmouth.edu/chancewiki/index.php/I_was_quoting_the_statistics%2C_I_wasn%27t_pretending_to_be_a_statisitcian"