Sandbox
Incorrect metrics
U.S. health care is superior by most world measures
Letters, Wall Street Journal, 30 March 2010
As indicated by the well known aphorism, "It is not the heat, it’s the humidity," choosing the right metric is very important. Bayesians like to clobber the incorrect metric known as p-value, as can be seen by the following table due to Freeman (Freeman PR. The role of p-values in analysing trial results. Statistics in Medicine. 1993 Aug; 12(15-16): 1443-52).
| Number of Patients Receiving A and B |
Numbers Preferring A:B |
Percent Preferring A |
two-sided p-value |
| 20 | 15: 5 | 75.00 | 0.04 |
| 200 | 115: 86 | 57.50 | 0.04 |
| 2000 | 1046: 954 | 52.30 | 0.04 |
| 2000000 | 1001445: 998555 | 50.07 | 0.04 |
The same "statistically significant .04" appears in each row yet the practical significance is wildly different as the sample size varies, indicating that p-value as a metric is deficient.
The choice of an appropriate metric in health care is exceedingly important. The first letter to the Wall Street Journal link above shows how misleading the five-year survival rate metric can be. A letter writer defending the U.S. against the evils of socialized medicine replies to someone who was critical of the U.S. health care system compared to other countries:
For prostate cancer specifically, the rather astounding numbers are 92% in the U.S. versus 51% in the U.K. I am sure that the additional four out of 10 men in the U.K. who will die of prostate cancer find that to be significant, even if Dr. Krock does not.
Those "astounding numbers" refer to five-year survival rates. From Gigerenzer we learn of the misinterpretation common to patients and physicians: reliance on survival rates rather than mortality rates. The paper begins with former New York City mayor Rudy Giuliani thanking God for being treated for prostate cancer in the capitalist U.S. where his five-year survival chances were 82% as compared to "only 44% under socialized medicine." Turns out that "Giuliani’s numbers, however, are meaningless" because in the U.S. prostate cancer is being diagnosed earlier, a lead-time bias, and the cancer is being over diagnosed, that is, a pseudodisease is detected,--"screening-detected abnormalities that meet the pathologic definition of cancer but will never progress to cause symptoms in the patient’s lifetime." When it comes to mortality rates, there are "About 26 prostate cancer deaths per 100,000 American men versus 27 per 100,000 in Britain." The data "suggests that many American men have been unnecessarily diagnosed (i.e., overdiagnosed) with prostate cancer during the PSA era and have undergone unnecessary surgery and radiation treatment, which often leads to impotence and/or incontinence." In a nutshell, "The problem is that there is no relationship between 5-year survival and mortality."
Discussion
1. The "lead-time bias" is due to screening, i.e., mass testing, which is much more common in the U.S. than in other countries. Google the recent discussions regarding the efficacy of breast cancer screening in woman below the age of 50.
2. Gigerenzer’s article contains other interesting observations regarding relative risk vs. absolute risk. It also discusses the need to present Bayes theorem in a non-confusing way so that a patient and a doctor can understand the often remarkable numerical difference between Prob(test+|diseased) and Prob(diseased|test+).
Placebos getting stronger?
The growing power of the sugar pill
by Alix Spiegel, NPR, 8 March 2010
The randomized double-blind placebo-controlled experiment is regarded as the "gold standard" in medical research. This story describes new research indicating that our response to placebo treatments may be getting stronger over time. One example of this so-called "placebo drift" phenomenon is provided by Arthur Barsky, the director of psychiatric research at Brigham and Women's Hospital in Boston. Barsky compared trials on antidepressants from the 1980s with studies done in 2005, and found that the reactions to placebos had become twice as strong.
Ted Kaptchuk of the Harvard Medical School suggested a number of possible explanations. First, antidepressants have become more widely accepted in society. The more that people expect treatment to be successful, the better they tend to react to being treated, even with a placebo. Another possibility may related to funding, which increasingly comes from pharmaceutical companies. This creates pressure for the research to succeed, which could potentially bias the evaluation of the treatment. Finally, he notes that increased demand for research subjects may lead to the enrollment of marginally depressed people in the studies. Such subjects, who require less help to begin with, might plausibly be more responsive to placebos.
Kaptchuk has conducted earlier research on placebos. In a 2006 report entitled All placebos not created equal, Kaptchuk demonstrated that a fake acupuncture treatment could exhibit a stronger placebo effect than a fake pill in treating pain. In this case, Kaptchuk attributed the difference to the "medical ritual" associated with the acupuncture treatment.
Submitted by Bill Peterson