Chance News 73: Difference between revisions

From ChanceWiki
Jump to navigation Jump to search
Line 209: Line 209:


Submitted by Margaret Cibes based on a suggestion by Jim Greenwood
Submitted by Margaret Cibes based on a suggestion by Jim Greenwood
==Variation in Medicare hospitalizations==
[http://surface.syr.edu/cgi/viewcontent.cgi?article=1008&context=cpr&sei-redir=1#search=%22dan+wennberg,+dartmouth%22 “Variations Among Regions and Hospitals in Managing Chronic Illness: How Much Care Is Enough?”]<br>
by Dr. John E. Wennberg, Dartmouth Medical School<br>
Syracuse Policy Brief lecture series working paper, January 1, 2006<br>
Dr. Wennberg and his colleagues began the ongoing Dartmouth Atlas Project in 1993 to “study health care markets in the United States, measuring variations in health care resources and their utilization among geographic areas.”  They are focusing on Medicare databases because of the availability of its comprehensive data.  Their goal is to “define where people go for medical care they receive, and whether increasing investments in health care resources and their use result in better health outcomes.”<br>
This working paper contains charts showing variation in hospitalization stays and total expenses for several chronic health conditions.  Some conclusions are:
<blockquote>When looking at patterns of practice across the United States, sometimes, but rarely, the variation actually reflects medical need.  ….<br>
[I]f you look at the actual survival, you see that patients with hip fractures and colon cancer and heart attacks actually had <i>higher</i> mortality rates if they lived in the higher-spending regions than if they lived in the lower-spending regions.  There was no difference in patients’ functional status and satisfaction with their care, and their perceptions were that their access to care was worse.<br>
[T]here’s a lot of hard work ahead. The general rule in my experience has been about 10 years of denial of data, five years of blaming somebody else for the problem, and finally a crisis in which they say “Maybe we should roll up our sleeves and get to work.” We don’t have that kind of time in terms of the train wreck that Medicare is heading toward—that our whole system is heading toward.</blockquote>
For more information and data, see:<br>
(a)  [http://www.dartmouthatlas.org Dartmouth Atlas of Health Care], showing hospital-specific data for all U.S. hospitals<br>
(b)  [http://patients.dartmouth-hitchcock.org/documents/Wrestling_with_variation.pdf “Wrestling With Variation”], interview of Jack Wennberg, Syracuse University, 2004<br>
(c)  <i>More Harm Than Good</i>, by Alan Zelicoff MD and Michael Bellomo, 2008, in which the authors discuss the Dartmouth project and argue for more evidence-based health care and statistical/scientific training for doctors.<br>
Submitted by Margaret Cibes

Revision as of 14:20, 6 June 2011

Quotations

From The Flaw of Averages, by Sam L. Savage, Wiley, 2009

  • “Our culture encodes a strong bias either to neglect or ignore variation. We tend to focus instead on measures of central tendency, and as a result we make some terrible mistakes, often with considerable practical import.” (Stephen Jay Gould, cited p. 11)
  • “Plans based on average assumptions are wrong on average.” (Savage, p. 11)
  • “Far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise.” (John W. Tukey, cited p. 38)
  • “I have found that teaching probability and statistics is easy. The hard part is getting people to learn the stuff.” (Savage, p. 49)
  • “Statisticians often describe a numerical uncertainty using the Red Words, RANDOM VARIABLE, but I will stick with ‘uncertain number.’ …. [S]top thinking of uncertainties as single numbers and begin thinking of them as shapes, or distributions. …. If you think of an uncertain number as a bar graph, you will not be seriously misled.” (Savage, p. 59ff)
  • “Joe Berkson, a statistician at the Mayo Clinic, developed his own criterion, which he termed the IOT Test, or Inter Ocular Trauma Test, requiring a graph that hit you between the eyes.” (Savage, p. 325)

See Chance News 52 for a review of The Flaw of Averages by Laurie Snell.

Submitted by Margaret Cibes


“[O]n average Bill Gates and I can afford a new Rolls and a winter home in Provence.” (p. 36)

Howard Wainer, in Picturing the Uncertain World Princeton University Press, 2009

Submitted by Margaret Cibes


"Using a model of no greater sophistication than that employed by Benjamin Franklin (weather generally moves from west to east), I was able to predict that the area of precipitation currently over Ohio would be hitting New Jersey by tomorrow and would stay over us until the weekend. Any fool could see it. The improvement in forecasting has not been entirely due to improvements in the mathematical models of the weather. The enormous wealth of radar and satellite data summarized into a multicolored and dynamic graph can turn anyone into an expert."

Wainer, in Graphic Discovery A Trout in the Milk and Other Visual Adventures, p. 15

Submitted by Paul Alper


"This is about visual thinking and visual evidence …. It's not about commercial art. The last thing in the world that's needed here is a designer. What's needed is an analytical, statistical, quantitative approach. Reporting is different from pitching. Artists who design for marketing purposes inherently have problems with credibility. This is something very different in spirit. It's about accountability and transparency—with heavy, heavy amounts of data."

Remarks by Edward Tufte, on describing his assignment to help “track and explain $787 billion in recovery stimulus funds,” as a member of the Recovery Independent Advisory Panel (2010-2011), to which he was appointed by President Obama; cited by blogger at Bloomberg Businessweek[1]

Submitted by Margaret Cibes

Forsooth

“It is Friday 13th today and though it is still only ten in the morning some awfully unlucky things have happened. I stubbed my toe; the cat caught a shrew and left it in the middle of the kitchen floor, which was unlucky for me because I almost stepped on it, and was even more unlucky for the shrew. It is a black cat too. Clear evidence that superstition works, even for small rodents. Or perhaps not. Yesterday I broke my fingernail, but it wasn’t Friday 13th then, so that wasn’t the fates being lined up against me, it was just an accident.”

Julian Champkin, in “Friday 13th and black cats”, Significance online, May 2011

Submitted by Margaret Cibes


"[A] Public Policy Polling survey released Thursday found that Gingrich's favorable rating with GOP voters has dropped 27 points in the last month--from 52 percent to 38 percent."

Yahoo News, 26 May 2011

Submitted by Paul Alper


"On another occasion, Bailey and other staffers spent hours voting repeatedly to manipulate a television opinion poll on Palin’s decision to reject part of the federal government’s economic stimulus funding."

Review of Blind Allegiance to Sarah Palin,
Washington Post, 19 May 2011

Never mind the politics--this illustrates the worth of [voluntary] television opinion polls.

Submitted by Paul Alper


"Connecticut has the 2nd highest incidence of breast cancer in the U.S.

Connecticut. DON'T settle for second place!"

Race for the Cure ad, The Hartford Courant, May 2011

Submitted by Margaret Cibes

What's in a name?

Peter, Deborah popular names for CEOs
VPR News Morning Edition, 29 April 2011

"If your name is Peter or Deborah, you're more likely to be a CEO. That's what the social networking site LinkedIn found." You can listen to the rest of this Vermont Public Radio broadcast here.

The story was featured in a variety of news outlets:

Discussion Questions
1. A tweeted comment on this site says: "Great analysis, although this can be explained mostly by the age group ..." What are the implications of this? How might you explore them?

2. Paul Alper pointed out a post from Andrew Gelman's blog, where are reader cited a gruesome infographic that accompanied the original LinkedIn story.

Submitted by Jeanne Albert

Scaling the normal curve

Picturing the Uncertain World
by Howard Wainer, Princeton, 2009, p. 171.

This book is a collection of articles that Wainer had authored/co-authored in Chance (2000-2007), American Scientist (2007), and American Statistician (1996).

In Chapter 16, "Galton's Normal," Wainer gives an example of the relative heights of the points on a standard normal curve and of why our sketches of normal curves do not, and cannot, come close to accurate scale drawings.

He calculates that, even if the height at z = 13 were only 1 mm, then the height of the normal curve at the center, z = 0, would be about 5 x 10^30 km, or 5.3 x 10^17 light years. This is equivalent to a height that would be 3.4 million times larger than the universe. (His figures check out.)

Even if the height were 1 mm at z = 6, the height at z = 0 would be 66 km. Thus it still could not be drawn to scale.

Submitted by Margaret Cibes

Defined by rankings

In a data-heavy society, being defined by the numbers
by Alina Tugend , New York Times, 22 April 2011

On a humorous note, the author confesses that since joining Twitter she can't help regularly checking her number of followers. But the more serious question is this: Are we as a society too dependent on numerical rankings? The article quotes MIT professor Sherry Turkle: "One of the fantasies of numerical ranking is that you know how you got there. But the problem is if the numbers are arrived at in an irrational way, or black-boxed, so we don’t understand how we got there, then what use are they? "

The article gives several examples, two of which happen to correspond to stories that were recently discussed in Chance News 71, namely college rankings and New York City's formula for rating teachers.

Submitted by Bill Peterson

Identifying Osama bin Laden

“Matching Science of DNA With Art of Identification”
by Carl Bialik, The Wall Street Journal, May 7, 2011
“DNA and Bin Laden’s Positive ID”
by Carl Bialik, The Numbers Guy Blog, The Wall Street Journal, May 6, 2011

Bialik discusses the prosecutor’s fallacy in the context of reporting about the identification of Osama bin Laden’s body. He does not dispute the identification made by government officials, said to have been based on a number of factors, including DNA. However, he reminds readers that “claimed match probabilities, such as 99% or 99.99%, can be misstated or misleading” and that further details about a DNA test, as well as evidence related to other factors, must be taken into account before having confidence in an identification.

The problem boils down to this: A very small chance of a false positive in a genetic test isn't the same thing as a very large chance of a positive identification.

And he adds:

One complicating factor with interpreting genetic-identity tests … is that the probability of a positive match depends on what other information is available to confirm or reject it — despite the so-called prosecutor’s fallacy that confuses the two. These other factors, collected in what is called “prior odds” of a positive match, can be difficult to measure. “Many factors (e.g., age, sex, appearance, clothes, etc.) are relevant to prior odds, and there are no standard rule[s] for quantify[ing] them, [according to a University of South Texas scientist].”

Submitted by Margaret Cibes

Superstitution

“Friday 13th and black cats”
by Julian Champkin, Significance online, May 2011

Champkin wrote this brief column about superstitution, probably because May 13, 2011, was a “Friday the 13th.” (See his Forsooth quotation above.)

In one part of the column, he says that there are 25 finalists, on average, in the Eurovision Song Contest and goes on to suggest a winning strategy for betting:

Did you know that if you touch your left ear with your right thumb and wiggle your toes when the country you want to win begins to sing, that country will inevitably lose? It is a superstition that I have just invented, but I bet it works. Try it tomorrow and see.

Champkin goes on to say, “My bet will work overwhelmingly well, on average.”

Discussion

Give a statistical reason why Champkin’s method would work well, on average, with or without touching your ear and wiggling your toes.

Submitted by Margaret Cibes

Think tanks and common sense

On the economics of mass transit and the value of common sense
by Nate Silver, FiveThirtyEight blog, New York Times, 20 May 2011

Silver criticizes a Brookings Institution study of mass transit in the U.S.; he is surprised that it came to such strange numerical results: "New York, however, ranked just 13th. Washington ranked 17th. And Chicago ranked 46th — well behind Los Angeles (24th). Instead, the top 10 metro areas [for public transport] according to Brookings were Honolulu; San Jose, Calif.; Salt Lake City; Tucson; Fresno, Calif.; Denver; Albuquerque; Las Vegas; Provo, Utah; and Modesto, Calif."

He concludes with:

I want to point out that just because a study uses objective criteria, that doesn’t make it sensible. In fact, studies that try to rank or rate things seem especially susceptible to slapdash, unthoughtful methodology (here is another example: a study which concludes that Gainesville, Fla., is a more gay-friendly city than San Francisco). If you come up with a result that defies common sense — like Modesto’s having better public transit than New York — then once in a blue moon, you may be on to something: conventional wisdom is fallible. But much, much more often, it’s a sign that you’ve done something wrong, and it’s time to reconsider your assumptions before publishing.

Submitted by Paul Alper

Data display website

Nathan Yau is a Ph.D.candidate in Statistics at UCLA, who has a wonderful website about data displays, FlowingData. It contains lots of examples, including “ugly” ones[2].

Yau also has a book, Visualize This[3], coming out in July from Wiley, that “teaches you how to create graphics that tell stories with real data, and … have fun in the process.” It is said to help the reader to “make statistical graphics in R, design in Illustrator, and create interactive graphics in JavaScript and Flash & Actionscript.”

Submitted by Margaret Cibes based on a reference in Significance online[4]

Former UK Prime Minister as statistician

According to an article in Significance online[5], Harold Wilson, who served twice as UK Prime Minister (in the 1960s and the 1970s), was educated as an economist and also worked as a statistician, serving a term as President of the Royal Statistical Society (in the 1970s).

Submitted by Margaret Cibes

Unemployment vs. presidential support

“On the Maddeningly Inexact Relationship Between Unemployment and Re-Election”
by Nate Silver, FiveThirtyEight blog, The New York Times, June 2, 2011

Silver warns that, although higher unemployment in November 2012 will probably mean a lower likelihood of Obama being re-elected, there is no “magic number” for a rate or its behavior that would be sufficient by itself to predict the outcome of that election.

He provides a table[6] of raw data, for presidents elected in the period 1912-2008: unemployment rates at the start of a term and at the election date, the percent changes in rates, and the popular vote margins. He also provides six scatter plots:
(a) unemployment rates and margins of victory for incumbent parties 1912-2008 (R^2=0.0004) and 1948-2008 (R^2=0.0119)
(b) changes in unemployment rates and margins of victory for incumbent parties 1912-2008 (R^2=0.1222) and 1948-2008 (R^2=0.0106)
(c) unemployment rates and margins of victory when incumbent presidents sought second terms 1912-2008 (R^2=0.0162) and changes in rates for those presidents 1912-2008 (R^2=0.265).

While noting that the limited number of data points, as well as other aspects of the economy and voter attitudes toward it, make drawing any conclusions difficult, he feels that common sense tells us that “the unemployment rate should have some effect on a president’s re-election chances.”

He concludes, “[T]his is an inexact science – more so than either journalists or political scientists tend to acknowledge.”

See also “Gas Prices vs. Presidential Support” in Chance News 72.

Submitted by Margaret Cibes based on a suggestion by Jim Greenwood

Variation in Medicare hospitalizations

“Variations Among Regions and Hospitals in Managing Chronic Illness: How Much Care Is Enough?”
by Dr. John E. Wennberg, Dartmouth Medical School
Syracuse Policy Brief lecture series working paper, January 1, 2006

Dr. Wennberg and his colleagues began the ongoing Dartmouth Atlas Project in 1993 to “study health care markets in the United States, measuring variations in health care resources and their utilization among geographic areas.” They are focusing on Medicare databases because of the availability of its comprehensive data. Their goal is to “define where people go for medical care they receive, and whether increasing investments in health care resources and their use result in better health outcomes.”

This working paper contains charts showing variation in hospitalization stays and total expenses for several chronic health conditions. Some conclusions are:

When looking at patterns of practice across the United States, sometimes, but rarely, the variation actually reflects medical need. ….

[I]f you look at the actual survival, you see that patients with hip fractures and colon cancer and heart attacks actually had higher mortality rates if they lived in the higher-spending regions than if they lived in the lower-spending regions. There was no difference in patients’ functional status and satisfaction with their care, and their perceptions were that their access to care was worse.

[T]here’s a lot of hard work ahead. The general rule in my experience has been about 10 years of denial of data, five years of blaming somebody else for the problem, and finally a crisis in which they say “Maybe we should roll up our sleeves and get to work.” We don’t have that kind of time in terms of the train wreck that Medicare is heading toward—that our whole system is heading toward.

For more information and data, see:
(a) Dartmouth Atlas of Health Care, showing hospital-specific data for all U.S. hospitals
(b) “Wrestling With Variation”, interview of Jack Wennberg, Syracuse University, 2004
(c) More Harm Than Good, by Alan Zelicoff MD and Michael Bellomo, 2008, in which the authors discuss the Dartmouth project and argue for more evidence-based health care and statistical/scientific training for doctors.

Submitted by Margaret Cibes