Reading Between The Numbers by Wendy McElroy

This story appeared first on FoxNews.com

Used with permission of the author

| Chapter 4 — Domestic Violence Statistics |

| Next — NIMH studies of domestic violence |

Wednesday, March 24, 2004 — Last week, sociologist Dr. Ellie Lee c ast doubt upon the increasingly popular theory that postnatal depression affects as many as one in five new mothers, telling the BBC that the research underlying that claim is “wrong.”

“According to 'experts,' growing numbers of women are traumatized by childbirth and are not capable of child-rearing without professional help,” Lee said, prompting a much larger question than the mental status of post-natal women:

How can we trust research?

The question is not trivial, since studies and statistics form the basis for many of the laws under which we live. If they are wrong, then the laws may be as well.

Short of taking a course on statistics and poring over data, the best way to get a sense of which data to trust is through common sense. There are five questions you should demand of any statistic.

Who Says So? The purpose of this question is to discover possible bias on the part of those offering the data. The bias may be conscious — such as research conducted expressly to win a government grant. The bias may be unconscious — such as research conducted by those who have deep ideological convictions that influence the questions asked.

Bias does not invalidate findings. Just because a researcher seeks funds or has a personal opinion doesn't mean his finding that 2+2=4 is false. But it does mean you should look at the math more closely.

How Does He Know? Imagine a researcher who rang doorbells at random to ask, “Are you a criminal” or “Do you suffer stomach gas often?” That researcher might discover the world to be both crime and gas-free — not because it actually is, but because many people will not admit to either. One of the most common methodological mistakes is to rely upon an unrepresentative sampling, such as polling only Baptists or limiting your sample to 10 people.

“Seventy-five percent of Americans prefer milk to lemonade” is an impressive finding until you realize that only 12 people were sampled, all of whom were Wisconsin dairy farmers. At that point, the surprising statistic is that 25 percent preferred lemonade. Demand to know the exact question asked or studied, the size of the sampling and whether it was random.

What does the competition say? Many studies contradict past findings or constitute “surprising revelations.” It could be that past or competing studies were flawed; times may have changed. Data, like opinions, can vary and a finding that “eating cheese increases your chance of cancer by 12 percent” should be considered in light of past or current studies that render different results. It is possible for a multitude of small surveys to be conducted until one of them produces the desired results.

For example, after tossing a coin many times, it will land “heads up” nine times in a row. From that isolated experience, a researcher could conclude that a tossed penny will come up heads 90 percent of the time. Do other findings contradict him?

What is missing? Does the data tell you enough to evaluate its statements? Consider the statement, “the average salary at this company is $30,000 a year.” Ninety percent of employees may make much less than that amount but, when total incomes are divided by total employees, $30,000 may be the “mean” result. A “median” result reflects what the person at the exact middle of the earning range takes home. The “mode” is nothing more than the most frequently encountered figure. Does the figure $30,000 indicate a mode, a median or a mean?

Did Someone Change the Subject? A newscaster states, “reports of domestic violence have increased” and concludes that “domestic violence is on the rise.” This conclusion is not justified because the increased reporting may reflect nothing more than a greater willingness on the part of women to contact the police or a greater willingness of police to file the reports. The newscaster has changed the subject from increased reporting to increased incidents.

Does It Make Sense? Never allow a statistical finding to override common sense or your own perceptions: guesstimate. That technique involves taking a statistic to its logical conclusion and seeing if it reduces to absurdity. Consider the alarming statement, “over 3,000,000 teenage girls on welfare became pregnant this year.”

Start with the total population of the U.S. — roughly 300 million. Assume that roughly half are male, leaving 150 million. Assume a uniform female age-spread of one to 75 years, with teenagers (13-19) constituting approximately 9.3 percent, or 14 million. Assume every teenage girl can become pregnant. Divide this figure by the reportedly three million pregnant welfare teens and the ratio you get is 4.67. One in five teenage girls is not only on welfare but has also become pregnant in the last year. Does this make sense, does it accord with your own perceptions?

The research on postnatal depression may or may not be valid. Lee accuses its advocates of constructing a problem, of “medicalizing motherhood.” Lee states, “There is every possibility that...parents will come to experience the normal disruption that parenting brings with it, as highly disabling, and find themselves less able to manage...This risks branding an essential part of life a hazard.”

Our society rewards those who construct problems. They receive financing and media attention, write books and become “experts.” Statistics are tools and those who wield them should be neither glamorized nor ignored. But they should be required to answer basic questions before being included in that rare category: purveyor of truth.

Wendy McElroy is the editor of ifeminists.com and a research fellow for The Independent Institute in Oakland, Calif. She is the author and editor of many books and articles, including the new book, Liberty for Women: Freedom and Feminism in the 21st Century (Ivan R. Dee/Independent Institute, 2002). She lives with her husband in Canada.

Top

| Chapter 4 — Domestic Violence Statistics |

| Next — NIMH studies of domestic violence |

| Back — Gathering statistics |

This site is supported and maintained by the Equal Justice Foundation.

Last modified 12/18/24