While the general public may find anecdotal evidence highly compelling, most scientists are suspicious of data that rests on anecdotes. The idea research can be based solely on pooling more than one anecdote to create data is problematic – more than one anecdote is just more anecdote(s), not data.  The study released recently by Earthworks’ Oil and Gas Accountability Project, “Gas Patch Roulette” is an example.

There are many reasons why studies that rely on anecdotal information cannot be validated. For one, the study results/symptoms are vague and can’t be well defined. The problem being that rhetorical symptoms are subjective and rely on an individual’s recollection and judgment. For example a symptom defined as a sinus/respiratory issue could include a simple lingering regular cold.  Anecdote-based studies offer vague outcomes that are interpreted subjectively.  Subjective findings should not be allowed to take the place of quantitative measures and physical findings.

A recent study done by the Texas Department of Health, which set out to determine the relationship between natural gas activities and health, relied on physical findings and is a good example of how research should be done. The study used physical (blood levels) evidence to determine if there was a relationship between air emissions and the resident’s health.  The researchers concluded there was no connection.  This report casts doubt on the connection of air emissions to health issues; unlike the anecdotal reports from residents experiencing self-reported symptoms.  It does not imply residents did not experience symptoms they described, but that the symptoms could have been caused by another source.

Earthworks OGAP Report “Gas Patch Roulette” – An Example of Anecdotal Evidence Parading as Science

Another problem with studies based on anecdotal responses is confirmation bias, where researchers tend to seek out information that confirms what they already believe, want to believe or want to avoid.  The study released today by Earthworks, Oil and Gas Accountability Project, Gas Patch Roulette, is such an anecdotal based study.  It references another study also based on anecdotal findings (Bamberger, M. and Oswald, Robert E. “Impacts of Gas Drilling on Human and Animal Health.” New Solutions: A Journal of Environmental and Occupational Health Policy. Vol. 22(1)51-77, 2012.), doubling down on the anecdotal approach.

The placebo effect also comes into play. The media constantly covers the potential health risks associated with natural gas development and, in the process, broadcasts the shared symptoms. This may have altered or influenced the way an individual recollected or reported an ailment.

On the bright side, these recently released reports, the Grassroots Environmental Education’s “Human Health Risks and Exposure Pathways of Proposed Horizontal Hydrofracking in New York State” and the Earthworks report offer an opportunity to bridge the “facts” and “belief” gap. These two documents, weak and unscientific as they may be, offer the crux of anti’s argument against natural development. It is an opportunity to offer the reasons, point by point, why safe natural gas development can occur without the assumed human health impact, using real scientific evidence, such as the aforementioned Texas report.  There are others as well, such as this Australian study on the health of petroleum industry employees.

The following is a list of some other resources the reader may want to consider to get a better perspective on the difference between scientific and anecdotal evidence when it comes to natural gas:

• EID’s critique of a Colorado health impact study
• How ozone levels declined with Barnett Shale development
• How ozone data has been manipulated by natural gas opponents
• Another debunking of supposed health impacts in Texas
• An overall view of the health discussion surrounding natural gas development
• A debunking of Josh Fox’s “Sky Is Pink” assertions about natural gas and health
• A review of Bureau of Labor Statistics data demonstrating industry safety
• Another debunking of the Oswald and Bamberger study of animal impacts
• A summary of Texas air quality findings
• A University of Texas study of groundwater contamination from hydraulic fracturing
• Data demonstrating public health impacts from natural gas production are greatly overstated
• A DEP study of air quality in Southwest Pennsylvania
• A DEP study of air quality in Northeast Pennsylvania

Changing people’s perception about anecdotes is asking people to mistrust their feelings.  It is not an easy task. People tend to strongly believe personal experiences override objective quantitative evidence, even when good objective studies offer different conclusions.  They tend to conclude the objective study is wrong and work hard to find fault in it.  They even go further; they believe the personal stories, without any proof and without even trying to verify the story themselves; to them personal stories stand firmer than physical evidence supported studies.  Health, nonetheless, is not a matter of personal opinion, but a matter of objective facts. Senator Daniel Patrick Moynihan said it best – “You are entitled to your opinion. But you are not entitled to your own facts.”

Note:  An earlier version of this post appears on Uni Blake’s blog “Shale Health“