A new study on methane was published this week, arguing that emissions from seven well pads emit methane “2 to 3 orders of magnitude greater than US Environmental Protection Agency estimates” during the drilling phase of a well.  Unsurprisingly, several news outlets put out the expected headlines portending the usual doom.  Greenwire proclaims: “Significant methane leaks found from wells still in drilling process.”  The LA Times put it this way: “EPA drastically underestimates methane released at drilling sites.”

That all sounds quite serious, until you realize who the authors of the study are: Anthony Ingraffea and Robert Howarth. It’s no secret that this team of activists wants to ban hydraulic fracturing, so it’s also not surprising that they arrived at a conclusion to advance that cause. If you need a refresher, Ingraffea is the president of Physicians Scientists & Engineers for Healthy Energy, an organization that is not only funded by the Park Foundation, but which has taken institutional stances against fracking (Ingraffea actually listed PSEHE ahead of his Cornell affiliation for the study’s author page). In 2012, Howarth signed a “pledge of resistance” to hydraulic fracturing, joining Bill McKibben, Josh Fox, Jakob Dylan, and other “scientists.”

Many of the news reports on the study omitted those details entirely, which is probably what they preferred.

As with their previous research, this latest paper also contains a number of glaring flaws, ranging from small sample sizes to a curious disregard of established climate science.  Here are five things to know about their latest study:

Fact #1: Researchers specifically target high emission areas, admit little to no emissions from other well pads

The report states,

“An instrumented aircraft platform was used to identify large sources of methane and quantify emission rates in southwestern PA in June 2012.” (p. 1; emphasis added)

If they were using an aircraft to locate areas with higher methane readings, then why is it “news” that they found high methane readings? Even worse is the researchers’ acknowledgment that the well sites they studied were outliers: The study is clear that the researchers did not detect high emissions in any other well pads other than the seven they focused on; therefore, these readings cannot be taken as representative of Marcellus development as a whole:

“Although we only quantitatively sampled pads where we saw significant enhancement above the background, it is important to note that we could detect little to no emission from many other pads, particularly in the region north of the OSA, from Washington north to Pittsburgh.  Thus, we do not intend for our regional flux estimate to be taken as necessarily representative of the Marcellus as a whole but only for the region defined as the OSA for these days.”  (p. 3; emphasis added).

In Washington and Allegheny Counties – which include Washington and Pittsburgh, respectively – there are over 800 wells. Yet, the researchers detected little to no emissions from any of those, save for just seven well pads. Undoubtedly, leading with that fact would not have furthered the notion that methane leaks are out of control, so it was buried.

Fact #2: Ingraffea and Howarth use modeling exercises instead of direct measurements

One of the biggest flaws of the report is that it relies on data gathered in just four flights over only two days in southwestern Pennsylvania.  Here’s how the researchers characterize their method:

“Here we use an aircraft-based approach that enables sampling of methane emissions between the regional and component level scales and can identify plumes from single well pads, groups of well pads, and larger regional scales, giving more information as to the specific CH4 emission sources.  We implemented three types of flights over 2 d in June 2012 […] We show that the methane emission flux from the drilling phase of operation can be 2 to 3 orders of magnitude greater than inventory estimates, providing an example and improved understanding of the differences between observed data and bottom-up inventories.”  (p. 2)

In other words, the researchers are capturing methane data, and then running modeling exercises to determine the source. That’s a well-understood method of gathering emissions data, but as the report essentially admits, it is also rife with uncertainty:

“In the region between Washington, PA and south to the border of WV we observed multiple high concentration methane plumes and investigated areas where initial observations revealed well pads with potentially high methane emission rates.  The high density of pads in the region and the prevailing wind direction (SW) during the time of measurement combined to make plume attribution to single pads difficult. (p. 3; emphasis added).

Contrast this methodology with direct measurements, which have shown that methane emissions are far below what alarmists have claimed.  The University of Texas/Environmental Defense Fund study from last fall – which took the first direct measurements of methane from production sites – looked at 190 natural gas production sites, finding that total annual methane emissions are “comparable” to EPA’s estimates.  It suggests a leakage rate of only about 1.5 percent, which is far below the threshold where natural gas would presumably lose its greenhouse gas benefits.

So, essentially, we are comparing direct measurements at the source of nearly 200 wells (UT/EDF, 2013) against a flyover of a handful of wells in one section of one state (Ingraffea/Howarth, 2014) – and most of the wells examined in the latter, it’s worth repeating, revealed little to no emissions, although they were largely ignored.

Fact #3: Ingraffea blasted previous research based on small sample size, but his latest work examined far fewer sites

When the University of Texas last year released what has been called the most comprehensive study on methane to date, Ingraffea said it was “fatally flawed” because it was “based on a small sampling of hydraulically fractured wells which may not adequately represent national oil and gas activity and the variability within and across production basins.”

Yet, this week Ingraffea seems to have no problem releasing a report that studies only seven well pads in one particular region of Pennsylvania.

The news has extrapolated broader meaning (i.e. questioning EPA’s entire Greenhouse Gas Inventory) from Ingraffea’s latest work on methane. But if a study that looked at far more wells was “fatally flawed” based on its sample size, will he be contacting the newspapers and asking for corrections? Something tells us he and his fellow activists won’t.

Fact #4: Funding is provided by the organization that funds Howarth’s Cornell professorship

The acknowledgements section of the report states that funding was provided by the David R. Atkinson Center for a Sustainable Future at Cornell University. Coincidentally, Robert Howarth is “The David R. Atkinson Professor of Ecology and Environmental Biology,” and the Center has long been a supporter of Howarth’s research: For example, they have partnered with the well-known anti-fracking Park Foundation to fund Howarth’s many papers, which also were attacking hydraulic fracturing.

Of course, just because researchers received funding from a particular source does not automatically disqualify their research. But when the funding comes from an organization that also funds one of the most outspoken anti-fracking researchers of the day, it does give one pause.  Also of note, while Ingraffea complained loudly that the UT study received some funding from industry, he doesn’t seem too worried about the affiliations of the funders of his own report.

Fact #5: As natural gas production has gone up, methane emissions have fallen dramatically

According to the researchers, “The identification and quantification of methane emissions from natural gas production has become increasingly important owing to the increase in the natural gas component of the energy sector” (p. 1).

As noted above, using direct measurements, the University of Texas/Environmental Defense Fund study found methane leakage rates that were “comparable” to EPA’s estimates from last year.  In last year’s report, EPA found that methane emissions from natural gas systems had fallen 10.2 percent since 1990, and emissions from field production had fallen 38 percent since 2006.  EPA’s latest Greenhouse Gas Inventory shows that emissions fell 16.9 percent since 1990, with field production emissions falling more than 40 percent since 2006. From 2011 to 2012 (the most recent year for which data were available), methane emissions from natural gas systems declined by 12 percent.

Granted, the “news hook” of this latest research is to suggest that the EPA’s estimates are too low. As such, the EPA’s GHG Inventory cannot in and of itself refute this paper’s allegations. But given the enormous quantity of data that went into EPA’s inventory, versus the handful of sites in this latest paper, it’s worth asking: Can we legitimately question EPA’s data on methane based on outliers, especially when there is plenty of evidence to suggest that EPA’s methane measurements are actually too high?

The timing here is interesting too.  The U.N. Intergovernmental Panel on Climate Change (IPCC) recently released its latest report, which specifically credits fracking and natural gas with reducing greenhouse gas emissions:

“A key development since AR4 is the rapid deployment of hydraulic fracturing and horizontal drilling technologies, which has increased and diversified the gas supply… this is an important reason for a reduction of GHG emissions in the United States.” (p. 18)

Which leads to the question: whom should we trust?  University researchers who take direct measurements form 190 natural gas sites, the U.S. EPA, and the world’s most prominent climate scientists?  Or researchers whose claim to fame has been their advocacy against the very process they are researching?