UPDATE (8/6/2013; 3:03pm ET): Energy and climate expert Michael Levi has posted a series of thoughts and concerns about the latest NOAA paper. Among the issues identified: the researchers “only have three hours of observations, and no direct way of knowing whether those observations are representative over longer periods of time.” Levi explains why the sample date itself is problematic:

“The paper argues that well spudding and completion activity during the week surrounding its observations is normal. What it fails to mention is that well spudding and completion activity on the day of its observations is more than two times the normal level (all data obtained from the Utah DNR). If anyone wants to claim that fracking itself (rather than activity surrounding oil and gas production in general) is generating the observed methane leaks in Uintah basin, they need to contend with and account for this fact.” (emphasis added)

Levi does believe the paper provides value in understanding methane emissions, although he does appear to recommend a different interpretation of the data to more accurately reflect what those emissions really are (namely, lower than what the researchers claim). He also recommends we “avoid hasty extrapolation” of the findings to other fields, owing to their extremely limited nature.

—Original post, August 5, 2013—

Last year, researchers from the National Oceanic and Atmospheric Administration (NOAA) released a controversial – and flawed – paper alleging very high methane leakage rates from a natural gas field in Colorado. The story unfolded as you might imagine: The press had a field day with it, the activist community blogged about it, and our favorite anti-fracking activist Tony Ingraffea still cites it as if it presented air tight scientific facts about methane leakage rates everywhere, rather than just one location.

(Ingraffea’s own research on methane leaks, by the way, has been almost universally cast aside by the scientific community as “biased,” agenda-driven, “flawed,” and just flat out “wrong” – even by research funded by the Sierra Club!)

Nonetheless, as we’ve seen time and again with these types of reports – and naturally after folks had worked themselves into an adequate frenzy – it was calmly and carefully reviewed and challenged shortly after its publication. For example, Michael Levi, an energy and climate expert at the Council on Foreign Relations, called the findings “unsupportable.”

Now the NOAA team is back, this time in Utah’s Uinta Basin. After scoring some early press coverage from Nature back in January, based on a Power Point presentation one of the authors gave at a conference, the abstract was published online this week by Geophysical Research Letters.

But this time, it wasn’t just Michael Levi of CFR raising concerns (although Levi had criticized Nature for pretending a conference presentation is equivalent to a peer reviewed paper). The Environmental Defense Fund wrote back in January about the second NOAA study’s limitations, explaining that “neither [of the NOAA studies] is a systematic measurement across geographies.” Steven Hamburg from EDF added:

 For this reason, conclusions should not be drawn about total leakage based on these preliminary, localized reports. Drawing conclusions from such results would be like trying to draw an elephant after touching two small sections of the animal’s skin: the picture is unlikely to be accurate. (italics included in original)

In fact, Hamburg once again sounded a note of caution today after reviewing the study, even as he cited EDF’s ongoing studies into fugitive methane and advocated for tighter emissions controls:

Findings are based on readings from airplane flights that measured methane in the air on a single day and estimated the proportion of those emissions that came from the oil and gas infrastructure —production, gathering systems, processing and transmission of the gas out of the region…

Emissions from gathering, processing, and other various ancillary activities taking place in the basin were also captured in the overflight, with no way to attribute the high emissions among these various elements of the natural gas supply chain.

In other words, there is as of yet no data to tell us whether the emissions are coming from production, gathering, processing or other activities … More investigative work is needed before we can claim to understand what is driving these apparently large emissions.

You read that right. EDF has already reviewed the latest NOAA paper and says the conclusion – an emissions rate of between 6.2% and 11.7% of average hourly gas production in Uintah County during February 2012 – is based on single day’s data, with no way to measure where the methane came from. But according to PoliticoPro (sub req’d), the authors are already anticipating even more criticism:

One criticism, [lead researcher Colm Sweeney] acknowledged, was the short duration of the study.

Scientists flew over the region on 12 different days in February 2012, but the paper only uses data from one day, noting that “non-ideal meteorological conditions (in particular, low, variable and sometimes recirculating winds…) on the eleven other flight days made direct mass-balance analysis of CH4 emissions impossible.”

That means that the paper’s conclusions are based solely on data collected during a Feb. 3, 2012, flyover, although Sweeney said the calculations collected were “incredibly accurate and precise.”

So, according to the researchers, after throwing out the data from 11 days of flights because of “impossible” conditions, the flight on the 12^th day was “incredibly accurate and precise.” And the paper’s findings rest entirely on one day of data. Yes, we can see why the authors would anticipate some criticism on that point, especially when you consider that we are talking about just one day of data from just one gas field, combined with some estimates from researchers whose methodologies – past and present – have been called into question.

The anti-industry activists will no doubt try to distort one day of data from one gas field into a year-round indictment against every gas field in the United States, even though the only solid conclusion reached by the authors is “the need for further atmospheric measurements to determine the representativeness of our single-day estimate.”

Activists are forced (and will be forced in the future) to rely on distortion because the wider story about methane emissions is actually very good. Since 1990, natural gas production in the United States has skyrocketed, with an increasingly large share coming from deep shale formations. But according to the U.S. EPA, methane emissions from natural gas systems have actually fallen by 11 percent over that same time period. Also according to EPA data, the methane “leakage” rate from natural gas systems is about 1.5 percent – well below the threshold that experts say allows natural gas to maintain its climate advantage. And some experts, while acknowledging the need to reduce methane leakage as much as possible, have even stated that the methane “leak” issue could actually be overblown, especially in the discussion about climate change.

Some will see the publication of NOAA’s latest research as a reason to gin up fear about methane leaks from natural gas (and probably another excuse to write a story with “fracking” in the headline). But those of us who actually pay attention to details, data, and the growing consensus within the scientific community know better.