In a recent report, the Environmental Defense Fund questions whether oil and natural gas producers are adequately reporting flaring activities to Texas authorities, but the report relies on data with significant limitations. As a result, it is worth reviewing this report with skepticism.

EDF published what it called a “new analysis” of satellite data, which the group claims to show flaring rates that are “two times higher than what industry reports to the Texas Railroad Commission,” the agency that regulates oil and natural gas production in Texas.

The data came from the National Oceanic and Atmospheric Administration’s (NOAA) Visible Infrared Imaging Radiometer Suite, although EDF provided no information on how it used the 2017 data to reach its conclusions. EDF linked to another analysis conducted by S&P Global, which “also shows significant discrepancies between industry-reported flaring data and satellite data in Texas.”

But multiple scientific studies raise legitimate concerns about the limitations of satellite data in measuring precise volumes of flared methane.

How Reliable Are the Data?

Both the S&P report and a 2015 NOAA-supported study note the accuracy of flare-volume estimates from NOAA satellite data is plus-or-minus 9.5 percent, a hefty margin of error. That does not completely undermine EDF’s analysis, but it does paint a far less definitive picture than some of the headlines about the EDF report would have you believe.

More alarming is that the NOAA-supported study starts with a significant limitation in using satellite data for this purpose:

“…satellite sensors are an attractive option for global monitoring of gas flares. However, none of the existing sensors have been designed specifically for the detection and monitoring of gas flares.” (emphasis added)

A study published by the National Academy of Sciences also raised questions about the reliability of satellite-based methane measurements, as compared to the ground-based “bottom-up” method, where measurements are taken onsite. The study found:

“Bottom-up methods provide information about the magnitudes and patterns of emissions from specific sources, and as a result they provide the type of information necessary to mitigate emissions.”

“…satellite-based retrievals will never be able to be as accurate or precise as ground-based in situ instruments, as the measurements are always affected by other confounding factors such as aerosols, which can never be fully eliminated with passive remote sensing.” (emphasis added)

EDF and S&P relied on a top-down approach to isolate data from subsets of the Permian basin and state-owned University Lands. This can be useful as part of a broader assessment, but it only gives us part of the picture. Getting an accurate measure of flaring rates – like any other methane measurement – would require a combination of top-down and bottom-up assessments.

Recurring Problem of Overestimating Methane

Last year, a study published by the Proceedings of the National Academy of Sciences raised serious questions about studies showing methane emissions were higher than previously thought. Those studies similarly relied on top-down measurements that failed to reconcile operational realities and onsite conditions, which in turn meant that the measurements only represented a snapshot in time. Researchers employing those top-down methods, however, had effectively extrapolated their findings to be indicative of operations around the clock. This has led to significant overestimates of methane emissions from oil and natural gas operations.

Curiously, EDF led one of the most prominent and recent examples of the kind of “top-down” studies that tend to overestimate methane emissions.

There are differences, of course, between measuring methane emissions and estimating flaring rates. But the consistency in how top-down analyses can exaggerate – and paint an incomplete picture of – these kinds of oilfield issues is notable.

Market Solutions Underway

None of this is to suggest that EDF’s use of top-down estimates is completely without merit. As stated above, these kinds of analyses can help us develop a more complete picture of industry emissions. But they cannot accurately convey the full picture, and as the PNAS study from last year confirmed, they can even miss important operational details and thus overestimate emissions.

Just last year, as EID noted at the time, the World Bank’s Global Gas Flaring Reduction Partnership reported the amount of natural gas flared in the United States had fallen 20 percent from 2015. In the Permian Basin specifically, more than 5 billion cubic feet per day (bcf/d) of natural gas pipeline capacity is currently either planned or under construction.

The annual flared volume of gas that EDF alleges in its new report represents less than 6 percent of the additional pipeline capacity coming online in the Permian over the next two years.

For these reasons, EDF’s analysis should be reviewed with some caution, as data limitations and ongoing market developments demonstrate the full picture is more complex.