Bloomberg’s Misleading Hydraulic Fracturing Poll
Tuesday, December 18th, 2012 | 7 Comments | Tagged in: Bloomberg, Energy In Depth, Hydraulic fracturing, natural gas, oil, Poll, regulation, shale, Water
Simon
Research Director
Measuring public opinion can be a tricky task. The answers you get really depend on who you ask, and how you ask the question, and – less appreciated – where in the interviewing process you ask the question. Ask the wrong group of people a question, or ask the question in the wrong way, or ask the question after a series of other questions, and the results of your public opinion poll won’t accurately measure public opinion.
Unfortunately, a recent Bloomberg National Poll that found an increase in public support for more regulation on hydraulic fracturing appears to have made all three mistakes – asking a question of the wrong group of people, asking it in the wrong way, and asking it after a series of other questions that may have affected the results. As a result, this poll doesn’t add any substance to the debate over hydraulic fracturing, and is actually quite misleading.
The Dec. 13 Bloomberg poll found 66% of the American public wants “more” regulation of hydraulic fracturing – the technology that makes the production of oil and natural gas from deep shale formations possible. That’s an increase of 10 percent since September. There were more than 20 questions in the survey, which took in a wide range of subjects, including the meaning of President Obama’s election win, why Grover Norquist’s Taxpayer Protection Pledge makes Republican lawmakers resistant to raising taxes, and which Democrats would make good presidential candidates in 2016. To conduct the poll, Bloomberg surveyed 1,000 people randomly selected across the country.
Mistake #1 – Asking a national audience about state-by-state issues
It’s understandable that when fielding a national poll, Bloomberg would want to add a question about hydraulic fracturing – in some eastern U.S. states, it’s a topic that’s in the news a good bit. But sampling a group of 1,000 randomly selected adults from all across the country doesn’t provide a meaningful measure of public sentiment toward hydraulic fracturing and the American energy production it makes possible.
In recent years, opponents of the oil and gas industry have intensely lobbied the news media to nationalize the debate over shale development to support a political outcome – the federal government taking regulation of fracturing technology away from the states. It hasn’t worked, and states remain the primary regulators, as they have been since fracturing was first pioneered in the 1940s. As a result, the debate over hydraulic fracturing is also very state-centric, and this issue isn’t covered or talked about equally across the country. In some states, it’s a hot topic, and in others, it’s barely discussed at all. In Florida, few have probably even heard of the thing.
Rather than ask 1,000 randomly selected adults from across the country about hydraulic fracturing, a more relevant exercise would have been polling in states where tight oil and gas development is actually relevant. Had Bloomberg done this, they may have found the public more evenly divided on the question. For example, recent polls from New York have found much narrower margins on questions tied to hydraulic fracturing – 42%-38% support (Siena College Poll), and 44%-42% support (Quinnipiac Poll).
Again, it’s understandable that Bloomberg wanted to write a story about what “the American public” thinks about hydraulic fracturing. But besides generating some (of its own) headlines, polling a national audience fails to provide any meaningful information about how public opinion in individual states may influence regulation in those states. Just as national opinion polls don’t show who’s winning the battleground states during a presidential election, a national poll about hydraulic fracturing tells you nothing about how the issue is being handled state by state.
Mistake #2 – Asking a clearly biased question
The National Council on Public Polls has the following warning for reporters who write about poll numbers:
“You must find out the exact wording of the poll questions. Why? Because the very wording of questions can make major differences in the results. Perhaps the best test of any poll question is your reaction to it. On the face of it, does the question seem fair and unbiased? Does it present a balanced set of choices?”
In other words, how you ask the question can influence the answer you get. Let’s take a look at the question in Bloomberg’s poll:
“A process known as ‘hydraulic fracturing’ or ‘fracking’ involves injecting liquids into the ground. It has resulted in a significant increase in production of natural gas, accompanied by a steep drop in its price. Critics have said it is linked to tainted water supplies and earthquakes. Based just on what you know, do you think there needs to be more regulation or less regulation of fracking?”
This question is hardly neutral. It starts with a poor explanation of hydraulic fracturing, and follows with a description of natural gas production and price trends that’s devoid of any context about job creation and economic growth – topics that most Americans care deeply about. Then the question quotes misinformation from oil and gas critics, and doesn’t even mention what supporters of hydraulic fracturing say. The net result is a question that invites someone to express support for more regulation, because if they don’t, they’re effectively expressing support for “tainted water supplies and earthquakes.”
Bloomberg’s pollsters should rewrite this question to be fair to all sides of the debate. As a starting point, they need look no further than the news copy from the Bloomberg reporter who wrote up the results of the survey:
“Industry groups for chemical, fertilizer and steel companies are trumpeting gains for the U.S. economy as natural gas supplies become more accessible. Low-cost natural gas could generate $72 billion in capital investment as petrochemical companies relocate or boost investments in the U.S., according to the American Chemistry Council.
‘This trend is vital to America’s prosperity,’ Rayola Dougher, chief economist at the American Petroleum Institute, told reporters yesterday. ‘However, costly or duplicative regulation of hydraulic fracturing could be incredibly’ harmful, she said.
And while it’s perfectly reasonable to reference environmental concerns in a question about oil and gas development – or any commercial activity, for that matter – the question shouldn’t simply parrot alarmist and ideologically motivated allegations from anti-industry activists. On “tainted water supplies,” Bloomberg’s pollsters should review what environmental regulators have been saying for years, and the expert conclusion of Stanford University geophysicist Mark Zoback, an advisor to U.S. Secretary Steven Chu on hydraulic fracturing:
“…the injection is typically done at depths of around 6,000 to 7,000 feet and drinking water is usually pumped from shallow aquifers, no more than one or two hundred feet below the surface. Fracturing fluids have not contaminated any water supply and with that much distance to an aquifer, it is very unlikely they could.”
As for earthquakes, Bloomberg’s pollsters should also read the National Research Council’s recent study, which concludes “[t]he process of hydraulic fracturing a well as presently implemented for shale gas recovery does not pose a high risk for inducing felt seismic events.” And they should talk to U.S. Geological Survey geophysicist Bill Ellsworth, who set the record straight after his research was misrepresented by the news media: “We don’t see any connection between fracking and earthquakes of any concern to society.”
Finally, notice that Bloomberg’s question said nothing about the current state of regulation. So when they reference “more regulation or less regulation” … that’s compared to what, exactly? The question provides no information on what regulations are currently in place. For all the respondents know, hydraulic fracturing could be unregulated altogether. This is especially possible among those respondents in areas where “hydraulic fracturing” is rarely if ever discussed (see Mistake #1).
This is actually very important, because despite Bloomberg’s lack of context, the reality is that shale development is heavily regulated. For example, here’s how the U.S. Department of Energy and state-led Ground Water Protection Council summarized it:
“A series of federal laws governs most environmental aspects of shale gas development. For example, the Clean Water Act regulates surface discharges of water associated with shale gas drilling and production, as well as storm water runoff from production sites. The Safe Drinking Water Act regulates the underground injection of fluids from shale gas activities. The Clean Air Act limits air emissions from engines, gas processing equipment, and other sources associated with drilling and production. The National Environmental Policy Act (NEPA) requires that exploration and production on federal lands be thoroughly analyzed for environmental impacts.”
And that doesn’t even include all of the strong state regulations in place, many of which have been updated since the above-referenced report was published.
Would the respondents have given the same answer on “more regulation or less regulation” if they were presented with even a paraphrasing of what’s listed here? How can you ask someone to register an informed opinion about whether hydraulic fracturing needs additional regulation when you don’t even provide basic baseline information?
Mistake #3 – “Priming” the respondents
When conducting a public opinion poll, there’s another concern about how you ask the question: where it appears in the survey relative to other questions. According to the American Association for Public Opinion Research:
“In some surveys, the order of the questions may be designed to ‘lead’ the respondent to a kind of conclusion that produces a predictable response. This form of bias would not have been present if the prior questions had not been asked. This is also referred to as setting up a ‘context effect.’”
Among pollsters, this is commonly called “priming.” A respondent’s answer to a question can be influenced by the question asked immediately beforehand, because the earlier question “primed” them to think about a particular topic.
In the Bloomberg survey, respondents to the hydraulic fracturing question were primed not just with a question about environmentalism, but a question about one of the biggest environmental issues of the past 20 years:
“Do you believe the temperature of the Earth is or is not warming because of human activity?”
Anyone who has followed the debate over global warming knows that asking someone whether they blame human activity for rising temperatures is just like asking: “Do you care about the environment, or not?” Asking a question like this immediately before the hydraulic fracturing question primes the respondents to give an answer that’s perceived as environmentally conscious. Coupled with the fact that no information on current regulations was provided, it’s easy to see how respondents were primed for a particular answer.
Conclusion
None of this is to suggest that Bloomberg deliberately misled the public. As described above, it’s understandable why a national news outlet would try to gauge national opinions on a subject like hydraulic fracturing. But the combined effect of a biased question and a primed audience clearly steered the respondents towards the “more regulation” answer.
Tags: Bloomberg, Energy In Depth, Hydraulic fracturing, natural gas, oil, Poll, regulation, shale, Water
7 Responses
-
Lee James says:
I can accept this survey as-is. There will be additonal surveys… and, yes, you are correct in saying that you need to see how the quetion is phrased, etc. in order to interpret the results.
-
I’m sorry but you simply haven’t made a good enough case that Bloomberg poll intentionally misled the public. For example, concerning mistake #1, who says that the question about hydraulic fracturing is a question best left to run a state-by-state basis? There are many states where this form of drilling is being used so it really is a national issue, especially since the federal EPA has the right to regulate any industry that affects groundwater quality. With mistake #2, it’s not the pollsters fault but rather that of the journalists report on the poll if they don’t find out the exact wording used. Blame them, not Bloomberg.
Having said all that, I will say that 1000 people is hardly a large enough sample to do any type of conclusion on a polling question. For all we know, the pollster only asked men or women or Democrats or Republicans the questions used in the poll, a point any journalists who wrote about this should have seized upon right away when covering the poll’s results.
-
Thanks for commenting. As explained in the post, our goal was not to show that Bloomberg “intentionally misled the public.” Rather, it was to show the problems with the poll itself, and how the methodology impacted the results. Also, please note that in the last paragraph we state clearly: “None of this is to suggest that Bloomberg deliberately misled the public.”
Regarding your critique of Mistake #1, whether one wants hydraulic fracturing to be regulated in Washington, DC or not, the reality of the current regulatory regime is not up for debate: individual states are in charge (although multiple federal laws do apply to shale development). Given that fact, it is perfectly legitimate to raise concerns about a question asking for attitudes on regulation, especially when the question itself makes no mention of what regulatory system is already in place!
Once again, thanks for your reply.
-
Fritz Buzzard says:
You folks sound really frightened, as opposed to worried or concerned. I live in West Virginia, and I can tell you that the backlash is growing. What the timber and coal companies have started (i.e. destroying the state’s ecology) the fracking industry seems determined to finish. What was a beautiful state is rapidly being ruined by wellheads and pipelines as far as the eye can see.
Now that the sweepstakes mentality has largely faded, people are starting to look more closely at what they got for their money. Crime is up, helped along by large numbers of transient workers, air quality is no better and is probably getting worse, as a number of doctors have stated. Respiratory disease is way up. And in general a lot of the landscape just looks like hell.
Personally, I can’t wait to see “Promised Land”. Judging from your massive over-reaction and attempts at damage control, the movie is probably right on the money.
We have eyes – we can see what damage has been done. We have ears also – we are hearing lots of disappointment and regret from those who, willingly or unwillingly, drank the “Promised Land” kool aid
I’m guessing that, for the fracking industry, the party is just about over.
-
Eric says:
Wow, so watching a documentary of people all over the country in fracking areas lighting tap water on fire, dead wildlife in and near streams, chemicals in their blood, cancer, and permanent brain damage mean absolutely NOTHING. It all boils down to how you word poll questions? It amazes me just how a pile of little green pieces of paper in some way or another is worth more than the lives of our families. People REALLY need to do the right thing, not just make as much money as they can as fast as they can at any expense.
-
Hi Eric, thanks for your comment.
Since you did not provide any specific information on the documentary you were referencing, I can only assume you were talking about Gasland. You should read EID’s full debunking of that movie, which was full of inaccuracies, distortions, and outright falsehoods — http://www.energyindepth.org/debunking-gasland/
For just one example, the “lighting tap water on fire” scene was investigated by Colorado regulators, who determined it was “not related to oil and gas activity.” Instead, it was a result of naturally occurring methane, a phenomenon well-known in that area, as residents have been able to light their tap water on fire long before any hydraulic fracturing was occurring. This is also true in many parts of Pennsylvania, Texas, and indeed all across the country. You might also be interested to know that those same Colorado regulators asked Gasland’s producer, Josh Fox, if they could appear in his movie and explain that issue in more detail. Fox wouldn’t allow it. I wonder why?
As for the cancer claim, you should read the Associated Press investigative report on the subject. What they discovered might surprise you — http://bigstory.ap.org/article/experts-some-fracking-critics-use-bad-science
In any event, thanks again for your response.
Wow. You guys are really grasping at straws now.
It depends what you ask, who you ask, and when you ask it, huh?
Or maybe you just cant accept the fact that the public is ever increasingly onto what the gas industry is REALLY doing to their quality of life. After all, the smoke fades and the mirrors end up breaking after a while…
The people are getting it. You can try to spin things all you want, its really the only thing the industry is exceptionally good at.
We’re watching.