Why You Can’t Always Trust Energy Efficiency Data

Share Button

The energy efficiency  industry is mad for data, especially anything that will give insight into what spurs consumers to save energy or buy green products. But unfortunately, not all data is good data and some leads the industry astray.

Take for example the oft-cited surveys that consumers are willing to pay more for green energy. This led several utilities to offer green premium programs where customers pay higher electric charges to support development of renewable energy. Some of these programs have been successful but others flopped with less than one percent of customers enrolling.

What went wrong? Probably more than one thing. But the problem may have started with how consumers responded to the initial surveys. It’s not that people lie; it’s just that we’re not good at predicting how we will behave when we move from speculation to action.

So explains Stephen Bickel, director of market research at D&R International. Bickel and colleagues have set out to bring more accurate data to the energy industry through a service called Better Data Better Design.

Surveys can fail because “people tend to give you the socially preferable answer and not necessarily the truth,” Bickel said in a recent interview.

Or in other cases consumers are simply confused. Take for example the discrepancy D&R International found when doing research on purchases of certain Energy Star appliances. Those who were surveyed indicated that they bought far more of the appliances than was possible – twice the number that were actually shipped.

Bickel believes this mathematically impossible response was the result of label confusion. Consumers may have been mixing up the ubiquitous Energy Guide Label   – which estimates an appliance’s impact on energy bills  – with the rarer Energy Star Label, an energy efficiency designation.

“So we are very leery of self-report survey data, and there is a lot of it out there still,” Bickel said. “And even if the sample sizes are big, if you are trying to use it to do something precise, it is often not accurate.”

‘Situational factors’ also create problems. Context and placement heavily influence our behavior –  far more than most of us realize. Give us a large plate and we serve ourselves more food than if we have a small plate. We double the amount of fruits and vegetables we buy if our grocery store cart is divided with half of the cart designated specifically for fruits and vegetables. (See work by Brian Wansink for more on “mindless eating.”)

Better Data Better Design marries this understanding of human behavior with knowledge of the market to improve studies, generate more accurate data, and provide deeper insight. The company uses data from billing records, meter readings, shipping reports and other reliable sources, and brings together utilities and others in collaboration to tap group ideas and resources.

The group offers some interesting insight for the energy efficiency industry in its 72-page “Residential Lighting Market Profile 2012.” Here are few takeaways, gleaned from successful programs.

  • We buy light bulbs using the automatic part of our brain, which relies on simple rules. If something is right in front of us and looks like a good deal, we’re more likely to bite. Walmart sold 137 million CFLs in 2007 by tapping into this reality. The success came when the store made such changes as placing CFLs at eye level and incandescents lower, or putting the CFLs in heavily trafficked parts of the store.
  • When a light burns out, 90 percent of us go first to our own closets (not the store) looking for a replacement. We tend to randomly select a bulb from the closet. So if the closet contains CFLs and incandescents, we might walk away with either. This says that it’s important for stores to offer what the report calls “grab-and-go multi-packs” of efficient light bulbs (and place them in visible locations like the check-out aisle.) If we buy a multi-pack of efficient light bulbs, we’re more likely to choose a CFL next time we reach into the closet.

It is clear based on these findings that the industry and regulators can significantly influence energy savings without wheedling and admonishing the consumer. Simple changes can make a big difference. We just need solid data to show us how.

Share Button

About Elisa Wood

Elisa Wood is the chief editor of MicrogridKnowledge.com. She has been writing about energy for more than three decades for top industry publications. Her work also has been picked up by CNN, the New York Times, Reuters, the Wall Street Journal Online and the Washington Post.


  1. Excellent article! Thanks for it. I tried to highlight here (http://smartenergyefficiency.eu/if-you-cant-measure-it-you-cant-manage-it-really) the idea that measurement is necessary but indeed not sufficient to implement successful energy conservation measures.

  2. I agree with the general thrust of the article, but I’d like to offer a different interpretation than “the problem may have started with how consumers responded to the initial surveys.” The broader problem is that the energy policy community has done a poor job of incorporating good social science into policy development and implementation. In the case of surveys, what the article attributes to consumers I attribute to the designers of the surveys. It’s obvious that bad surveys will produce bad data, but implementers rarely value excellence in survey design. Similarly, we know a fair amount about behavior generally, but we know little about how to apply those research concepts to energy efficiency implementation. What’s worse, we’re not investing significantly to figure it out.

    Bickel is a thought leader in incorporating social science into energy policy, and I applaud his work. More is needed.