This piece was originally published in the July/August 2019 issue of electroindustry.

Maria Karpman, BEMP, CEM, LEED, AP Principal, Karpman Consulting, and  Michael Rosenberg, FASHRAE, CEM, LEED AP; Chief Scientist, Pacific Northwest National Laboratory

Buildings are complex systems composed of numerous interacting components that are influenced by external factors such as weather and occupant behavior. Building energy modeling (BEM) tools use physics-based equations to calculate building energy use at hourly or sub-hourly timesteps. Typical applications of energy modeling include optimizing building designs, documenting compliance with energy codes, demonstrating above-code performance for programs such as LEED, and evaluating the cost- effectiveness of building retrofits. Millions of dollars in utility program incentives are awarded to projects  based on their modeled performance. The American Institute of Architects (AIA) emphasizes the role of energy modeling for achieving carbon-neutral buildings: “Our numbers continue to demonstrate that energy modeling is an essential component of success” (AIA’s 2030 by the Numbers—2016 summary).

Studies have shown that modeled energy use often deviates significantly from measured use, bringing into question the feasibility of relying on energy models for decision-making. For example, on projects that participated in a large-scale modeling-based incentive program, the projected savings were within 25 percent of the measured savings for only 39 percent of projects¹ (Figure 1). So, what makes the models non-predictive?

Figure 1: Modeled versus Realized Savings for Retrofit Projects

UNREALISTIC EXPECTATIONS

Modeling may be used to compare efficiency options, comply with code or above-code programs, or predict future building performance.² Depending on the modeling application, different levels of modeling resources are required with different areas of focus. When models are created to compare design alternatives, such as two competing HVAC system types, many aspects of the design may still be in flux. While it is imperative to get the system details and their differences characterized in the model, using Standard operational and plug-load  assumptions may give a reasonable result at a low cost to the building owner.

Modeling protocols used for code compliance and LEED certification include rules that can conflict with the actual anticipated operation. For example, California’s energy code requires that default receptacle loads and schedules of services are used, regardless of the plan for the specific building. ASHRAE  Standard 90.1 modeling rules for demonstrating code compliance and beyond-code performance require that cooling is modeled in all conditioned spaces whether  or not it is specified and that fans run continuously while the building is occupied. These rules practically guarantee deviation from post-construction utility bills but serve a useful compliance purpose.

Sometimes modelers are specifically tasked with creating models that project future energy use. In those cases, the modeler needs to gather detailed information on how the building will be operated, interview future occupants regarding their room schedules and equipment, and visit similar structures. On retrofit projects, comprehensive site measurements are required. These tasks can be expensive and are not necessary for every modeling project.

MODELER EXPERTISE AND PROJECT KNOWLEDGE

Professionals with no specialized training often develop commercial energy models and learn the craft as they go. Modeling budgets and schedules are typically tight, and the work often goes to the lowest bidder. The Building Energy Modeling Innovation Summit³ cited the difference in the results obtained by different modelers simulating the same building in the same tool as one of the top issues affecting BEM credibility.

Modelers are often disconnected from the design team and are not fully aware of changes made to the building design. On retrofit projects, the impactful site conditions may not be communicated to the modeler and thus not captured. Errors can be reduced by requiring that modelers have specialized certifications (such as BEMP or BESA) and by rigorous quality control of modeling-based submittals. (Currently, the review time varies dramatically from program to program, from less than one hour to well over 40 hours per project.)

DIFFERENCE BETWEEN “IDEAL” AND ACTUAL OPERATION

Models often assume that building systems and controls operate as specified. However,  this is rarely the case. Building controls are frequently not commissioned, and thus not configured to operate as intended by the code.⁴ Economizers were shown to save 6 percent to 32 percent of cooling energy use depending on climate, but they malfunction in more than 70 percent of the installations.⁵ Thus, models reflecting correct economizer operation substantially underestimate cooling energy use on many projects.

Further, building operators may disable or override energy savings features. These problems are best solved not by the modeler or modeling software but by better building commissioning and training of building operators.

IMPACT OF OCCUPANT BEHAVIOR, DEMOGRAPHICS, WEATHER, AND OCCUPANT-INSTALLED  EQUIPMENT

Building occupants can operate a building in a variety of ways; while estimating these parameters is possible, reliably predicting future behavior is not.

• Temperature setpoints, hours of occupancy, opening and closing windows, and shades all impact energy use.
• Service hot water usage differs by a factor of four or more depending on whether an apartment is occupied by seniors or a family with
• Weather used in models represents a typical year and is different from actual post-occupancy or post- retrofit conditions.⁶
• Buildings have systems and equipment installed by occupants such as office computers, kitchen appliances, task lighting, and industrial equipment. The energy use of these systems is difficult to predict. In one study, office equipment with a 3.5 W/SF load based on the nameplate rating consumed around W/SF based on field measurements⁷; heat gains from desktop computers may differ by a factor of three depending on the manufacturer, processor speed, and RAM.⁸

For these reasons, incentive programs that rely on post-construction energy use to confirm modeled savings estimates recognize that accurate prediction is impossible, and often include a post-occupancy calibration step to verify operation and adjust for installed equipment and actual weather conditions.

LIMITATIONS OF BEM TOOLS

It is well documented that simulation results can vary significantly depending on the BEM tool used. Studies by Lawrence Berkeley National Laboratory and Texas A&M University showed heating energy differences of over 100 percent and 27 percent, respectively, for the same building modeled in commonly used simulation tools.⁹, ¹⁰ A study by Gard Analytics showed a 50 percent difference in annual cooling load between two tools when running a Standardized test by ASHRAE Standard 140.

Furthermore, most BEM tools support multiple methods for calculating common conditions and technologies, such as infiltration or daylighting, and energy use projected by the same BEM tool for a given project may vary significantly depending on the method that modeler picks. There is little or no evidence of which results are correct because the peer-reviewed comparative testing of BEM tools is limited, and validation using the actual performance data is scarce. ASHRAE Standard 140¹¹ currently covers only a small subset of common building systems and provides no formal pass/fail criteria. The challenges are exacerbated by the diversity of commercial building designs, the rapid development of new systems and technologies, and the complexity of the underlying physics.

Implications

Building models rely on numerous assumptions, such as operating hours, occupant demographics, lighting and equipment use schedules, and weather. They often reflect the ideal operation of building systems and are affected by limitations of BEM tools, modeler expertise, etc. Even when the “right answer” (the actual energy use of the existing building) is known upfront, bringing a model within 5 percent of the utility bills, as required by ASHRAE Guideline 14, is a laborious process that does not guarantee that the model is representative of the actual building. Models are underdetermined systems, and multiple combinations of inputs may produce the same annual use.

Programs that set a fixed modeled target (e.g., target energy use intensity) to qualify projects for incentives or demonstrate compliance with code must prescribe unregulated loads and operating conditions and require that all projects use the same BEM tool, similar to the passive house (PHIUS and PHI) modeling protocols, or allow only the tools that pass rigorous sensitivity testing to ensure close alignment in results. These requirements will cause modeled energy use to vary even more compared to actual energy use than if anticipated unregulated loads and occupancy conditions are allowed.

Programs and approaches that are based on comparing two models (baseline and proposed) are less affected  by the differences between BEM tools because most simulation inputs are the same between the models, and if one of the common inputs is wrong, it will likely not have a significant impact on the outcome.6 For example, savings from lighting fixture replacement largely depend on the change in fixture wattage and modeled runtime. Comparative approaches that are based on percent improvement between two models (ratio instead of delta) reduce the impact of operational uncertainty—e.g., lighting runtime hours are canceled out to a degree when savings from a lighting retrofit  are expressed as a percentage reduction in building energy use.

Conclusions

Models don’t necessarily need to accurately predict future building energy use to be useful for comparisons and compliance. When it is the desired outcome, a more accurate prediction is possible, but it requires more resources and will always be confounded by the inability to accurately predict human behavior and weather. This is not unlike fuel efficiency and appliance ratings that are widely used in the marketplace (Figure 2) and are considered of great value. Expecting models to accurately predict the future is asking for the impossible. ei

----------------------

1. Chris DeAlmagro and Maria Karpman, “Comparison of Projected to Realized Savings for Projects that Participated in a Modeling-Based Incentive Program” (presentation, 2017 ASHRAE Building Performance Analysis Conference, Atlanta, GA, September 27-29, 2017)
2. Building Energy Modeling for Owners and Managers, Rocky Mountain Institute, August 30, 2013, https://rmi.org/wp-content/uploads/2017/05/Building-Energy- Modeling-for-Owners-and-Managers-2013.pdf
3. Building Energy Modeling Innovation Summit, Rocky Mountain Institute
4. Michael Rosenberg et , Implementation of Energy Code Controls Requirements in New Commercial Buildings, PNNL-26348, Pacific Northwest National Laboratory, March 2017
5. HVAC Economizers 101, Battelle, https://buildingretuning.pnnl.gov/training/economizers/PNWD-SA-8511%20HVAC%20Economizers%20101-Section%203.pdf
6. The amount of service hot water used in an apartment building varies from 12 to 44 gallons/day per person depending on the occupant demographics based on the ASHRAE Applications Handbook
7. 2017 ASHRAE Handbook—Fundamentals (p.18.13, Figure 4)
8. 2017 ASHRAE Handbook—Fundamentals (p.18.12, Table 8A)
9. Joe Huang et , Evaluating the Use of EnergyPlus for 2008 Nonresidential Standards Development, LBNL-58841, Lawrence Berkeley National Laboratory
10. Simge Andolsun, Charles H. Culp, and Jeff Haberl, EnergyPlus vs DOE-2: The Effect of Ground Coupling on Heating and Cooling Energy Consumption of a Slab-on-Grade Code House in a Cold Climate, 2010, Fourth National IBPSA-USA Conference, New York, USA, http://repository.tamu.edu/handle/1969.1/93374
11. ANSI/ASHRAE Standard 140-2017 Standard Method of Test for the Evaluation of Building Energy Analysis Computer Programs
12. Michael Rosenberg et al., Roadmap for the Future of Commercial Energy Codes (Figure 2.6), PNNL-24009, Pacific Northwest National Laboratory, January 2015, https://www. pnnl.gov/main/publications/external/technical_reports/PNNL-24009.pdf