Our study of 6,917 homes across Canada

"Can we determine the EnerGuide score & GHG emissions of a home using only public data?"

We spent most of 2022 answering this question. We realized that targeting and qualification are critical missing pieces when it comes to home energy upgrades.

Properate proudly presents its whitepaper, which compares the accuracy of Properate with traditional on-site energy ratings in a study of 6,917 buildings across Canada. The results show an average weighted accuracy of 92%, demonstrating the potential for community-scale home deep energy retrofits and the advantages of Properate in achieving that goal.

Background

The challenge of upgrading energy efficiency in low-rise residential buildings is present from the energy rating stage, the first step of an energy upgrade. The typical approach of on-site rating cannot cover all buildings in large communities. To address this, two new rating options have been proposed: Virtual Rating, which can be done using public data, and Remote Rating, which requires homeowner engagement. The remote rating process can be done in a DIY fashion, or professionally, and involves a questionnaire or pictures of the home. To increase efficiency, virtual and remote ratings should be connected and combined with on-site assessments. The methodology and scope of virtual and remote ratings must be clear and comprehensive to be as effective as on-site assessments.

Maps: Shows the Energy Rating of every home in a neighborhood. If Remote Rating or On-Site Rating data is not available for a home, Virtual Rating results are displayed for that home.
Wizard: A questionnaire interface for Remote Rating. It can be operated by homeowners in a DIY fashion, or by professionals in a concierge fashion.
Studio: Made for professionals for on-site or remote energy modeling. Studio's interface allows for complete data collection from a building and the results provided have the highest fidelity.
API: Programmatic interfacing with the RBEST and estimators such as costing and GHG for third-parties. RBEST powers the core instant building energy simulation functionality of Properate, while other technologies prepare the inputs and outputs. The following section provides a use-case of how Properate orchestrates all these technologies in the context of community-level energy labeling.

Technology Review

To deliver Energy Rating, we have identified 4 key requirements that a Building Energy Modeling (BEM) software must deliver:

Reproducibility: Must output the same results for a set of input conditions

Expandability: Must be able to handle "unseen" buildings, especially for low-income housing

Auditability: Model outputs must be validatable at the component-level with a well-studied & common energy modeling tool

Lightness: Must be computationally light enough to be used for millions of homes and run periodically on generally accessible computer hardware.

In the whitepaper we discuss the approaches to energy rating modeling:

White Box Modeling is the current industry standard for building stock energy modeling, which is reliant on physics-based equations. The transparency of these models, however, comes with a high computational cost. EnergyPlus is one example of such a model, where load balancing calculations require iterative computation for each building component, leading to increased time complexity. This computational burden becomes a challenge for upgrade planning and remote rating at the community level. To resolve this, researchers have either scaled computation with supercomputers/cloud computing, simplified the use-case with archetypes, or used previously computed results with surrogate models.

On the other hand, Black Box Modeling uses statistics and the advancements of data science and machine learning to simplify building energy modeling architecture. This simplification, however, may come at a cost as these models often lack auditability, reproducibility, and expandability for remote rating. An example of using a regression model to infer the EnerGuide score of a home based on its heated area, highlights the limitations of black box models. Limitations such as the need for vast amounts of training data and the lack of transparency in the model's workings. In summary, while black box models provide lightness, they may not fully meet the requirements for remote rating.

Properate uses a Grey Box Modeling approach through our R-BEST (Rapid Building Energy Simulation Tool) which combines the advantages of both white-box and black-box models. R-BEST starts with a white-box approach to parse building components for simulation and then employs black-box modeling where a white-box model would have computed through iterative calculations. R-BEST can use a single-shot multivariate approach to “co-simulate” multiple building conditions or standards simultaneousl. R-BEST provides expandability and auditability of a white-box model while being much more resource-efficient. The biggest challenge in developing R-BEST was at the transition between white-box and black-box modules. R-BEST's design choice of validation after input data parsing makes the black-box module more robust but also creates challenges in understanding anomalies. The solution is an error reporting subroutine that translates R-BEST errors into user-understandable messages.

Methodology of the case-study

The data used to evaluate the buildings included building addresses, satellite imagery, property records, building massing information, fuel types, and upgrades.

In the study, we used a modified blind-study methodology, which means that the data and on-site evaluations were not examined by Properate prior to the study. The goal was to see how close Properate's simulations could match the on-site evaluations. We simulated the buildings and evaluated the accuracy of their results using three metrics: Mean Bias Error, Mean Absolute Error, and Normalized Root Mean Square Error. These metrics measure the difference between the on-site evaluations and Properate's simulations.

Finally,the metrics were normalized and absolute and weighted accuracy was calculated.

Results & Next Steps

The results showed an Absolute Accuracy of 94-95.5% and Weighted Accuracy of 91-92% even with missing data, indicating the flexibility of Properate. The results have limitations and may not be generalizable to all homes in the studied Climate Zones, but the analysis of the data and features suggest high confidence in the statistical significance of the results for the building types studied.

Following the completion of this study, there are several areas for further research to enhance and expand upon the findings. The study is currently the largest of its kind for remote energy ratings, but there is room to gather more data to make the results more representative across Canada. Additionally, it would be valuable to investigate using building digital twins with other co-simulators beyond EnerGuide to gain access to specialized energy modeling activities, such as mechanical system sizing and climate adaptation. Finally, it would be important to perform sensitivity analysis on inputs to understand the impact of inaccurate inputs on the accuracy of the results.

The results of the study demonstrate the potential for community-scale home deep energy retrofits and the benefits of using Properate in delivering that. We believe that this study will open up new avenues for further research and will help drive the industry towards more sustainable and efficient energy practices.

For those interested in reading more about the study, please click the button below.

If you have any media inquiries or would like to learn more about Properate, please reach out to Noura Seifelnasr at noura@properate.io.