Metrics are replacing Scores!

These are cool, and it may just to learn more about them. I don’t necessarily comprehend them fully, but I do appreciate some of the clarification, namely that runtime per degree day “lower is better”.

That being said, I’m a little confused by what might be causing my high score. More or less, all of my other heating scores are middle of the pack,but my runtime per degree day is 34 min which is very high apparently. What is driving that then? Does that mean it takes my system a long time to achieve it’s desires setpoint, or it must run a lot to maintain?

@coneslayer, @ezchief. This is fixed. Your profiles and metrics will update automatically sometime within the next week.

1 Like

It simply means that your heat runs more than others when adjusting for outdoor temperature. This particular metric is really basic…just a sum of the runtime divided by the sum of the degree days. Neither value looks particularly off. Maybe it’s just because your aux runtime is comparatively low? I don’t have that metric on here yet to check against.

Looks like it’s already showing updated values; these are about what I would expect. Thanks!

So, on the temperature profiles, is it safe to say that “good” is a mostly flat grey resist line, and very steep lines for heat and cool? Maybe the best is flat lines for all 3?

The changes look good. Nice work. The Resist score was interesting but is no longer visible. Just wondered if its coming back.

Thanks.

The new metrics are more aggressive about what a valid comparison is. This update requires that the system type and stages for both heat and cool match.

This seems too aggressive. In my case, I have a 1-stage boiler for heat but no cooling. It only shows 32 matching properties with “Global & type semi-detached” or with “Nearby & Type all”, and just 2 properties with “Global & Type very similar” or “Nearby & type semi-detached”!

(Also, I gather that if I manually change my system type in Beestat, the comparisons won’t be reflected until the metrics are recalculated next week? That’s not made clear.)

It makes sense that some stats are only meaningful compared to similar homes, but my home’s resist value when neither heat nor cool are running is purely a function of the house’s insulation; the heating or cooling system makes no difference.

I think the better fix is to match differently depending on what’s being compared:

  • Property and Resist: Match anything according to selected filters
  • Setpoint: If you have heating/cooling, match all other types of heating/cooling, respectively. If you don’t, the associated value wouldn’t be calculated or shown, so you’re effectively only matching “none” for the respective system.
  • Runtime: Match exactly the system type but only for heating/cooling, respectively. The other type should be ignored; the type of cooling doesn’t affect my house’s heating runtime, and vice versa.

Balance Point

This doesn’t show up for me unless I select “Property type: very similar”, and then I only get Resist but not Heat 1.

The order is also very unusual, whether it’s in 1, 2, or 3 columns. In 1 column I get “Setback, setpoint, balance point, property, runtime”, while in 3 columns reading top-down from the left I get “Setback, balance point, property, setpoint, runtime”.

The outdoor temperature at which your indoor temperature does not change. Heat pump users will enjoy this for their heating, but all users can still see the resist value.

I wonder which is more useful:

  • balance point (which tells you what temperature your home naturally drifts towards; I concur this only seems meaningful for resist, but not so much for heating and cooling)
  • slope of the line (which tells you how much outdoor temperature impacts performance), or
  • temperature-change-per-hour compared at preselected temperatures (which could be used to compare your house to others given similar weather conditions, for the same reason as using degree-days to compare runtime)

Setback
The difference in your home vs away setpoints. Determined just by your current schedule.

Actually, I see this value even though I don’t use “away” in my schedule since we work from home. :smiley: In fact, since installing my Ecobee I’ve never once set the system to “away”, as the house has never been empty long enough to need it (thanks to quarantine). But I’m just being pedantic. :wink:

A flat resist line is an indication of a well-insulated home. You see very little temperature change when the HVAC system is off.

Heat and cool “flatness” is sort of a different thing. The slopes really only matter for compressors; you can sort of use them to see how the efficiency changes across outdoor temperatures. Flatter is probably better in general, but more important is the average y-value of a profile and whether or not the value where y=0 is appropriate for your home.

I don’t currently have any plans for the scores to come back. The best replacement for your resist score is the Resist Balance Point metric. It’s not exactly the same thing, but it will show you where you stand.

1 Like

This is a lot to reply to. :smiley:

First of all, you’re totally right that different metrics really necessitate different comparison rules. The reason I didn’t attempt that yet is that metrics take a hot second to fetch. I need to look into some optimizations here so it doesn’t take a minute+ to fetch them. I think this is an improvement worth making.

If you change your system type the metrics are instantly* re-calculated. First of all, your thermostat has a profile. This is generated once a week and nothing you do on the compare tab changes this. Then there are metrics. These are generated on the fly as you click different comparison options or change your system type, but then they are cached for a week. I agree it’s not exactly clear so I’m open to suggestions. Right now I just show the date your profile was generated to keep it simple.

Sounds like a bug; I’ll investigate.

Heat balance point is an important property for heat pump users as it indicates where they need to use aux heat. Resist balance point is of questionable utility but for now it’s there. I appreciate feedback like yours as it’ll help me figure out what some actual good metrics actually are. The trick is building them in a way that’s consumable for most users but powerful enough for those that want more detail.

I would have to look specifically at your data to see; if you want me to take a look just PM me your thermostat serial number and I’ll dive in.

Just wanted to say that these metrics are so much more useful than the scores, but I also want to put out that there’s more to efficiency and energy savings that thermostat setbacks.

I have a modulating condensing boiler and use outdoor reset–I could give a long-winded explanation of this, but essentially–it’s a low and slow approach. The longer my boiler runs (at lower water temperatures) the more efficient it is. For me–longer boiler times and therefore more runtime per degree day is a win.

Comparing my system to houses that are geared to thermostat setbacks is kind of comparing two opposite strategies that will could skew each other’s data. The goals of each strategy (long vs short runtimes) are at different ends of the spectrum.

It’d be nice to be able to compare my energy use to others who use this strategy/technology.

That’s the beauty of metrics. There’s no definitive “this is better”. It’s all just a bunch of statistics - you vs other beestat users with boilers. If you think more runtime is better, then you can do so but you also get to see how other users have their systems set up.

If there are boiler-specific metrics that would be useful let me know and I can research and add them.