Unmatched entities in MQTT payload #196

glitter-ball · 2021-11-14T18:55:23Z

Bug description
EMS-ESP32 regularly sending the same few points with no matching data.
2021-11-13 23:00:05 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'

At one level, this is trivial! But...

It's generating megabytes of log file every day - in 20 hours, my HA instance has logged this error 17,000 times!
It's hammering the SSD on HA
It's usually the same few points, so maybe there's a bug somewhere causing this.

I can delete the point with MQTT Explorer, but it will be back - presumably when I restart EMS-ESP32? If the point has no associated data, can it be stopped from sending the template? 'tempautotemp' is far and away the biggest culprit! Is there something special about the definition of this point?

Steps to reproduce
Run EMS-ESP32 and HA : HA log file fills up with this error.

Expected behavior
No HA errors!

Screenshots

Device information```

EMS-ESP version 3.3.0b2

Boiler: Worcester Logamax Plus/GB192/Condens GC9000 (DeviceID:0x08 ProductID:208, Version:01.01)
heating active: off
warm water active: off
selected flow temperature: 28 °C
burner selected max power: 100 %
heating pump modulation: 96 %
outside temperature: 11.3 °C
current flow temperature: 26.5 °C
gas: off
flame current: 0.0 uA
heating pump: on
fan: off
ignition: off
heating activated: on
heating temperature: 75 °C
burner pump max power: 100 %
burner pump min power: 60 %
pump delay: 3
burner min period: 5
burner min power: 0 %
burner max power: 100 %
hysteresis on temperature: -10 °C
hysteresis off temperature: 6 °C
burner current power: 0 %
burner starts: 19281 times
total burner operating time: 173 days 6 hours 51 minutes
total heat operating time: 132 days 16 hours 41 minutes
total UBA operating time: 1200 days 18 hours 11 minutes
last error code: EA(227) 09.10.2021 10:27
service code number: 204
maintenance message:
maintenance scheduled: off
maintenance set time: 6000 hours
maintenance set date: 01.01.2014
ww selected temperature: 60
ww set temperature: 60
ww type: buffer
ww comfort: hot
ww flow temperature offset: 40
ww max power: 100
ww circulation pump available: off
ww charging type: 3-way valve
ww hysteresis on temperature: -5
ww hysteresis off temperature: 0
ww disinfection temperature: 70
ww circulation pump frequency: off
ww circulation active: off
ww current intern temperature: 55.2
ww current extern temperature: 55.2
ww current tap water flow: 0.0
ww storage extern temperature: 55.2
ww activated: on
ww one time charging: off
ww disinfection: off
ww charging: off
ww recharging: off
ww temperature ok: on
ww active: off
ww heating: off
ww starts: 5233
ww active time: 40 days 14 hours 10 minutes

Thermostat: RC300/RC310/Moduline 3000/1010H/CW400/Sense II (DeviceID:0x10, ProductID:158, Version:33.03)
error code: (0)
date/time: 18:52:15 14/11/2021
floor drying: off
damped outdoor temperature: 11.8 °C
floor drying temperature: 0 °C
building: medium
minimal external temperature: -5 °C
ww set temperature: 60
ww mode: own_prog
ww set low temperature: 45
ww circulation pump frequency: own_prog
ww charge duration: 60
ww charge: off
ww circuit 1 extra: 0
ww disinfection: off
ww disinfection day: tu
ww disinfection time: 120
hc1 selected room temperature: 18.5
hc1 current room temperature: 18.7
hc1 mode: auto
hc1 mode type: comfort
hc1 eco temperature: 15.0
hc1 manual temperature: 18.0
hc1 comfort temperature: 21.0
hc1 summer temperature: 17
hc1 design temperature: 45
hc1 offset temperature: 0
hc1 min flow temperature: 25
hc1 max flow temperature: 82
hc1 room influence: 9
hc1 current room influence: -1.7
hc1 nofrost temperature: 5
hc1 target flow temperature: 28
hc1 heating type: radiator
hc1 set summer mode: auto
hc1 summer mode: off
hc1 control mode: optimized
hc1 program: prog_1
hc1 fast heatup: 0

Controller: ErP (DeviceID:0x09, ProductID:209, Version:01.03)

Dallas temperature sensors:
Sensor 1, ID: 28-BE2A-7791-0402, Temperature: 26.2 °C (offset 0.0)

The text was updated successfully, but these errors were encountered:

proddy · 2021-11-14T19:23:10Z

I know it's annoying, it happens when there is a homeassistant/sensor/ems-esp/<name>/config topic in the MQTT broker (which is retained with the retain flag set) but the actual data in the ems-esp/thermostat-data topic doesn't yet exist. It can usually happen during the first few seconds when EMS-ESP is booted. But then the warnings should stop, unless somehow that old topic is still lingering about. So I'm puzzled...

if you fire up MQTTExplorer and in the Search bar at the top type in tempautotemp what do you see? Hopefully 3 topics...?

proddy · 2021-11-14T19:28:43Z

and if you're worried about the size of the log file (not a solution I know but a temporary workaround) disable the logging in HA with

logger:
  default: critical
  logs:
    homeassistant.components.mqtt: error

https://www.home-assistant.io/integrations/logger/

glitter-ball · 2021-11-14T19:32:20Z

I get a little flurry at start-up as things sort themselves out and templates get populated with all the necessary variables. But, this one just keeps on coming! I only get the one topic...

There's something weird/different about this one point. Loads of disk space, so log file not a catastrophic issue, but I'd rather HA wasn't wasting time writing out 20k+ errors every day *8)

proddy · 2021-11-14T19:37:35Z

that's odd. EMS-ESP wouldn't create that homeassistant topic if there wasn't a value. Are you sure it sneaks back in after you delete it in MQTTExplorer and restart EMS-ESP?

glitter-ball · 2021-11-14T19:39:24Z

Now the fun. I deleted the 'tempautotemp' topic above, so no entries in MQTT Explorer. I then went to my thermostat and changes the temperature by spinning the wheel - nothing else. So now we have a TEMPorary AUTO TEMPerature set.

MQTT Explorer now shows...

So now I have matched data and no HA errors. I suspect things get interesting when the temporary temperature period ends. At that point, tempautotemp probably has no definition but the template is still being sent...

glitter-ball · 2021-11-14T19:43:55Z

I realised I turned the temperature down last night when I was the last to bed, relatively early! The errors started started at 23:00:05 when the thermostat would have gone from my temporary temperature back to the time programme. At that point, there was no temporary info to send any more.

I've just tried resetting the temp and reverting to AUTO but this hasn't replicated the problem, so it may depend on reaching the next programme time/temperature change.

glitter-ball · 2021-11-14T19:46:16Z

So... the logic is there to define the point when the stat begins to publish it, but when it stops publishing it (and the point disappears from the payload), the definition payload remains. Weird. Nasty.

proddy · 2021-11-14T19:48:56Z

yes, that's it! If there is no value for tempautotemp then it will not be included in the thermostat_data payload, and HA will throw the warning. I need to consult with Michael as this entity is something he recently added so I'm not that confident I fully understand how it works.

I think the solution is either

we add a new attribute type to a device value (like tempautotemp) identifying it as something that is volatile, i.e. it can come and go. Then we remember the state, and if it has no value (i.e. it goes) we delete the HA config topic
or, we just use a dummy value for these volatile types so its gets included in the _data payloads, and tell HA to ignore them somehow.

glitter-ball · 2021-11-14T19:53:41Z

Great - glad we at least understood what's going on pretty quick. I think there might be other points doing other weird stuff like this. I'll try and collect evidence this week. One was something to do with ww but I'll wait and check.

I guess the first option above is technically better, but I don't know about the coding practicalities.

glitter-ball · 2021-11-14T19:54:34Z

Shame the thermostat doesn't handle it better but I suppose there's no point in it sending data that doesn't exist any more!

glitter-ball · 2021-11-14T20:02:27Z

Yep, the errors are back and only the definition payload is being sent. The data payload disappears when...

The next programmed time is reached
I flick the stat from Auto to Manual back to Auto

glitter-ball · 2021-11-14T20:06:53Z

Going back in the logs, one of the changes I made also caused a bigger burp, albeit short-lived. Thermostat is obviously having a good cough then...

2021-11-14 19:56:53 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-14 19:56:53 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{ value_json.hc1.mode }}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.summermode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.hc1.summermode}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwcharge}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.wwcharge}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwdisinfect}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.wwdisinfect}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.mode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.modetype}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.ecotemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.manualtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.comforttemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.summertemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.id}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.errorcode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.building}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.floordry}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.floordrytemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.seltemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.designtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.offsettemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.minflowtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.maxflowtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.roominfluence}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.nofrosttemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.targetflowtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.controlmode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.program}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.dampedoutdoortemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.currtemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.heatingtype}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.datetime}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwmode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwcircmode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwsettemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwsettemplow}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwextra1}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.minexttemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.summersetmode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.curroominfl}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwchargeduration}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.fastheatup}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwdisinfectday}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.wwdisinfecttime}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.hatemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.hc1.hatemp}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from 
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.seltemp}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.hc1.seltemp}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.components.mqtt.climate] Could not parse temperature from 
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.hamode}}'
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Error parsing value: 'value_json' is undefined (value: , template: {{value_json.hc1.hamode}})
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.components.mqtt.climate] Invalid modes mode: 
2021-11-14 19:57:03 ERROR (MainThread) [homeassistant.helpers.template] Template variable error: 'value_json' is undefined when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-14 19:57:04 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-14 19:57:04 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'

proddy · 2021-11-15T09:43:31Z

I've implemented my option 1 (delete the HA config when the value is bogus). I'll test it for a bit and see how well it stands up

proddy · 2021-11-15T14:33:05Z

@glitter-ball looks like its working, I haven't seen any errors in the HA logs yet. Do you want to give it a go? It's called 3.3.0b7 and you can find the binary here.

if it works I'll do a PR against the EMS-ESP dev branch.

fixes #196 (HA missing entities), add row click to devices, change max limit on integer types to avoid NaN, refactored generation of device value JSON objects

glitter-ball · 2021-11-16T18:19:49Z

Will try and check it tonight...

glitter-ball · 2021-11-16T20:28:08Z

I've installed 3.3.0b7 and in the process of restarting everything, I now wonder if there's another one brewing...

2021-11-16 20:23:29 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-16 20:23:30 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'

Similar evidence in MQTT Explorer...

... but I see the Retained bit is set. I don't know if this points to another problem point or whether I've restarted things or cleared payload in MQTT Explorer in the wrong order! So, probably need to establish a consistent approach to test this.

proddy · 2021-11-17T21:46:30Z

the logic I wrote does

if a device value (entity) is found and has a valid value (e.g. lastcode)
- it will add it to the main EMS-ESP topic, like thermostat_data or boiler_data
- it will create the HA config topic in homeassistant/sensor/ems-esp/..... as a RETAINED topic. This is because if the MQTT broker restarts Home Assistant won't shit it's pants when the it tries to render the lovelace dashboard
if a device value (entity) suddenly changes from having a valid value to not having a value
- it will no longer be included in the main topic like boiler_data
- it will remove the HA entry for the topic in homeassistant/sensor/ems-esp/.....

it's not bulletproof by all means and I'm sure there will be some edge cases I haven't thought about.

please keep an eye on it!

glitter-ball · 2021-11-19T16:12:40Z

I wonder whether the handling of the Retained flag is implicated somehow? I had this...

2021-11-19 16:06:56 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:56 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:58 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:58 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:59 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:59 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:06:59 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:00 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:01 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:02 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:03 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:06 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:06 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:06 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:07 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:09 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-19 16:07:11 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'

... clear the Retained flag on 'lastcode' and, of course, the errors stop. So, when an entity disappears, is it as simple as clearing its Retained flag too? If it re-appears, the addition code will set the flag again, so I don't think we lose anything. If the broker restarts, the previous retained one it still there.

I wonder if this guy... https://discord.com/channels/816637840644505620/816958041345884180/910424597663453234 ... is having the same issue?

proddy · 2021-11-20T14:39:11Z

when the entity disappears, it deletes the whole config topic along with the retained flag, so that can't be it. And in the log you should see a warning like "Device value lastcode gone silent. Removing HA config topic..."?

glitter-ball · 2021-11-20T15:57:12Z

That's weird because I was seeing the topic still in the broker with the retained flag set. As soon as I cleared it manually, the topic disappeared and errors cleared.

Helpfully, of course, it's not doing it now. I'll keep an eye on it and try to work out what's happening.

proddy · 2021-11-20T16:32:27Z

oh, bugger. so maybe the code I wrote to remove the topic isn't working. I never really tested it in a live situation. I'll do so now!

glitter-ball · 2021-11-20T19:07:33Z

Uh-oh... 'tempautotemp' is back!


2021-11-20 19:01:04 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-20 19:01:05 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'
2021-11-20 19:01:37 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'tempautotemp' when rendering '{{value_json.hc1.tempautotemp}}'

I turned the stat up earlier and it would have reverted to timed programme this evening, so no more data to send. Same picture in MQTT Explorer - template topic with no matching data, retained flag set...

So, my money would be retained flag isn't clearing? According to System page, I'm running 3.3.0b7 - hopefully the one you pointed me to above.

proddy · 2021-11-20T20:26:33Z

my bad, I made a typo. the homeassistant/sensor/ems-esp/... config topics should be removed now. I should test more thoroughly. Build is 3.3.0b9.

glitter-ball · 2021-11-20T20:58:51Z

Phew! Thinking I was missing something... I've loaded b9 and will see what happens. Thanks.

glitter-ball · 2021-11-20T21:07:15Z

Oh dear. This one's back now...

2021-11-20 20:59:33 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-20 20:59:41 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'
2021-11-20 20:59:42 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'

MQTT Explorer shows the usual stuff...

Could this be related to the problem in #214 somehow?

proddy · 2021-11-20T21:18:16Z

that's odd. the boiler_data should be in sync with the homeassistant/sensor/ems-esp/... topics, so if there is lastcode in boiler_data that topic you showed should not exist. I'm sure you deleted everything from homeassistant/ before testing? Did you see anything in the System Log like the warning message "Lost device value for lastcode. Removing HA config"?

proddy · 2021-11-22T08:14:14Z

should be fixed in the latest 3.3.0b10. I haven't seen any issues since, but please check and re-open if necessary

stefboerrigter · 2022-02-14T08:54:08Z

This issue seems to work counter-productive for my setup.
I do use an IRT <-> EMS converter in between, so might be that that causes an issue here.

Issue observed:
After a little while (less than a day) the EMS-ESP32 seems to throw away most of my sensors in HA. snippet from the log:

002+04:43:17.415 W 529: [mqtt] Lost device value for boiler_wwsetpumppower. Removing HA config
002+04:43:17.415 W 530: [mqtt] Lost device value for boiler_wwmixertemp. Removing HA config
002+04:43:17.415 W 531: [mqtt] Lost device value for boiler_wwtankmiddletemp. Removing HA config
002+04:43:17.415 W 532: [mqtt] Lost device value for boiler_wwstarts. Removing HA config
002+04:43:17.415 W 533: [mqtt] Lost device value for boiler_wwworkm. Removing HA config
002+04:43:17.415 W 534: [mqtt] Lost device value for thermostat_errorcode. Removing HA config
002+04:43:17.415 W 535: [mqtt] Lost device value for thermostat_lastcode. Removing HA config
002+04:43:17.415 W 536: [mqtt] Lost device value for thermostat_datetime. Removing HA config
002+04:43:17.415 W 537: [mqtt] Lost device value for thermostat_hc1_seltemp. Removing HA config
002+04:43:17.415 W 538: [mqtt] Lost device value for thermostat_hc1_currtemp. Removing HA config
002+04:43:17.415 W 539: [mqtt] Lost device value for thermostat_hc1_mode. Removing HA config```

proddy · 2022-02-14T09:21:49Z

Think you’re on an older build?

…

On Mon, 14 Feb 2022 at 09:54, Stef Boerrigter ***@***.***> wrote: This issue seems to work counter-productive for my setup. I do use an IRT <-> EMS converter in between, so might be that that causes an issue here. *Issue observed:* After a little while (less than a day) the EMS-ESP32 seems to throw away most of my sensors in HA. snippet from the log: 002+04:43:17.415 W 529: [mqtt] Lost device value for boiler_wwsetpumppower. Removing HA config 002+04:43:17.415 W 530: [mqtt] Lost device value for boiler_wwmixertemp. Removing HA config 002+04:43:17.415 W 531: [mqtt] Lost device value for boiler_wwtankmiddletemp. Removing HA config 002+04:43:17.415 W 532: [mqtt] Lost device value for boiler_wwstarts. Removing HA config 002+04:43:17.415 W 533: [mqtt] Lost device value for boiler_wwworkm. Removing HA config 002+04:43:17.415 W 534: [mqtt] Lost device value for thermostat_errorcode. Removing HA config 002+04:43:17.415 W 535: [mqtt] Lost device value for thermostat_lastcode. Removing HA config 002+04:43:17.415 W 536: [mqtt] Lost device value for thermostat_datetime. Removing HA config 002+04:43:17.415 W 537: [mqtt] Lost device value for thermostat_hc1_seltemp. Removing HA config 002+04:43:17.415 W 538: [mqtt] Lost device value for thermostat_hc1_currtemp. Removing HA config 002+04:43:17.415 W 539: [mqtt] Lost device value for thermostat_hc1_mode. Removing HA config``` — Reply to this email directly, view it on GitHub <#196 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJMO6E3DMDBEM4O6PHBRZDU3C7LXANCNFSM5IAE6NGQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

stefboerrigter · 2022-02-14T09:38:50Z

I'm on 3.3.1 which is latest official version. this was merged back to 3.3.0 it seems?

System Status
EMS-ESP Version
v3.3.1

Think you’re on an older build?
[…]

proddy · 2022-02-14T11:37:00Z

Ah ok. This is fixed in 3.4 On Mon, 14 Feb 2022 at 10:39, Stef Boerrigter ***@***.***> wrote:

…

I'm on 3.3.1 which is latest official version. this was merged back to 3.3.0 it seems? System Status EMS-ESP Version v3.3.1 Think you’re on an older build? … <#m_3445462048818379591_> On Mon, 14 Feb 2022 at 09:54, Stef Boerrigter *@*.*> wrote: This issue seems to work counter-productive for my setup. I do use an IRT <-> EMS converter in between, so might be that that causes an issue here. Issue observed: After a little while (less than a day) the EMS-ESP32 seems to throw away most of my sensors in HA. snippet from the log: 002+04:43:17.415 W 529: [mqtt] Lost device value for boiler_wwsetpumppower. Removing HA config 002+04:43:17.415 W 530: [mqtt] Lost device value for boiler_wwmixertemp. Removing HA config 002+04:43:17.415 W 531: [mqtt] Lost device value for boiler_wwtankmiddletemp. Removing HA config 002+04:43:17.415 W 532: [mqtt] Lost device value for boiler_wwstarts. Removing HA config 002+04:43:17.415 W 533: [mqtt] Lost device value for boiler_wwworkm. Removing HA config 002+04:43:17.415 W 534: [mqtt] Lost device value for thermostat_errorcode. Removing HA config 002+04:43:17.415 W 535: [mqtt] Lost device value for thermostat_lastcode. Removing HA config 002+04:43:17.415 W 536: [mqtt] Lost device value for thermostat_datetime. Removing HA config 002+04:43:17.415 W 537: [mqtt] Lost device value for thermostat_hc1_seltemp. Removing HA config 002+04:43:17.415 W 538: [mqtt] Lost device value for thermostat_hc1_currtemp. Removing HA config 002+04:43:17.415 W 539: [mqtt] Lost device value for thermostat_hc1_mode. Removing HA config``` — Reply to this email directly, view it on GitHub <#196 (comment) <#196 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJMO6E3DMDBEM4O6PHBRZDU3C7LXANCNFSM5IAE6NGQ <https://github.com/notifications/unsubscribe-auth/AAJMO6E3DMDBEM4O6PHBRZDU3C7LXANCNFSM5IAE6NGQ> . You are receiving this because you modified the open/close state.Message ID: @.*> — Reply to this email directly, view it on GitHub <#196 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJMO6BJJGTF4UUU4NYD6R3U3DETNANCNFSM5IAE6NGQ> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

stefboerrigter · 2022-02-14T12:03:44Z

I'll have a test later today. will update accordingly.

stefboerrigter · 2022-02-16T10:10:03Z

tested with latest 3.4 branch 2 days ago. EMS-ESP Version - v3.4.0a18
Issue seems gone, but behavior is the same.
I don't see the messages on removing the topic anymore in EMS-ESP32 node. but in my HASS the node has become unavailable which is the same result as with the previous issue.

I do see these messages every 10 seconds: (Filtered with Node-Red) `

2/16/2022, 8:32:14 AM[node: 14e98a01.d7dd16](http://192.168.188.98:1880/#)
ems-esp/thermostat_data : msg.payload : string[87]
"{"datetime":"08:30:35 16.02.2022","hc1":{"seltemp":19.5,"currtemp":19.2,"mode":"auto"}}"

I have digged a bit deeper, seems like they are notified in HASS, but not as the climate but as sensor (number) entity.
Seems like the climate entity is no longer updated (or removed?) after a day or so.

This snippet is only seen once after starting the service (or reconnecting) which is most likely only used for auto discovery of the sensor. Then the ems-esp topic should update the sensor value?

homeassistant/climate/ems-esp/thermostat_hc1/config : msg.payload : string[649]
"{"~":"ems-esp","name":"Thermostat hc1","uniq_id":"thermostat_hc1","mode_stat_t":"~/thermostat_data","mode_stat_tpl":"{%if value_json.hc1.mode=='manual'%}heat{%elif value_json.hc1.mode=='day'%}heat{%elif value_json.hc1.mode=='night'%}off{%elif value_json.hc1.mode=='off'%}off{%else%}auto{%endif%}","temp_cmd_t":"~/thermostat/hc1/seltemp","temp_stat_t":"~/thermostat_data","temp_stat_tpl":"{{value_json.hc1.seltemp}}","mode_cmd_t":"~/thermostat/hc1/mode","curr_temp_t":"~/thermostat_data","curr_temp_tpl":"{{value_json.hc1.currtemp}}","min_temp":"5","max_temp":"30","temp_step":"0.5","modes":["auto","heat","off"],"dev":{"ids":["ems-esp-thermostat"]}}"

Is there a mismatch in naming between discovery and update(s)?

proddy · 2022-02-16T15:08:51Z

if EMS-ESP can't find a device entity (e.g. a thermostat setpoint value) it will mark it as inactive and also remove the config discovery topic from MQTT. I'm not sure what the problem is with your setup but worth using MQTTExplorer to see what topics and payloads are available

MichaelDvP · 2022-02-16T16:30:03Z

@stefboerrigter

tested with latest 3.4 branch 2 days ago. EMS-ESP Version - v3.4.0a18

Please use the dev build, which is the latest v3.4, The v3.4 branch is alpha and outdated.

stefboerrigter · 2022-02-18T12:15:19Z

Thanks @MichaelDvP,
moved to dev branch;

I have noticed that all topics kept updating except for the climate part.
In Home Assitant all EMS-ESP values where alive after a few days, but the "Climate" node was gone. i'll see for a few days how this beta behaves.
After upgrading i see this:

Isn't there a mismatch between: "Command_topic" given in config and node buildup from ems-esp? (note that i'm not up to speed on mqtt behaviour in HASS).

proddy · 2022-02-18T13:02:34Z

the command_topic is the name of the topic that MQTT discovery uses when changing a value using the component. It's unrelated to the other topics and only picked up by EMS-ESP which subscribes to it.

Back to the issue, is it working or not working? I'm not sure what you mean by a Climate node?

stefboerrigter · 2022-02-18T13:04:34Z

Thanks for explaining;
regarding the issue; I have updated the fw to the latest dev branch and it takes > 24 hrs to validate if it is succesfull or not. So far it looks OK. keep you posted.

proddy · 2022-02-18T13:05:56Z

ok. Still strange. The MQTT topics should all come in within the first 60 seconds

stefboerrigter · 2022-02-18T13:06:59Z

They appear directly(<30 sec), which is ok. the problem is that HASS mentions them as "inactive" after a few (>12) hours. for some reason. seems like (wrong/no) update?

proddy · 2022-02-18T19:29:49Z

not sure. Haven't seen this behaviour before. Please open a specific GH issue if you think this is a new defect.

glitter-ball added the bug Something isn't working label Nov 14, 2021

proddy self-assigned this Nov 15, 2021

proddy mentioned this issue Nov 15, 2021

fixes #196 (HA missing entities), add row click to devices, change max limit on integer types to avoid NaN, refactored generation of device value JSON objects #197

Merged

proddy added a commit that referenced this issue Nov 15, 2021

Merge pull request #197 from proddy/dev

5f6033c

fixes #196 (HA missing entities), add row click to devices, change max limit on integer types to avoid NaN, refactored generation of device value JSON objects

proddy closed this as completed Nov 22, 2021

glitter-ball mentioned this issue Dec 4, 2021

v3.3.1b0: Template variable warning: 'dict object' has no attribute 'tempautotemp' #229

Closed

proddy reopened this Feb 16, 2022

proddy closed this as completed Feb 18, 2022

stefboerrigter mentioned this issue Feb 21, 2022

Climate.hc1 thermostat goes missing after ~24 hrs in HomeAssistant #371

Closed

matigda123 mentioned this issue May 24, 2022

Problem with connecting to Junkers Cerapur Modul #517

Closed

Unmatched entities in MQTT payload #196

Unmatched entities in MQTT payload #196

Comments

glitter-ball commented Nov 14, 2021

proddy commented Nov 14, 2021

proddy commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

proddy commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

proddy commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

glitter-ball commented Nov 14, 2021 • edited Loading

glitter-ball commented Nov 14, 2021

glitter-ball commented Nov 14, 2021

proddy commented Nov 15, 2021

proddy commented Nov 15, 2021

glitter-ball commented Nov 16, 2021

glitter-ball commented Nov 16, 2021

proddy commented Nov 17, 2021 • edited Loading

glitter-ball commented Nov 19, 2021

proddy commented Nov 20, 2021

glitter-ball commented Nov 20, 2021

proddy commented Nov 20, 2021

glitter-ball commented Nov 20, 2021

proddy commented Nov 20, 2021

glitter-ball commented Nov 20, 2021

glitter-ball commented Nov 20, 2021

proddy commented Nov 20, 2021

proddy commented Nov 22, 2021

stefboerrigter commented Feb 14, 2022

proddy commented Feb 14, 2022 via email

stefboerrigter commented Feb 14, 2022 • edited Loading

proddy commented Feb 14, 2022 via email

stefboerrigter commented Feb 14, 2022

stefboerrigter commented Feb 16, 2022 • edited Loading

proddy commented Feb 16, 2022

MichaelDvP commented Feb 16, 2022

stefboerrigter commented Feb 18, 2022 • edited Loading

proddy commented Feb 18, 2022

stefboerrigter commented Feb 18, 2022

proddy commented Feb 18, 2022

stefboerrigter commented Feb 18, 2022

proddy commented Feb 18, 2022

glitter-ball commented Nov 14, 2021 •

edited

Loading

proddy commented Nov 17, 2021 •

edited

Loading

stefboerrigter commented Feb 14, 2022 •

edited

Loading

stefboerrigter commented Feb 16, 2022 •

edited

Loading

stefboerrigter commented Feb 18, 2022 •

edited

Loading