[RfC] Redundant channel information for managed things

[jlaur]

This is to be considered a discussion issue and a follow-up to #3330 (comment). I thought it would be more clean to have a distinct issue for tracking this.

Recently there were some discussions in this forum thread that triggered me to come back to this topic:
https://community.openhab.org/t/come-back-and-learn-how-to-use-file-based-configuration/147067

The issue (as I see it) is the difference between managed things:

{
  "danfossairunit:airunit:-1062731542": {
    "class": "org.openhab.core.thing.internal.ThingStorageEntity",
    "value": {
      "isBridge": false,
      "channels": [
        {
          "uid": "danfossairunit:airunit:-1062731542:main#current_time",
          "id": "main#current_time",
          "channelTypeUID": "danfossairunit:currentTime",
          "itemType": "DateTime",
          "kind": "STATE",
          "label": "Aktuel tid",
          "description": "Aktuel tid rapporteret af air unit\u0027en",
          "defaultTags": [],
          "properties": {},
          "configuration": {}
        }
      ],
      "label": "Danfoss Air Unit",
      "configuration": {
        "host": "ccm.local",
        "refreshInterval": 10,
        "updateUnchangedValuesEveryMillis": 60000
      },
      "properties": {
        "modelId": "w2/a2",
        "serialNumber": "xxx",
        "thingTypeVersion": "2",
        "vendor": "Danfoss"
      },
      "UID": "danfossairunit:airunit:-1062731542",
      "thingTypeUID": "danfossairunit:airunit"
    }
  },

[...]

and file-based things:

Thing danfossairunit:airunit:a2 "Danfoss HRV" [host="ccm.local", refreshInterval=10,
updateUnchangedValuesEveryMillis=60000]

For managed things there is some information stored which is not stored for file-based things:

• Label
• Description
• Properties
• Channels (including label, description, channel type UID, auto update policy and properties)

All this is redundant. In my experience redundancy usually will haunt you because of bugs in synchronization, i.e. failure to
properly maintain the redundant data.

This seems to be the case here also.

In 4.0 channel migration is partially solved by #3330 thanks to a massive effort by @J-N-K. With "partially" I mean no disrespect, but we still have at least these issues:

• Thing upgrade gives different result than recreating thing #3569
• Thing migration: channels order is lost #3584
• thing upgrades: missing support to add/change/remove channel properties #3610
• Auto update policy on channel level cannot be updated
• Removal of channel type at the same time of removing last usage (with instructions provided) causes a two minute delay and a warning: "[core.thing.internal.ThingManagerImpl] - Channel types or config descriptions for thing 'xxx' are missing in the respective registry for more than 120s. In case it does not happen immediately after an upgrade, it should be fixed in the binding."

And this would have been avoided:

• Framework sends DecimalType instead of QuantityType to Thing handler #3669
• [mqtt.espmilighthub] Fix upgrade instructions openhab-addons#16127

Example - migrated Thing:

Recreated Thing with correct channel order and new channels in configured language:

Additionally, even with this migration mechanism in place, contributors are required to provide upgrade instructions for maintaining managed things (they are not needed for file-based things). These instructions are redundant themselves, since actually all information is already available. Otherwise file-based things wouldn't work. They are also error-prone and need proper testing for each binding removing/updating or adding channels.

It seems also that there is a lot of complexity for being able to synchronize/maintain things.

Therefore, I'm wondering if managed things could be simplified by aligning with file-based things in the sense that the redundant information is removed from jsondb and instead created runtime - similar to how this works for file-based things. I'm suggesting/asking this only from a logical standpoint - I'm not familiar with the code.

When re-reading #3330 (comment), if I understand it correctly, this comment assumes that we have to deal with this redundancy. I understand this can be difficult, and just to be clear: I do not suggest to complicate it even further, but actually to completely remove everything from jsondb which is redundant.

The benefits would be:

• Simplification - leading to fewer bugs.
• Easier and less error-phone for contributors to add/update/remove channels.
• Existing bugs/short-comings would be automatically resolved, for example the three mentioned above.
• One benefit of using file-based things would be eliminated - the argument that managed things from time to time might need to be recreated to be fully correct would disappear.
• Changing language would have effect on labels and descriptions (at least after a restart).

I don't know if this is feasible at all and/or complete nonsense, and maybe it has already been discussed in the past?

Cc @J-N-K, @lolodomo, @kaikreuzer - tagging you directly as you probably have some direct feedback on these thoughts.

[J-N-K]

The information is NOT redundant. You can change the label or description of a channel (I don't know how to do that in text files, I guess you can do it also there). So in general the label and the description should be preserved when changing something like Number to Number:Temperature because there could be a good reason why the user changed the label.

[jlaur]

You can change the label or description of a channel

How do you do that?

[J-N-K]

Go to the "Things", select a thing, go to "Channels", select a channel. Click either on "Channel details" or "Configure channel" (depending on the channel). Change/add label or description, press "Done".

[jlaur]

Go to the "Things", select a thing, go to "Channels", select a channel. Click either on "Channel details" or "Configure channel" (depending on the channel). Change/add label or description, press "Done".

Thanks, I didn't see this in 3.4, but it works like you described in 4.0. Is it a new feature? And do you know why it's possible to edit label and description for a channel? I really don't see the use, so probably I'm missing something.

[lolodomo]

Same here, I do not see the use to modify a channel label/description.

[J-N-K]

For extensible channels it's absolutely necessary, because there inheriting from the channel-type would result in channels with different ids but the same label.

But there is also a use-case for other devices: Think of a two-channel unwire input, standard labels "Input 0", "Input 1" but my device has them labeled as "Input 1" and "Input 2", changing the channel-label then makes a lot of sense. On an 8-channel switch the channels might be labeled "Relais 0" to "Relais 7", adding a description what is actually linked to the channel might be a good idea to keep track of that.

[jlaur]

For extensible channels it's absolutely necessary, because there inheriting from the channel-type would result in channels with different ids but the same label.

Good point, agreed, and this also corresponds to the behavior in .things files where you can at least provide label for such channels.

But there is also a use-case for other devices: Think of a two-channel unwire input, standard labels "Input 0", "Input 1" but my device has them labeled as "Input 1" and "Input 2", changing the channel-label then makes a lot of sense. On an 8-channel switch the channels might be labeled "Relais 0" to "Relais 7", adding a description what is actually linked to the channel might be a good idea to keep track of that.

Wouldn't it be sufficient to use proper item labels/descriptions for such cases? Renaming labels for channels provided by a binding seems to just obscure things. You also risk having the channel labels/descriptions overridden by an upgrade.

Anyway, this seems like a corner case that of course could be discussed. We still need channel label, at least for extensible channels, but what about the other data provided in the JSON mentioned - do we agree that it contains at least some redundancy, especially for channels, and that this has been - and still is - causing problems?

[openhab-bot]

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/thoughts-on-upgrading-bindings-to-use-new-uom-features-for-humidity/153615/10

[splatch]

The information is NOT redundant. You can change the label or description of a channel (I don't know how to do that in text files, I guess you can do it also there). So in general the label and the description should be preserved when changing something like Number to Number:Temperature because there could be a good reason why the user changed the label.

As far I know thing files provide only a way to specify channel label. They DO NOT have any way to provision description. The xtext/xbase syntax haven't been extended since a long time, I believe its mostly the same. Its still smells with ESH. ;-) Same thing applies to channel and thing properties - both elements will be read only. Thing defined in file can't be amended by handler through editThing() call, channel list can't be mutated.
I am not certain how important thing properties are. To my knowledge in some situations they are used as sort of workaround to pass information between discovery and acceptance of a thing (framework automatically maps matching properties to config parameters). They store sometimes serial number etc. which is displayed only for information purposes.

Nonetheless, these are not available when using xtext stuff.

[lsiepel]

Agree with @jlaur regarding the partial 'duplicate' information. There should be support for overriding the labels as mentioned, i guess there might be other ways to do that. Similar for the description, but let's not focus on that.

AFAICT all the other attributes seem redundant and can be detected based on the xml structure similar to how file based config does.

Are there other reasons besides the labels to keep the thing upgrades?

If we want to keep the thing upgrade instructions i hope we can extend the features. On my wishlist is:

• renaming channel id's (finally consistent naming convention whoop whoop)
• above renaming also updates item/channel links
• support all attributes (auto update policy , tags, possibly more)

[jlaur]

If we want to keep the thing upgrade instructions i hope we can extend the features.

That is the opposite of my proposal. 😉

• above renaming also updates item/channel links

Can you elaborate on this?

• support all attributes (auto update policy , tags, possibly more)

Updating auto update policy is not needed, since it is not part of the serialized JSON for managed Things, IIRC.

[lsiepel]

If we want to keep the thing upgrade instructions i hope we can extend the features.

That is the opposite of my proposal. 😉

Depends. In its current form i also experience the downside and think it is better to remove it. So i'm with you on that one.
All i want to say is that i can imagine that when the upgrade routine was designed (i was not there when it was discussed) it's future was predicted to become much more impartant with a richer feature set.
I think the upgrade are an answer to some concerns about breaking changes and have to fix things (recreate things etc) after each upgrade. I can still see a feature for this upgrade routine, but then it needs some work.

• above renaming also updates item/channel links

Can you elaborate on this?

If a channel id changes from energy_now to energy-now i would like to rename the channel id. And to prevent orphan links also update all links to it. The current add/delete prevents me from creating upgrade instructions to make all channels of all bindings consistent according to the to the naming convention lower-case-hypen.