Add MQTT Override per Page / Multiple MQTT Server Support

[Humancell]

I'd like to separate the default "command and control" MQTT server from the MQTT server that is used by some pages.

I know that you are currently looking at the possibility of changing MQTT libraries, and thought this might be a good time to discuss this possibility.

My goal would be that I could have multiple MQTT servers - for example one inside my house on my LAN, one in the cloud in AWS, and maybe also have one that is in the cloud being used by my parents.

If a page had the option to override the default MQTT server then it's events would be sent to that server. And that server could then update objects on that page. This would allow me to have pages connected to my MQTT server in my house which could provide local services. Another page might be connected to my cloud server where other connectors are being provided to other services. Lastly, I might have a page that is monitoring and controlling devices at my parents house, using their MQTT server.

Any page that did not provide the override would use the default MQTT server.

This would also allow me to install pages on my panel where someone else owns and controls the data and services that it provides. They would secure their MQTT server using the standard credentials.

I do understand where some argument can be made to just use one MQTT server, but I can think of where I want one default MQTT server that might download pages that would interact with my local network.

I'm not sure how expensive the MQTT connections are, but if an array could be created as pages are created or loaded then the events and updates could maybe be routed using the array index.

I see this as a potential way to create a powerful ecosystem.

Thoughts?

[fvanroie]

I think there are better ways, like bridging MQTT brokers for specific topics:

• AWS IoT Greengrass MQTT bridge
• How to Bridge Mosquitto MQTT Broker to AWS IoT

I haven't seen IOT devices that implement multiple brokers on the device. Managing a list brokers on multiple plates can become tedious fast... That being said, I don't use cloud brokers. I understand the concepts of MQTT bridging but don't have experience with implementing an MQTT topology.

[Humancell]

Hello,

I'm sorry it has taken me so long to update this request. We've got so much going on, and have been overwhelmed creating a series of OpenHASP demos and architecting our portal solution. :-)

I think that I understand the original intent of OpenHASP and it's predecessor, however I really think that you have created something larger than you might be imagining. I'll see if I can elaborate with a few stories.

My team and I are currently working on a portal solution that we intend to also open source. It will have a variety of features that we need, and we believe can be useful to others. Here are some of the features that we're currently designing, and some of the thoughts behind them.

• A secure portal with accounts, users, and roles
• The ability to:
  • design pages visually in the browser
  • save the pages and tag them
  • support versioning of pages
  • share pages publicly or privately
• Create page groups with tags
• Add "devices" to your account
• Manage your devices configuration
• Associate pages or page groups to devices specifically or with tags
• Order the pages and pages within page groups per device
• Automate the deployment of pages to devices
• Provide bulk operations across devices

All of this is envisioning where a house might have 10, 20, 50, or more OpenHASP devices, and an owner or installer wants to be able to easily deploy and manage a large number of devices. And in commercial installations, it could be hundreds or even thousands. I'm talking about replacing every light switch that I can with OpenHASP. :-)

There are a few things that I believe have to be considered when looking at this scenario:

1. The MQTT protocol has to be thought of as critical to all of this working. The commands and events could become standardized in a way to work across a broad set of potential hardware.
2. Redundancy become critical. There ought to be a way to have at least a primary and secondary MQTT servers to ensure some level of fault tolerance.
3. Consider thinking about OpenHASP as a micro-OS and the pages as apps. So any particular OpenHASP device might be "running" any combination of pages, in any order.
4. In this architecture, the management of OpenHASP becomes separate from the pages it is hosting and "running".
5. If the owner or installer - or page provider - is able to determine where and how they install and use MQTT brokers, it would provide the maximum amount of flexibility. They could deploy an on-prem MQTT server. They could bridge the MQTT server. They could be entirely cloud based. Most importantly they could be a combination.

There are several other considerations in these scenarios ... like transferability. If I have a home with 20-30 OpenHASP devices, when I sell my house how do I transfer them to the new owner?

Anyhow ... just some more thoughts.

Scott

[fvanroie]

Thanks for the background info, it helps to understand where you are coming from. I don't think this problem is unique to openHASP:

Redundancy become critical. There ought to be a way to have at least a primary and secondary MQTT servers to ensure some level of fault tolerance.

Agreed! However all solutions I've found so far are architectural designs. These typically include a load-balancer with fail-over and/or a cluster of multiple brokers. Most brokers provide clustering support.

• Load balancing - MQTT broker clustering
• How to Build an High Availability MQTT Cluster for the Internet of Things

It should suffice for the device to have a network connection to communicate. For critical setups, the redundancy needs to be on the network layer (load-balancer) or server side (cluster). Of all the components making up an MQTT infrastructure, the IOT client has the least resources to handle possible fail-over scenarios. Large (commercial) buildings with critical services usually already have some redundant infrastructure already in place.

While it is technically possible to run multiple clients on the device, neither Tasmota nor ESPhome implement failover/secondary brokers on the client. Are there any IOT devices out there that implement this?