How to use connectedhomeip as a submodule of a real openwrt project?

[plan44]

Hi @ksperling-apple,

Thanks a lot for your work for building matter on Openwrt - this is very useful stuff! 😀

With the help of the openwrt gn package and scripts/configure I managed to build my matter bridge daemon p44mbrd, but needed to patch two small obstacles, see below.

The basic difference between the provided example build of lighting-app in the matter-netmon package and my setup is that in the latter, connectedhomeip is itself a submodule of the main project, while in the former the root repo is connectedhomeip itself, and the project lives in the example subdirectory. To make it work:

• scripts/configure requires that --project is relative to the connectedhomeip directory (but ../.. style paths work), and insists that the project is a subdirectory, and not a parent (check on line 93). I just commented out line 92 and 93 for now. I think --project should also accept absolute paths outside connectedhomeip, I'll post a PR for that.

• the new openwrt build system option PKG_SOURCE_SUBMODULES only works on the first level, so with connectedhomeip as a submodule it does not help, because it is one level deeper where we want to select only a few modules from the large set in the SDK. Also before trying to build in openwrt, I solved this by adding update = none and ignore = all to all entries in .gitmodules.

These fixes need using a modified connectedhomeip tree, but as I need some not-yet accepted PR anyway, for now I use my own fork where I rebase my set of small mods onto master from time to time.

But I guess any non-example matter app will not have its sources in the connectedhomeip tree, and most of them would want to avoid pulling 34GB of unneeded submodules - so establishing generic mechanisms in the SDK to do so would be nice 😉

[plan44]

@ksperling-apple As promised, I just posted a PR in connectedhomeip to allow scripts/configure to work with project roots outside the chip repo: project-chip/connectedhomeip#29627

[ksperling-apple]

Thanks, merged project-chip/connectedhomeip#29627. The Matter SDK has scripts/checkout_submodules.py that attempts to help with checking out the right submodules (though what's considered required for a particular platform is still pretty broad), but that doesn't really help with an OpenWrt style build.

Maybe PKG_SOURCE_SUBMODULES can be modified to work with nested submodules? It would involve dropping the --recursive option in favor of something like working out which paths are nested inside other paths and then separately initializing those iteratively.

[plan44]

Maybe PKG_SOURCE_SUBMODULES can be modified to work with nested submodules? It would involve dropping the --recursive option in favor of something like working out which paths are nested inside other paths and then separately initializing those iteratively.

What about:

• not loading submodules at all in the first step (PKG_SOURCE_SUBMODULES:=skip)
• then downloading the CHIP repo without --recursive in a separate step as described in Download Override in the OpenWrt Wiki.
• then patching .gitmodules to disable auto-download of all the submodules not needed (could be the standard OpenWrt patch mechanism doing that)
• only then calling git submodule update --init --recursive

[ksperling-apple]

I think working with nested submodules is something the OpenWrt build system should support without complicated contortions in the package Makefile... alternatively hooking in a custom download mechanism could be made less complicated (the re-packaging into a tar should really be separate from the actual download to make it easier to implement a custom method).

I think this is an issues for core OpenWrt though rather than this repo.

[plan44]

I think working with nested submodules is something the OpenWrt build system should support without complicated contortions in the package Makefile... alternatively hooking in a custom download mechanism could be made less complicated (the re-packaging into a tar should really be separate from the actual download to make it easier to implement a custom method).

I think this is an issues for core OpenWrt though rather than this repo.

Basically, I agree. It would be nice to have better support for nested submodules in OpenWrt, and worth trying to find a implementation for that.

But as your PKG_SOURCE_SUBMODULES patch does not seem to make it into 23.05, let alone 22.03, or at least it seems to take months (I have similar experiences with other openwrt contribution attempts) and I am developing a OpenWrt based real world app now, I am wondering about more pragmatic solutions right now 😉. But here's the wrong place, I agree.

[ksperling-apple]

Yeah it's unfortunate that that patch didn't get picked up for 23.05 at least...