feat(muxpi): provisioning: skip user creation, boot check via URL

[thp-canonical]

Description

1. Allow skipping of user creation when deploying an image (for pre-configured images).
2. Allow a custom check URL to allow for testing services running in images.

Documentation

Documentation with the newly-added keys is added. As part of this, the recognized image types (for user creation) are also documented.

Tests

Untested proposal / request for comment.

[plars]

Overall, this looks pretty good and thanks a lot for also adding documentation around it! I have a couple of review comments below, but more generally, I was wondering what sort of command you would need to specify in order to determine that the device booted? especially since you can't easily template the IP of the dut as part of it.
I'm wondering if we should just handle this as a special image type and check whether it booted in a static way, because we don't normally allow injection of commands like this during the provision process, and I think this could be fraught with problems, and potential for allowing someone to inadvertently render the device unusable until someone fixes it.

[thp-canonical]

Thanks for your review. PR updated, responses to your questions inline below.

I was wondering what sort of command you would need to specify in order to determine that the device booted?

Any command that e.g. checks if a HTTP endpoint is available:

curl --connect-timeout 2 "http://$DEVICE_IP:1234/foo/bar"

especially since you can't easily template the IP of the dut as part of it.

That has now been fixed, I did have env.update() in there that exposes $DEVICE_IP, but I forgot shell=True in the subprocess.check_output() call, this should now work.

I'm wondering if we should just handle this as a special image type and check whether it booted in a static way, because we don't normally allow injection of commands like this during the provision process, and I think this could be fraught with problems, and potential for allowing someone to inadvertently render the device unusable until someone fixes it.

One thing that could work is instead of allowing a generic boot_check_cmd, we just allow a boot_check_tcp_port (say), and if set, it will consider the device booted once a TCP connection to the destination port can be established (or boot_check_http_port, which will consider a successful GET / HTTP/1.1 request on that port to mean the device is online). WDYT?

[plars]

I like your idea of replacing boot_check_cmd with boot_check_tcp_port or boot_check_http_port, however I think for the latter, it would suffice to just call it boot_check_http and then give it the URL with the port if needed.

Also, you probably saw, but it also looks like the auto-spell-check got tripped up on "xz" of all things.

[thp-canonical]

I like your idea of replacing boot_check_cmd with boot_check_tcp_port or boot_check_http_port, however I think for the latter, it would suffice to just call it boot_check_http and then give it the URL with the port if needed.

Went for boot_check_url (as it can be anything that's supported by urlib.request.urlopen()), but can change to boot_check_http if that sounds better to you. I reverted all the DEFAULT_BOOT_CHECK_CMD changes back to how it was initially to not unnecessarily touch existing code.

I opted to use $DEVICE_IP instead of any other kind of templating/replacement to be consistent with the usage of $DEVICE_IP in test_cmds, even though in test_cmds, it really is a proper environment variable, whereas here we just replace a literal $DEVICE_IP with the value (so e.g. ${DEVICE_IP} wouldn't work here, even though it would work fine in test_cmds), this is hopefully made clear enough in the documentation:

a literal $DEVICE_IP in the URL will be replaced with the IP address of the DUT.

Also, you probably saw, but it also looks like the auto-spell-check got tripped up on "xz" of all things.

Done.

[plars]

/canonical/self-hosted-runners/run-workflows 552e8b3

[plars]

It looks like it's just failing the black formatting check. Would you mind re-running black on it and pushing it?

[thp-canonical]

It looks like it's just failing the black formatting check. Would you mind re-running black on it and pushing it?

Done. I also noticed that my PR fixes weren't signed commits, so I squashed all into a single signed commit.

[plars]

/canonical/self-hosted-runners/run-workflows b8a0e20'

[plars]

+1 thanks!