Add device connector to provision laptops with 24.04 using image-deploy.sh [OEX86-359]

[Artur-at-work]

Description

• Add device-connector oem_autoinstall which can be used to provision laptops with Ubuntu 24.04
• Add image-deploy.shscript which is used by oem-swe-x86 team to deploy OEM images starting from 24.04 (no stock Ubuntu). It was adapted for agent environment and relies on agent to download the ISO.

Resolved issues

• Resolves https://warthogs.atlassian.net/browse/OEX86-359

Documentation

• .github/CODEOWNERS was updated to assign ownership of image-deploy.sh to oem-swe-x86
• device-connector-types.rst added oem_autoinstall and provision_data keys for it

Tests

• Run "shellcheck image-deploy.sh"

• Test with agent:

1. Run agent locally with new device connector in testflinger-agent.conf:
   provision_command: testflinger-device-connector oem_autoinstall provision -c /home/artur/testflinger/agent/default.yaml testflinger.json
2. Add default.yaml with device_ip of laptop
3. Submit the job with attachments: user-data, authorization_keys, jenkins-token.txt
   Provisioned HP and Dell successfully:
   https://testflinger.canonical.com/jobs/7ce8e9b6-ff07-4f9a-b2aa-8cad7fdcbba8
   https://testflinger.canonical.com/jobs/cccb263e-7ebf-4f1f-8d3d-eb34fe0e41fa

[Artur-at-work]

/canonical/self-hosted-runners/run-workflows d573c9e

[Artur-at-work]

hello @plars ,
Could you help to review this please?
As I understand, Cert. team is going to develop device connector for 24.04 oemscript provisioning and requires image-deploy.sh script for it.

[plars]

Hi @Artur-at-work - thanks for bringing this to my attention. Can you give me some more information about this? I know that we have 3 oemscript device connectors already - one for each of HP, Lenovo, and Dell oem systems. Will the 24.04 oem images work with the existing device connectors? If not, then will this new script also work for provisioning oem images prior to 24.04?

[Artur-at-work]

No, 24.04 oem images won't work with the old script/connectors.
No, new script won't work with the older images

FYI, the ticket I saw related to device connector: https://warthogs.atlassian.net/browse/CERTTF-352

[plars]

@Artur-at-work So just to be clear, if a system is deployed with the previous oemscript using 20.04 or 22.04 oem images, is it possible to deploy a 24.04 image with this new image-deploy.sh on the same system, and then later use the previous script to deploy a 22.04 or 20.04 image on that same system? Or do we need separate systems for deploy either ONLY 24.04, or ONLY older images?

Have you been able to test whether deploying both older and newer images works on the same system? If we have a system in the lab where this is known to work, it would make a good one to test with.

[Artur-at-work]

@plars, sure, feel free if any questions.

"if a system is deployed with the previous oemscript using 20.04 or 22.04 oem images, is it possible to deploy a 24.04 image with this new image-deploy.sh on the same system" - Yes.
"and then later use the previous script to deploy a 22.04 or 20.04 image on that same system?" - No. 22.04 was relying on ubuntu-recovery and dell-recovery, but it's not available in 24.04. So we can't deploy 22.04 on 24.04 no matter with old or new script.

"Or do we need separate systems for deploy either ONLY 24.04, or ONLY older images?" - Yes. Since 24.04 platform can only re-deploy 24.04, we'll need a separate platform to keep provisioning 22.04.

Yes, I had DUT with 22.04 and ran image-deploy.sh from remote machine to install 24.04 on DUT successfully. Would you like to have access?

Few more things:

• In 24.04 we use only one image for all vendors, because we don't have ubuntu/dell-recovery.
• platforms with 22.04 were formatted with 5.7GB recovery partition. When 24.04 deployed next, it will keep using 5.7GB recovery partition. In long term when ISO size will grow above 5.7GB, it will fail to copy ISO contents into the recovery partition and fail the redeployment. At the moment, I manually plug USB with script to format storage and create 12GB reset partition, then run the image-deploy.sh script. So devices that migrate from 22.04 to 24.04 directly will also have to be formatted at some point in future.

[Artur-at-work]

@plars
I was able to test image-deloy.sh in my agent with updated device connector. The deploy script requires some changes.
Let me work on it, then upload the changes to script and connector later.

Changing this ticket to draft and will request review later.

[Artur-at-work]

@plars hi, I've updated the description and commit.
Could you help to review please? Thank you.

[plars]

Hi, thanks for working on this! A few comments below, but it would also be great if we could test it using a real machine. I'm not aware of any in the lab that are certified for 24.04 already where we could try this, are you?

[Artur-at-work]

hi @plars, thank you for the feedback, the commit was updated.
I've been provisioning the ISO image on the actual HP and Dell laptops. We don't have any certified 24.04 platform yet, but this Dell is work in progress. Can we consider these real machines? Let me know if you meant something else.

Could you review again please?

[plars]

Looks good, thanks for the fixes! +1

[plars]

@Artur-at-work it looks like the only thing missing is that there are unsigned commits? We can't override this. If you need to go back and sign them, you can use a process like this to update them.

[plars]

@Artur-at-work one other question: could we get a modification to this (and hopefully the previous oemscript also), to allow downloading the iso directly to the DUT and extracting it there? That would help quite a bit. Right now, especially with the previous oemscript method, we have hundreds of these agents which sometimes get triggered all at once. Having all of these 4GB images downloading to the same agent host consumes a LOT of space very quickly. It has sometimes caused some of them to fail because of this. If we could have it pass the url to the script and the script downloads it on the DUT instead of having a local file that gets SCPed that would be really helpful.

[Artur-at-work]

@plars thanks for merging.
I missed one commit, and created a follow up PR here: #331

Regarding downloading the ISO directly to DUT, I've assigned a new issue to myself:
#330
Let me work on that later, after I setup this new device connector in our lab.