Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use a custom android sdk manager with pinning and mirroring #59926

Merged
merged 1 commit into from
Apr 15, 2019

Conversation

pietroalbini
Copy link
Member

Google's own sdkmanager has two issues that make it unsuitable for us:

  • Mirroring has to be done manually, which is annoying because we need to figure out on our own all the URLs to copy (I couldn't find any documentation when building this PR, had to use mitmproxy).
  • There is no support for pinning, which means an update on Google's side can break our CI, as it happened multiple times.

This PR replaces all our usage of sdkmanager with a custom Python script which mimics its behavior, but with the two issues fixes.

sdkmanager's logic for installing packages is thankfully very simple: the package name (like system-images;android-18;default;armeabi-v7a) is the directory where the package should live (with ; replaced with /), so to install a package we only need to extract its contents in the right directory.

r? @alexcrichton
cc @kennytm
fixes #59778

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 12, 2019
@alexcrichton
Copy link
Member

Could you describe a bit what the process would look like to upgrade these images? (ideally in comments in the code as well!)

I'd just want to make sure that's documented somewhere and otherwise lgtm

@pietroalbini
Copy link
Member Author

There is already a bit of docs in the comments. Should I expand it or move it somewhere else?

The gist of it (for example to update the emulator) is:

src/ci/docker/scripts/android-sdk-manager.py add-to-lockfile src/ci/docker/arm-android/android-sdk.lock emulator
src/ci/docker/scripts/android-sdk-manager.py update-mirror src/ci/docker/arm-android/android-sdk.lock

@alexcrichton
Copy link
Member

Yeah I was wondering how we'd update all the hashes, but I see the words now and that seems reasonable to me, sorry for the oversight!

@bors: r+

@bors
Copy link
Contributor

bors commented Apr 15, 2019

📌 Commit 4e920f2 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 15, 2019
@pietroalbini
Copy link
Member Author

Do not rollup this.

@Centril
Copy link
Contributor

Centril commented Apr 15, 2019

@bors p=2

Putting behind a r0llup and its constituent PRs.

@bors
Copy link
Contributor

bors commented Apr 15, 2019

⌛ Testing commit 4e920f2 with merge 2975a3c...

bors added a commit that referenced this pull request Apr 15, 2019
ci: use a custom android sdk manager with pinning and mirroring

Google's own sdkmanager has two issues that make it unsuitable for us:

* Mirroring has to be done manually, which is annoying because we need to figure out on our own all the URLs to copy (I couldn't find any documentation when building this PR, had to use mitmproxy).
* There is no support for pinning, which means an update on Google's side can break our CI, as it happened multiple times.

This PR replaces all our usage of sdkmanager with a custom Python script which mimics its behavior, but with the two issues fixes.

sdkmanager's logic for installing packages is thankfully very simple: the package name (like `system-images;android-18;default;armeabi-v7a`) is the directory where the package should live (with `;` replaced with `/`), so to install a package we only need to extract its contents in the right directory.

r? @alexcrichton
cc @kennytm
fixes #59778
@bors
Copy link
Contributor

bors commented Apr 15, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: alexcrichton
Pushing 2975a3c to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 15, 2019
@bors bors merged commit 4e920f2 into rust-lang:master Apr 15, 2019
@pietroalbini pietroalbini deleted the android-sdk-manager branch April 16, 2019 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate pinning Android SDK components inside CI
5 participants