Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: split aws credentials in two separate users with scoped perms #64576

Merged
merged 1 commit into from
Sep 20, 2019
Merged

ci: split aws credentials in two separate users with scoped perms #64576

merged 1 commit into from
Sep 20, 2019

Conversation

pietroalbini
Copy link
Member

This commit changes our CI to use two separate IAM users to authenticate with AWS:

  • ci--rust-lang--rust--sccache: has access to the rust-lang-ci-sccache2 S3 bucket and its credentials are available during the whole build.
  • ci--rust-lang--rust--upload: has access to the rust-lang-ci2 S3 bucket and its credentials are available just during the upload step.

The new tokens are available in the prod-credentials library.

r? @alexcrichton

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 18, 2019
@pietroalbini
Copy link
Member Author

@rustbot modify labels: beta-nominated beta-accepted T-infra

Accepting this for beta backport on my own as if this is merged we're going to need the new credentials on beta as well.

@rustbot rustbot added beta-accepted Accepted for backporting to the compiler in the beta channel. beta-nominated Nominated for backporting to the compiler in the beta channel. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Sep 18, 2019
@alexcrichton
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Sep 18, 2019

📌 Commit 00c44af has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 18, 2019
@bors
Copy link
Contributor

bors commented Sep 18, 2019

⌛ Testing commit 00c44af with merge ea6044e018d3eac7182b45406ceee95ece8f71bc...

@bors
Copy link
Contributor

bors commented Sep 18, 2019

💔 Test failed - checks-azure

@rust-highfive
Copy link
Collaborator

Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log. 
2019-09-18T16:33:02.3158260Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/aa213ad5-69d3-43d5-8f8e-5f82dc8d87de.sh
2019-09-18T16:33:02.3229971Z Cloning into 'rust-toolstate'...
2019-09-18T16:33:03.1059997Z <Nothing changed>
2019-09-18T16:33:03.3141365Z Traceback (most recent call last):
2019-09-18T16:33:03.3142319Z   File "/home/vsts/work/1/s/src/tools/publish_toolstate.py", line 267, in <module>
2019-09-18T16:33:03.3142435Z     validate_maintainers(repo, github_token)
2019-09-18T16:33:03.3142896Z   File "/home/vsts/work/1/s/src/tools/publish_toolstate.py", line 73, in validate_maintainers
2019-09-18T16:33:03.3144083Z     'Accept': 'application/vnd.github.hellcat-preview+json',
2019-09-18T16:33:03.3144194Z   File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
2019-09-18T16:33:03.3233991Z     return opener.open(url, data, timeout)
2019-09-18T16:33:03.3234470Z   File "/usr/lib/python2.7/urllib2.py", line 435, in open
2019-09-18T16:33:03.3234600Z     response = meth(req, response)
2019-09-18T16:33:03.3234780Z   File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
2019-09-18T16:33:03.3235490Z     'http', request, response, code, msg, hdrs)
2019-09-18T16:33:03.3235616Z   File "/usr/lib/python2.7/urllib2.py", line 473, in error
2019-09-18T16:33:03.3235750Z     return self._call_chain(*args)
2019-09-18T16:33:03.3235835Z   File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
2019-09-18T16:33:03.3237885Z     result = func(*args)
2019-09-18T16:33:03.3238204Z   File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
2019-09-18T16:33:03.3238853Z     raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
2019-09-18T16:33:03.3241458Z urllib2.HTTPError: HTTP Error 401: Unauthorized
2019-09-18T16:33:03.3432349Z ##[error]Bash exited with code '1'.
2019-09-18T16:33:03.3478811Z ##[section]Starting: Upload CPU usage statistics
2019-09-18T16:33:03.3481676Z ==============================================================================
2019-09-18T16:33:03.3481759Z Task         : Bash
2019-09-18T16:33:03.3481874Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 18, 2019
@pietroalbini
Copy link
Member Author

@bors retry

Copied the wrong GitHub token, woops! Updated the variable group on the Azure Pipelines side.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 18, 2019
Centril added a commit to Centril/rust that referenced this pull request Sep 18, 2019
@Centril
Rollup merge of rust-lang#64576 - pietroalbini:split-aws-tokens, r=al…
aac0301 
…excrichton

ci: split aws credentials in two separate users with scoped perms

This commit changes our CI to use two separate IAM users to authenticate with AWS:

* `ci--rust-lang--rust--sccache`: has access to the `rust-lang-ci-sccache2` S3 bucket and its credentials are available during the whole build.
* `ci--rust-lang--rust--upload`: has access to the `rust-lang-ci2` S3 bucket and its credentials are available just during the upload step.

The new tokens are available in the `prod-credentials` library.

r? @alexcrichton
@Centril Centril mentioned this pull request Sep 18, 2019
Closed
@Centril
Copy link
Contributor

Centril commented Sep 19, 2019

Failed in #64594 (comment), @bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 19, 2019
@pietroalbini
Copy link
Member Author

@bors rollup=never

@pietroalbini
ci: split aws credentials in two separate users with scoped perms
5384a19 
This commit changes our CI to use two separate IAM users to
authenticate with AWS:

* ci--rust-lang--rust--sccache: has access to the rust-lang-ci-sccache2
  S3 bucket and its credentials are available during the whole build.
* ci--rust-lang--rust--upload: has access to the rust-lang-ci2 S3 bucket
  and its credentials are available just during the upload step.

The new tokens are available in the `prod-credentials` library.
@pietroalbini
Copy link
Member Author

Forgot to pass the new credentials to a step.

@bors r=alexcrichton rollup=never

@bors
Copy link
Contributor

bors commented Sep 19, 2019

📌 Commit 5384a19 has been approved by alexcrichton

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 19, 2019
@pietroalbini
Copy link
Member Author

@bors p=1 to get this into 1.39.

@bors
Copy link
Contributor

bors commented Sep 20, 2019

⌛ Testing commit 5384a19 with merge ea3ba36...

bors added a commit that referenced this pull request Sep 20, 2019
@bors
Auto merge of #64576 - pietroalbini:split-aws-tokens, r=alexcrichton
ea3ba36 
ci: split aws credentials in two separate users with scoped perms

This commit changes our CI to use two separate IAM users to authenticate with AWS:

* `ci--rust-lang--rust--sccache`: has access to the `rust-lang-ci-sccache2` S3 bucket and its credentials are available during the whole build.
* `ci--rust-lang--rust--upload`: has access to the `rust-lang-ci2` S3 bucket and its credentials are available just during the upload step.

The new tokens are available in the `prod-credentials` library.

r? @alexcrichton
@bors
Copy link
Contributor

bors commented Sep 20, 2019

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing ea3ba36 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 20, 2019
@bors bors merged commit 5384a19 into rust-lang:master Sep 20, 2019
@Mark-Simulacrum Mark-Simulacrum removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Sep 21, 2019
@Mark-Simulacrum Mark-Simulacrum mentioned this pull request Sep 21, 2019
Merged
bors added a commit that referenced this pull request Sep 21, 2019
@bors
Auto merge of #64661 - Mark-Simulacrum:beta-next, r=Mark-Simulacrum
1c65325 
[beta] rollup of last set of backports

* ci: split aws credentials in two separate users with scoped perms #64576
* Updated RELEASES.md for 1.38.0 #64283
* Add Compatibility Notes to RELEASES.md for 1.38.0 #64621
@pietroalbini pietroalbini deleted the split-aws-tokens branch November 7, 2019 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@alexcrichton alexcrichton

Labels
beta-accepted Accepted for backporting to the compiler in the beta channel. merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@pietroalbini @alexcrichton @bors @rust-highfive @Centril @Mark-Simulacrum @rustbot