-
Notifications
You must be signed in to change notification settings - Fork 6.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
compute.oslogin.service_account_ssh_test: test_main failed #7277
Comments
This should be resolved now that b/211071739 is resolved |
Looks like this issue is flaky. 😟 I'm going to leave this open and stop commenting. A human should fix and close this. commit: 32950b9 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 97, in test_main ssh_login() File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/retrying.py", line 212, in call raise attempt.get() File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/six.py", line 719, in reraise raise value File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) File "/workspace/compute/oslogin/service_account_ssh_test.py", line 90, in ssh_login main(cmd, project, test_id, zone, oslogin, account, hostname) File "/workspace/compute/oslogin/service_account_ssh.py", line 116, in main private_key_file = create_ssh_key(oslogin, account) File "/workspace/compute/oslogin/service_account_ssh.py", line 79, in create_ssh_key oslogin.users().importSshPublicKey(parent=account, body=body).execute() File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper return wrapped(*args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/googleapiclient/http.py", line 922, in execute resp, content = _retry_request( File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/googleapiclient/http.py", line 190, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/google_auth_httplib2.py", line 245, in request self.credentials.refresh(self._request) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/google/oauth2/service_account.py", line 410, in refresh access_token, expiry, _ = _client.jwt_grant( File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/google/oauth2/_client.py", line 193, in jwt_grant response_data = _token_endpoint_request(request, token_uri, body) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/google/oauth2/_client.py", line 165, in _token_endpoint_request _handle_error_response(response_data) File "/workspace/compute/oslogin/.nox/py-3-10/lib/python3.10/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response raise exceptions.RefreshError(error_details, response_data) google.auth.exceptions.RefreshError: ('invalid_grant: Invalid grant: account not found', {'error': 'invalid_grant', 'error_description': 'Invalid grant: account not found'}) |
Hi @m-strzelczyk - I need some help debugging this. Please reach out if you need any help - it may have to do with the default network being recreated. I'll ping you with a way to request more permissions if needed |
Seems like the Jenkins account is missing the |
I have made pull request, to play around with the test a bit (#7370 ), but it turns out it works quite fine there... I have noticed that the error we got ( I'm starting to think there's not much I could do here, as the test seems to work and the reason it failed has just went away 🤔 |
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: e7379a8 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 62, in test_main service_account_key = setup_resources( File "/workspace/compute/oslogin/service_account_ssh_test.py", line 211, in setup_resources compute.instances().setIamPolicy( File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper return wrapped(*args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/http.py", line 937, in execute raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: |
* WIP: Triggering a new PR to have a stage for running tests and experimenting. * fix(ssh-test): Trying to fix #7277 * Import reordering
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: 884d81d Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 97, in test_main ssh_login() File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/retrying.py", line 212, in call raise attempt.get() File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/six.py", line 719, in reraise raise value File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) File "/workspace/compute/oslogin/service_account_ssh_test.py", line 90, in ssh_login main(cmd, project, test_id, zone, oslogin, account, hostname) File "/workspace/compute/oslogin/service_account_ssh.py", line 131, in main profile = oslogin.users().getLoginProfile(name=account).execute() File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper return wrapped(*args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/http.py", line 922, in execute resp, content = _retry_request( File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/http.py", line 190, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/google_auth_httplib2.py", line 245, in request self.credentials.refresh(self._request) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/google/oauth2/service_account.py", line 410, in refresh access_token, expiry, _ = _client.jwt_grant( File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/google/oauth2/_client.py", line 193, in jwt_grant response_data = _token_endpoint_request(request, token_uri, body) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/google/oauth2/_client.py", line 165, in _token_endpoint_request _handle_error_response(response_data) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response raise exceptions.RefreshError(error_details, response_data) google.auth.exceptions.RefreshError: ('invalid_grant: Invalid grant: account not found', {'error': 'invalid_grant', 'error_description': 'Invalid grant: account not found'}) |
* WIP: Triggering a new PR to have a stage for running tests and experimenting. * fix(ssh-test): Trying to fix #7277 * Import reordering * fix(compute-ssh): Another fix for #7277 * Changing debian from 9 to 11 * Fixing something. * Testing things. Co-authored-by: Vadym Matsishevskyi <25311427+vam-google@users.noreply.github.com>
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: b868f40 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 62, in test_main service_account_key = setup_resources( File "/workspace/compute/oslogin/service_account_ssh_test.py", line 198, in setup_resources operation = compute.instances().insert( File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper return wrapped(*args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-9/lib/python3.9/site-packages/googleapiclient/http.py", line 937, in execute raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: |
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: 1cd2422 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 97, in test_main ssh_login() File "/workspace/compute/oslogin/.nox/py-3-8/lib/python3.8/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/workspace/compute/oslogin/.nox/py-3-8/lib/python3.8/site-packages/retrying.py", line 212, in call raise attempt.get() File "/workspace/compute/oslogin/.nox/py-3-8/lib/python3.8/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/workspace/compute/oslogin/.nox/py-3-8/lib/python3.8/site-packages/six.py", line 719, in reraise raise value File "/workspace/compute/oslogin/.nox/py-3-8/lib/python3.8/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) File "/workspace/compute/oslogin/service_account_ssh_test.py", line 93, in ssh_login assert assert_value in out AssertionError: assert 'Linux oslogin-test-525107' in "Creating key users/oslogin-test-525107@python-docs-samples-tests.iam.gserviceaccount.com and {'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaw2VfkFEi5lx/dV123SeFcLpTosbkFqFY+UOTCGi0qf+VOqP6b7QnmTmS+LQzDUsfpliShu3K6f+deifnRK1vEhS5m9h0rKwdjpWupRGQWjIelmen4GOQ8EKIXKdgOrNAcdzLHllfIvOi21lKennhdqirdTYgg5pOfdnyP5FcWPnVYbjuntgvBs6TTWs4FL+qRhAZNvaZjUecJ5d387aBUAJFAeHLmTP9rfakYRP3w0aJ93BvqdQ0SGz9vPbH0TePd8e7LmU696O5/4VkqEf9Jrs0CpRzDYhQSuhhsNab2zYdLPqAVk4fOFvlWaCP7KjuRRYs5aA7Fpn/Oh7e2kX7Q58NGdxhZyb3AZxMlryLI6vYxGgl2R6y60ZrqbZl10gTo+ir79RTczi2mBrWGmA/yS2NhFCNHcvwTUurT9MoRdyt6nhjsKDdEHACfmpiLPoOUokeWjhcJmZSDZtiBb1K5oFpWiuR/a1y8K6rQf+ldlec7NJ8GS67P9a9QEZmw80= kbuilder@kokoro-gcp-ubuntu-prod-750669605', 'expirationTimeUsec': 1660131791927780}\nssh: connect to host 34.138.210.107 port 22: Connection timed out\n" |
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: 54c72b8 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 97, in test_main ssh_login() File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/retrying.py", line 212, in call raise attempt.get() File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/six.py", line 719, in reraise raise value File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) File "/workspace/compute/oslogin/service_account_ssh_test.py", line 93, in ssh_login assert assert_value in out AssertionError: assert 'Linux oslogin-test-115031' in "Creating key users/oslogin-test-115031@python-docs-samples-tests.iam.gserviceaccount.com and {'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbJRHSGBlt3CRWbQy97a/HFURfT8nKOlr+xrQccJR2ULgCaTN5eUPXXFEAxlWI1HC6EP9m02hUrmjpanHINbhsWcxPKscsDjpt4bdBk44mkTVBEST6jx3OIpjIMRHweVqA59WIjLkrYfSQkVJdE/5YDljrYAtv3FLVJzBZwiQtvEr6y9zIgxrOu2q80+dcxyzhMtqtbJKtlfta6lZbJIrUcb3MUtoGoVSyanaQ0dZaUTepNrm9TlGEoxLbT8L0xPampher1A47V2D2rMFj1DVCsXytyXPUlCMBH9vPgSCSbrVwAo8p47kiSTsw79nE0GjTYERWayZMY3mKtb4sr64IZ5W5FuwpD4BNvlB0X9QyvnU0/A0XedssSiz+T1P/nafeF2D4Sp+gEa0Ck/XBfWgxi1Z8UdSdussJJ0RHkCqNrGsgEzrrBHJbgQVJeSHa/oH352TeTmx1fJYKVw0AG5QHQsLSMM6WgJzZTP3msjUM+GYtR+m69ZdBN29tbElTl3k= kbuilder@kokoro-gcp-ubuntu-prod-431126807', 'expirationTimeUsec': 1660246238189151}\nssh: connect to host 34.148.5.215 port 22: Connection timed out\n" |
Oops! Looks like this issue is still flaky. It failed again. 😬 I reopened the issue, but a human will need to close it again. commit: 4762700 Test outputTraceback (most recent call last): File "/workspace/compute/oslogin/service_account_ssh_test.py", line 66, in test_main machine_type, account_email) File "/workspace/compute/oslogin/service_account_ssh_test.py", line 225, in setup_resources 'role': 'roles/compute.osLogin' File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper return wrapped(*args, **kwargs) File "/workspace/compute/oslogin/.nox/py-3-7/lib/python3.7/site-packages/googleapiclient/http.py", line 937, in execute raise HttpError(resp, content, uri=self.uri) googleapiclient.errors.HttpError: |
Fwiw this test is also leaving leftovers - I just cleaned a bunch up in the test project because we had VM leftovers giving us quota issues |
Hi @leahecole, do you happen to have any more details on which resources are left over? cleanup_resources() looks like it deletes several types of resources. It's possible that these leftovers are caused when an exception is thrown, or the cleanup code is missing resource types. |
Note: #6894 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.
commit: 5ceff25
buildURL: Build Status, Sponge
status: failed
Test output
The text was updated successfully, but these errors were encountered: