Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identity e2e tests for running fail in Azure VM MSI #31814

Closed
v-jiaodi opened this issue Oct 28, 2022 · 15 comments
Closed

Identity e2e tests for running fail in Azure VM MSI #31814

v-jiaodi opened this issue Oct 28, 2022 · 15 comments
Assignees
Labels
Azure.Identity Client This issue points to a problem in the data-plane of the library.
Milestone

Comments

@v-jiaodi
Copy link
Member

Describe the issue:
Test the Testing azure-identity in Azure VM MSI with instructions (Repro steps). When run the MSI test, it displays the error :

image

The maven default version is 3.6.0 in VM. We also tried to upgrade the Maven version, but it is complex, we didn't achieve the expected result.

Environment:
Env: Azure VM
Package: https://search.maven.org/artifact/com.azure/azure-identity/1.7.0-beta.2/jar
Related issue: Azure/azure-sdk#815

Expected behavior:
Run the MSI test can pass.

@josefree for notification.

@ghost ghost added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Oct 28, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Oct 31, 2022
@joshfree joshfree added the Client This issue points to a problem in the data-plane of the library. label Oct 31, 2022
@joshfree
Copy link
Member

@billwert please take a look

@joshfree joshfree added this to the 2022-11 milestone Oct 31, 2022
@billwert
Copy link
Contributor

Working on updated maven install instructions.

@billwert
Copy link
Contributor

billwert commented Nov 1, 2022

@v-jiaodi

Please replace the instructions in the repro steps under Install Java & Maven on VMs for installing Java and maven with this instead:

echo -e `az vm run-command invoke \
    --ids /subscriptions/faa080af-c1d8-40ad-9cce-e1a450ca5b57/resourceGroups/billwertmvntest/providers/Microsoft.Compute/virtualMachines/billwerttestvm4 \
    --command-id RunShellScript \
    --scripts 'sudo apt-get update && sudo apt-get install openjdk-8-jdk -y && wget https://dlcdn.apache.org/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz -P /tmp && sudo tar xf /tmp/apache-maven-3.8.6-bin.tar.gz -C /opt && sudo ln -s /opt/apache-maven-3.8.6/ /opt/maven && echo export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64 | sudo tee -a /usr/local/bin/mvn && echo export M2_HOME=/opt/maven | sudo tee -a /usr/local/bin/mvn && echo export MAVEN_HOME=/opt/maven | sudo tee -a /usr/local/bin/mvn && echo export PATH='\''${M2_HOME}'\''/bin:'\''${PATH}'\'' | sudo tee -a /usr/local/bin/mvn && echo mvn \"\$\@\" | sudo tee -a /usr/local/bin/mvn && sudo chmod +x /usr/local/bin/mvn' \
    -o json \
    --query "value[0].message"`

Let me know if that works and if it does you can replace the instructions in the repro steps. Thanks!

@joshfree @g2vinay What are our thoughts on rolling forward Ubuntu? The fundamental problem here is that 18.04 includes maven 3.6.0. The package lists show that if we move to 20.04 we would get maven 3.6.3 which would let us go back to simply using apt to do this install instead of the thing I have written here.

@v-jiaodi
Copy link
Member Author

v-jiaodi commented Nov 1, 2022

@joshfree , @billwert I tried to follow your steps, but it had some error as follow:

image
image

In addition, I create VM use ubuntu 20.04 according to the python PR#27182. It can solve current issues,but I got the same error as #31812. The error message as follows:

image

@billwert
Copy link
Contributor

billwert commented Nov 1, 2022

Cool. Let's use 20.04 for testing now.

@g2vinay checked in a fix for the issue you're seeing now. We'll get a new package to you.

@billwert
Copy link
Contributor

billwert commented Nov 1, 2022

@v-jiaodi The fix is merged in #31848. Please re-sync the branch and test again.

@v-jiaodi
Copy link
Member Author

v-jiaodi commented Nov 2, 2022

@billwert This error still exists after re-sync the branch and test again. In addition, #31812 still exists when I verified on Cloudshell again.

@billwert
Copy link
Contributor

billwert commented Nov 2, 2022

@v-jiaodi Sorry about that! I've fixed this in my fork in this branch: https://github.com/billwert/azure-sdk-for-java/tree/fix-access-modifier

Can you try testing with this branch please?

@v-jiaodi
Copy link
Member Author

v-jiaodi commented Nov 2, 2022

@billwert I try to test with this branch: https://github.com/billwert/azure-sdk-for-java/tree/fix-access-modifier on Cloudshell again. The error as follow:

Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /home/di/azure-sdk-for-java/sdk/e2e/target/classes
[INFO] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/IdentityTest.java: Some input files use or override a deprecated API.
[INFO] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/IdentityTest.java: Recompile with -Xlint:deprecation for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[79,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[146,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[218,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient
[INFO] 3 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.850 s
[INFO] Finished at: 2022-11-02T05:31:20Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project azure-e2e: Compilation failure: Compilation failure:
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[79,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[146,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient
[ERROR] /home/di/azure-sdk-for-java/sdk/e2e/src/main/java/com/azure/endtoend/identity/WebJobsIdentityTest.java:[218,41] authenticateToManagedIdentityEndpoint(java.lang.String,java.lang.String,java.lang.String,java.lang.String,com.azure.core.credential.TokenRequestContext) has private access in com.azure.identity.implementation.IdentityClient

@billwert
Copy link
Contributor

billwert commented Nov 3, 2022

@v-jiaodi Here's some updated instructions.

The failure you got and shared with me in IM last night was due to a project changed that happened at some point in the pom file. It isn't automatically building dependencies now. To fix that, we'll build the dependencies after the clone but before running the test:

 echo -e `az vm run-command invoke \
    --ids /subscriptions/{subscription}/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Compute/virtualMachines/{vm} \
    --command-id RunShellScript \
    --scripts "cd /tmp/azure-sdk-for-java && export AZURE_VAULT_URL=https://$KEY_VAULT_NAME.vault.azure.net && mvn install -pl sdk/identity/azure-identity,sdk/core/azure-core,sdk/core/azure-core-http-netty,sdk/keyvault/azure-security-keyvault-keys,sdk/keyvault/azure-security-keyvault-secrets,sdk/keyvault/azure-security-keyvault-certificates,sdk/core/azure-core-test -T 1 -f pom.xml -Dgpg.skip -Dmaven.javadoc.skip -Drevapi.skip -DskipSpringITs -DskipTests -Dspotbugs.skip -Djacoco.skip -am -DfailIfNoTests=false" \                                                                                                                   -o json \
    --query "value[0].message"`

Then we can run the test (for example the system assigned):

echo -e `az vm run-command invoke \
    --ids /subscriptions/{subscription}/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Compute/virtualMachines/{vm} \
    --command-id RunShellScript \
    --scripts "cd /tmp/azure-sdk-for-java/sdk/e2e && export AZURE_VAULT_URL=https://$KEY_VAULT_NAME.vault.azure.net && mvn test -Dtest=ManagedIdentityCredentialLiveTest#testIMDSEndpointWithSystemAssigned*  -f pom.xml -Dgpg.skip -am -DfailIfNoTests=false" \                                                                                                                                                                      -o json \
    --query "value[0].message"`

Note the changes to the test command line: removal of -am, changed directory to include sdk/e2e, removal of -pl sdk/e2e.

With these changes I was able to successfully get a passing test with no changes to the code in main branch. You can go back to using azure/azure-sdk-for-java instead of any fork.

I'll consult with folks to figure out how we fix this going forward but this should unblock you for now. Thank you for your help!

@g2vinay
Copy link
Member

g2vinay commented Nov 3, 2022

@v-jiaodi

I merged a PR to unblock the test setup.
Can you run the cloudshell test again from source code on main branch ?

This is the output I see for Cloudshell tests:
Screen Shot 2022-11-02 at 1 42 12 PM

If you still see same issue, then follow updated instructions here for cloudshell testing. Azure/azure-sdk#795 (comment)

Can you also try Web jobs with updated source code ?

Thank you.

@v-jiaodi
Copy link
Member Author

v-jiaodi commented Nov 3, 2022

@billwert According to the updated instructions above, I tried to test on the VM again, and it worked normally. Please help update the instructions .

@Azure Azure deleted a comment from v-jiaodi Nov 4, 2022
@billwert
Copy link
Contributor

billwert commented Nov 4, 2022

@v-jiaodi I'll get the update done. Thanks for letting me know it worked.

@billwert
Copy link
Contributor

billwert commented Nov 5, 2022

I'm removing blocking-release as this test is now passing and updating the instructions is not release blocking.

@billwert billwert removed the blocking-release Blocks release label Nov 5, 2022
@v-jiaodi
Copy link
Member Author

v-jiaodi commented Mar 10, 2023

We have updated the instructions . We will close this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Jun 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Azure.Identity Client This issue points to a problem in the data-plane of the library.
Projects
Development

No branches or pull requests

4 participants