az sql mi-arc list failes after update to 2.36.0

[rfsonders]

This is autogenerated. Please review and update as needed.

Describe the bug

I just updated to 2.36.0
Now...
PS C:\Users\Rob> az sql mi-arc list -k arc-services-ns --use-k8s
Unable to load extension 'arcdata: No module named 'azure.identity''. Use --debug for more information.
'mi-arc' is misspelled or not recognized by the system.

no other updates available
Command Name
az sql

Errors:

'mi-arc' is misspelled or not recognized by the system.

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az sql {} {} -k {} --use-k8s --debug

Expected Behavior

Environment Summary

Windows-10-10.0.19044-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.36.0

Extensions:
arcdata 1.3.1
connectedk8s 1.2.8
customlocation 0.1.3
k8s-configuration 1.5.1
k8s-extension 1.2.0

Dependencies:
msal 1.17.0
azure-mgmt-resource 20.0.0

Additional Context

PS C:\Users\Rob> az sql mi-arc list -k arc-services-ns --use-k8s --debug
cli.knack.cli: Command arguments: ['sql', 'mi-arc', 'list', '-k', 'arc-services-ns', '--use-k8s', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x024F8BB0>, <function OutputProducer.on_global_arguments at 0x025BD4F0>, <function CLIQuery.on_global_arguments at 0x025DB148>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'sql': ['azure.cli.command_modules.sql', 'azure.cli.command_modules.sqlvm']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: sql 0.015 45 178
cli.azure.cli.core: sqlvm 0.005 4 17
cli.azure.cli.core: Total (2) 0.020 49 195
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: Loaded 48 groups, 195 commands.
cli.azure.cli.core: Could not find a match in the command or command group table for 'sql mi-arc list'. The index may be outdated.
cli.azure.cli.core: Loading all modules and extensions
cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'config', 'configure', 'consumption', 'container', 'cosmosdb', 'databoxedge', 'deploymentmanager', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'natgateway', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: acr 0.011 30 126
cli.azure.cli.core: acs 0.253 10 64
cli.azure.cli.core: advisor 0.003 3 6
cli.azure.cli.core: ams 0.008 22 100
cli.azure.cli.core: apim 0.010 10 56
cli.azure.cli.core: appconfig 0.005 7 37
cli.azure.cli.core: appservice 0.021 66 240
cli.azure.cli.core: aro 0.008 1 7
cli.azure.cli.core: backup 0.006 15 55
cli.azure.cli.core: batch 0.045 33 96
cli.azure.cli.core: batchai 0.005 10 30
cli.azure.cli.core: billing 0.013 19 52
cli.azure.cli.core: botservice 0.006 12 42
cli.azure.cli.core: cdn 0.010 39 133
cli.azure.cli.core: cloud 0.003 1 7
cli.azure.cli.core: cognitiveservices 0.004 8 31
cli.azure.cli.core: config 0.003 2 7
cli.azure.cli.core: configure 0.003 3 9
cli.azure.cli.core: consumption 0.005 8 9
cli.azure.cli.core: container 0.004 1 11
cli.azure.cli.core: cosmosdb 0.014 47 167
cli.azure.cli.core: databoxedge 0.008 5 27
cli.azure.cli.core: deploymentmanager 0.004 7 30
cli.azure.cli.core: dla 0.006 23 62
cli.azure.cli.core: dls 0.005 7 41
cli.azure.cli.core: dms 0.003 3 22
cli.azure.cli.core: eventgrid 0.007 18 61
cli.azure.cli.core: eventhubs 0.007 18 61
cli.azure.cli.core: extension 0.002 1 7
cli.azure.cli.core: feedback 0.002 1 1
cli.azure.cli.core: find 0.003 1 1
cli.azure.cli.core: hdinsight 0.004 8 39
cli.azure.cli.core: identity 0.003 1 5
cli.azure.cli.core: interactive 0.001 1 1
cli.azure.cli.core: iot 0.057 17 74
cli.azure.cli.core: keyvault 0.012 20 122
cli.azure.cli.core: kusto 0.004 3 14
cli.azure.cli.core: lab 0.006 11 34
cli.azure.cli.core: managedservices 0.003 3 8
cli.azure.cli.core: maps 0.003 5 13
cli.azure.cli.core: marketplaceordering 0.007 1 2
cli.azure.cli.core: monitor 0.013 34 139
cli.azure.cli.core: natgateway 0.004 3 6
cli.azure.cli.core: netappfiles 0.009 16 69
cli.azure.cli.core: network 0.106 141 653
cli.azure.cli.core: policyinsights 0.004 6 12
cli.azure.cli.core: privatedns 0.006 14 66
cli.azure.cli.core: profile 0.003 2 9
cli.azure.cli.core: rdbms 0.215 46 199
cli.azure.cli.core: redis 0.004 5 27
cli.azure.cli.core: relay 0.005 10 37
cli.azure.cli.core: reservations 0.004 5 12
cli.azure.cli.core: resource 0.016 43 197
cli.azure.cli.core: role 0.004 16 56
cli.azure.cli.core: search 0.004 7 22
cli.azure.cli.core: security 0.007 44 94
cli.azure.cli.core: servicebus 0.007 19 68
cli.azure.cli.core: serviceconnector 0.058 4 53
cli.azure.cli.core: servicefabric 0.006 26 75
cli.azure.cli.core: signalr 0.004 6 20
cli.azure.cli.core: sql 0.009 45 178
cli.azure.cli.core: sqlvm 0.001 4 17
cli.azure.cli.core: storage 0.052 56 260
cli.azure.cli.core: synapse 0.019 52 231
cli.azure.cli.core: util 0.002 2 4
cli.azure.cli.core: vm 0.030 54 279
cli.azure.cli.core: Total (66) 1.178 1161 4693
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.knack.cli: Event: MainLoader.OnFailedExtensionLoad []
cli.azure.cli.core: Unable to load extension 'arcdata: No module named 'azure.identity''. Use --debug for more information.
cli.azure.cli.core: Traceback (most recent call last):
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/init.py", line 359, in _update_command_table_from_extensions
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 1076, in _load_extension_command_loader
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 1062, in load_command_loader
File "C:\Users\Rob.azure\cliextensions\arcdata\azext_arcdata_init.py", line 24, in load_command_table
from azext_arcdata.dc.commands import load_commands as dc_commands
File "C:\Users\Rob.azure\cliextensions\arcdata\azext_arcdata\dc\commands.py", line 7, in
from azext_arcdata.arm_sdk.dc.client import beget
File "C:\Users\Rob.azure\cliextensions\arcdata\azext_arcdata\arm_sdk\dc\client.py", line 7, in
from azext_arcdata.arm_sdk.azure.azure_resource_client import (
File "C:\Users\Rob.azure\cliextensions\arcdata\azext_arcdata\arm_sdk\azure\azure_resource_client.py", line 27, in
from azure.identity._credentials.azure_cli import AzureCliCredential
ModuleNotFoundError: No module named 'azure.identity'

cli.azure.cli.core: connectedk8s 0.007 1 9 C:\Users\Rob.azure\cliextensions\connectedk8s
cli.azure.cli.core: customlocation 0.034 1 7 C:\Users\Rob.azure\cliextensions\customlocation
cli.azure.cli.core: k8s-configuration 0.008 4 16 C:\Users\Rob.azure\cliextensions\k8s-configuration
cli.azure.cli.core: k8s-extension 0.007 1 5 C:\Users\Rob.azure\cliextensions\k8s-extension
cli.azure.cli.core: Total (4) 0.056 7 37
cli.azure.cli.core: Loaded 1157 groups, 4730 commands.
cli.azure.cli.core: Updated command index in 0.007 seconds.
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x0496B070>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\Rob.azure\commands\2022-04-28.19-57-51.sql.7432.log'.
az_command_data_logger: command args: sql {} {} -k {} --use-k8s --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x0490DB20>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x049B7100>, <function register_cache_arguments..add_cache_arguments at 0x049B7190>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x0554F8E0>]
urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 1726670
cli.azure.cli.core.extension._resolve: Candidates ['arcdata-1.3.1-py2.py3-none-any.whl', 'arcdata-1.3.0-py2.py3-none-any.whl', 'arcdata-1.2.3-py2.py3-none-any.whl', 'arcdata-1.2.2-py2.py3-none-any.whl', 'arcdata-1.2.1-py2.py3-none-any.whl', 'arcdata-1.2.0-py2.py3-none-any.whl', 'arcdata-1.1.3-py2.py3-none-any.whl', 'arcdata-1.1.2-py2.py3-none-any.whl', 'arcdata-1.1.1-py2.py3-none-any.whl', 'arcdata-1.1.0-py2.py3-none-any.whl', 'arcdata-1.0.0-py2.py3-none-any.whl', 'arcdata-0.0.2-py2.py3-none-any.whl', 'arcdata-0.0.1-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['arcdata-1.3.1-py2.py3-none-any.whl', 'arcdata-1.3.0-py2.py3-none-any.whl', 'arcdata-1.2.3-py2.py3-none-any.whl', 'arcdata-1.2.2-py2.py3-none-any.whl', 'arcdata-1.2.1-py2.py3-none-any.whl', 'arcdata-1.2.0-py2.py3-none-any.whl', 'arcdata-1.1.3-py2.py3-none-any.whl', 'arcdata-1.1.2-py2.py3-none-any.whl', 'arcdata-1.1.1-py2.py3-none-any.whl', 'arcdata-1.1.0-py2.py3-none-any.whl', 'arcdata-1.0.0-py2.py3-none-any.whl', 'arcdata-0.0.2-py2.py3-none-any.whl', 'arcdata-0.0.1-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['arcdata-1.3.1-py2.py3-none-any.whl', 'arcdata-1.3.0-py2.py3-none-any.whl', 'arcdata-1.2.3-py2.py3-none-any.whl', 'arcdata-1.2.2-py2.py3-none-any.whl', 'arcdata-1.2.1-py2.py3-none-any.whl', 'arcdata-1.2.0-py2.py3-none-any.whl', 'arcdata-1.1.3-py2.py3-none-any.whl', 'arcdata-1.1.2-py2.py3-none-any.whl', 'arcdata-1.1.1-py2.py3-none-any.whl', 'arcdata-1.1.0-py2.py3-none-any.whl', 'arcdata-1.0.0-py2.py3-none-any.whl', 'arcdata-0.0.2-py2.py3-none-any.whl', 'arcdata-0.0.1-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.3.1-py2.py3-none-any.whl' as 1.3.1 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.3.0-py2.py3-none-any.whl' as 1.3.0 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.2.3-py2.py3-none-any.whl' as 1.2.3 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.2.2-py2.py3-none-any.whl' as 1.2.2 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.2.1-py2.py3-none-any.whl' as 1.2.1 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.2.0-py2.py3-none-any.whl' as 1.2.0 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.1.3-py2.py3-none-any.whl' as 1.1.3 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.1.2-py2.py3-none-any.whl' as 1.1.2 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.1.1-py2.py3-none-any.whl' as 1.1.1 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.1.0-py2.py3-none-any.whl' as 1.1.0 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.0.0-py2.py3-none-any.whl' as 1.0.0 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-0.0.2-py2.py3-none-any.whl' as 0.0.2 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-0.0.1-py2.py3-none-any.whl' as 0.0.1 not greater than current version 1.3.1
cli.azure.cli.core.command_recommender: "-k" is an invalid parameter for command "sql mi-arc".
cli.azure.cli.core.command_recommender: "--use-k8s" is an invalid parameter for command "sql mi-arc".
urllib3.connectionpool: Starting new HTTPS connection (1): app.aladdin.microsoft.com:443
urllib3.connectionpool: https://app.aladdin.microsoft.com:443 "GET /api/v1.0/suggestions?query=%7B%22command%22%3A+%22sql+mi-arc%22%2C+%22parameters%22%3A+%22%22%7D&clientType=AzureCli&context=%7B%22versionNumber%22%3A+%222.36.0%22%2C+%22errorType%22%3A+%22UnknownSubcommand%22%2C+%22correlationId%22%3A+%221bdb60ad-e441-4ee6-8d0c-545aca2c78ac%22%2C+%22subscriptionId%22%3A+%22047d5a79-16ab-4705-afe1-9139b0cbe512%22%2C+%22eventId%22%3A+%225333da74-ef0e-41d0-b53e-29c24def29d0%22%7D HTTP/1.1" 200 None
cli.azure.cli.core.command_recommender: "-k" is an invalid parameter for command "sql mi-arc".
cli.azure.cli.core.command_recommender: "--use-k8s" is an invalid parameter for command "sql mi-arc".
cli.azure.cli.core.azclierror: 'mi-arc' is misspelled or not recognized by the system.
az_command_data_logger: 'mi-arc' is misspelled or not recognized by the system.

[yonzhan]

route to CXP team

[navba-MSFT]

@rfsonders Thanks for reaching out to us and sharing this feedback. I was able to reproduce this issue with AzCLI 2.36.0. However it worked fine on the 2.35.0. Below are my findings, while debugging this issue:

File "/root/.azure/cliextensions/arcdata/azext_arcdata/init.py", line 24, in load_command_table
from azext_arcdata.dc.commands import load_commands as dc_commands
File "/root/.azure/cliextensions/arcdata/azext_arcdata/dc/commands.py", line 7, in
from azext_arcdata.arm_sdk.dc.client import beget
File "/root/.azure/cliextensions/arcdata/azext_arcdata/arm_sdk/dc/client.py", line 7, in
from azext_arcdata.arm_sdk.azure.azure_resource_client import (
File "/root/.azure/cliextensions/arcdata/azext_arcdata/arm_sdk/azure/azure_resource_client.py", line 27, in
from azure.identity._credentials.azure_cli import AzureCliCredential
ModuleNotFoundError: No module named 'azure.identity'

The cause of the above error is:

Azure CLI has removed the dependency on azure-identity (#22124). This issue is now pending on Azure/azure-sdk-for-python#23932 to be released.

@jiasli @yonzhan for awareness.

Regarding the below error, I am checking the details. In the release notes of 2.36.0, I don't see any breaking changes for the az sql mi-arc list operation. I will get back once I have more details:

cli.azure.cli.core.extension._resolve: Skipping 'arcdata-1.0.0-py2.py3-none-any.whl' as 1.0.0 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-0.0.2-py2.py3-none-any.whl' as 0.0.2 not greater than current version 1.3.1
cli.azure.cli.core.extension._resolve: Skipping 'arcdata-0.0.1-py2.py3-none-any.whl' as 0.0.1 not greater than current version 1.3.1
cli.azure.cli.core.command_recommender: "-k" is an invalid parameter for command "sql mi-arc".
cli.azure.cli.core.command_recommender: "--use-k8s" is an invalid parameter for command "sql mi-arc".

[mdrakiburrahman]

I just hit this with azure cli 2.36.0-1~focal as well.

@navba-MSFT I think the issue is with Azure CLI, not arcdata (which implements mi-arc). I tried installing an older version of the arcdata extension and still hit the issue with latest azure cli.

However, the previous version of azure cli works fine with the latest arcdata.

@rfsonders Workaround - you might try to use the versions in this dockerfile - it works fine for me for Arc MI:
https://github.com/mdrakiburrahman/microk8s-arc/blob/e137fbcc6931e96a609269443ce94ca11373f899/.devcontainer/Dockerfile#L27

[navba-MSFT]

@mdrakiburrahman Thanks for sharing this. I am working with the product owners and I will update once I get more details from them.

@rfsonders In the meantime, Could you please check if the above suggestion helps you ? Awaiting your reply.

[likamrat]

The new release for arcdata merged today, we are testing it as well. Also, related to #22168

[navba-MSFT]

@likamrat Thanks for comment.

@rfsonders I was able to run the command successfully.

C:>az sql mi-arc list -k MyARCName --use-k8s
[
{
"name": "XXXXXXX",
"primaryEndpoint": "XXXXXXXX,1433",
"replicas": "1/1",
"state": "Ready"
}
]

Please update the arcdata extension az extension update -n arcdata and then test again. Awaiting your reply.

[navba-MSFT]

@rfsonders I wanted to do quick follow-up to check if you had a chance to look at my above comment. Please let us know if you need any further assistance on this. Awaiting your reply.

[navba-MSFT]

@rfsonders Please follow the action plan mentioned above to update the arcdata extension. We will now proceed with closure of this github issue. If you need any further assistance on this issue in future, please feel free to reopen this thread. We would be happy to help.