HTTPError: Valid url scheme and host required

[maheshsonavane]

Describe the bug

I am experiencing an error when attempting to retrieve user information from the API. This functionality was working as expected until yesterday, but I started encountering this issue this morning.

Expected behavior

Should return list of all users

How to reproduce

from msgraph import GraphServiceClient
scopes = ['https://graph.microsoft.com/.default']
credential = ClientSecretCredential(TENANT_ID,
CLIENT_ID,
CLIENT_SECRET)
client = GraphServiceClient(credentials=credential, scopes=scopes)
graph_client = GraphServiceClient(credentials=credential, scopes=scopes)
result = await graph_client.users.get()

SDK Version

latest

Latest version known to work for scenario above?

No response

Known Workarounds

No response

Debug output

HTTPError: Valid url scheme and host required
File , line 8
6 client = GraphServiceClient(credentials=credential, scopes=scopes)
7 graph_client = GraphServiceClient(credentials=credential, scopes=scopes)
----> 8 result = await graph_client.users.get()
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/msgraph/generated/users/users_request_builder.py:73, in UsersRequestBuilder.get(self, request_configuration)
70 raise Exception("Http core is null")
71 from ..models.user_collection_response import UserCollectionResponse
---> 73 return await self.request_adapter.send_async(request_info, UserCollectionResponse, error_mapping)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:186, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
183 parent_span.record_exception(REQUEST_IS_NULL)
184 raise REQUEST_IS_NULL
--> 186 response = await self.get_http_response_message(request_info, parent_span)
188 response_handler = self.get_response_handler(request_info)
189 if response_handler:
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:595, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
592 if claims:
593 additional_authentication_context[self.CLAIMS_KEY] = claims
--> 595 await self._authentication_provider.authenticate_request(
596 request_info, additional_authentication_context
597 )
599 request = self.get_request_from_request_information(
600 request_info, _get_http_resp_span, parent_span
601 )
602 resp = await self._http_client.send(request)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py:50, in BaseBearerTokenAuthenticationProvider.authenticate_request(self, request, additional_authentication_context)
47 request.headers = HeadersCollection()
49 if not request.headers.contains(self.AUTHORIZATION_HEADER):
---> 50 token = await self.access_token_provider.get_authorization_token(
51 request.url, additional_authentication_context
52 )
53 if token:
54 request.headers.add(f'{self.AUTHORIZATION_HEADER}', f'Bearer {token}')
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/kiota_authentication_azure/azure_identity_access_token_provider.py:78, in AzureIdentityAccessTokenProvider.get_authorization_token(self, uri, additional_authentication_context)
76 exc = HTTPError("Valid url scheme and host required")
77 span.record_exception(exc)
---> 78 raise exc
80 if parsed_url.scheme != "https" and (parsed_url.hostname not in self.LOCALHOST_STRINGS):
81 span.set_attribute(self.IS_VALID_URL, False)

Configuration

No response

Other information

No response

[cornielheemskerk]

Related to #1034, which also provides a workaround

[maheshsonavane]

Thanks a lot for the update. Workaround seems to be not working.

…

________________________________ From: Corniël Joosse ***@***.***> Sent: Thursday, January 30, 2025 11:42 AM To: microsoftgraph/msgraph-sdk-python ***@***.***> Cc: Mahesh Sonavane ***@***.***>; Author ***@***.***> Subject: Re: [microsoftgraph/msgraph-sdk-python] HTTPError: Valid url scheme and host required (Issue #1089) Related to #1034<#1034>, which also provides a workaround — Reply to this email directly, view it on GitHub<#1089 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKZQ2GWN63AIQ6LM46KW7TT2NIF3XAVCNFSM6AAAAABWE3KUEWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRUGI3DCNRZGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[tiborrr]

Root Cause

There's a type inconsistency in the GraphClientFactory:

1. Both create_with_default_middleware and create_with_custom_middleware correctly specify their host parameter as NationalClouds and api_version as APIVersion types
2. However, _get_base_url method incorrectly defines the host parameter as str and doesn't access the value property of either enum:

# Current problematic implementation
@staticmethod
def _get_base_url(host: str, api_version: APIVersion) -> str:
    base_url = f'{host}/{api_version}'  # Missing .value for both enums
    return base_url

Fix

Update the _get_base_url method in GraphClientFactory to properly handle both enums:

@staticmethod
def _get_base_url(host: NationalClouds, api_version: APIVersion) -> str:
    """Helper method to set the complete base url"""
    base_url = f'{host.value}/{api_version.value}'  # Access .value for both enums
    return base_url

Temporary Workaround

Until this fix is released, you can work around the issue by explicitly setting the base URL after creating the client:

from typing import Optional

from azure.identity.aio import ClientSecretCredential
from kiota_authentication_azure.azure_identity_authentication_provider import (
    AzureIdentityAuthenticationProvider,
)
from kiota_http.kiota_client_factory import KiotaClientFactory
from msgraph.graph_request_adapter import GraphRequestAdapter, options
from msgraph.graph_service_client import GraphServiceClient
from msgraph_core import GraphClientFactory, NationalClouds, APIVersion

from src.config import AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID

def get_request_adapter(
    credentials: ClientSecretCredential, scopes: Optional[list[str]] = None
) -> GraphRequestAdapter:
    if scopes:
        auth_provider = AzureIdentityAuthenticationProvider(
            credentials=credentials, scopes=scopes
        )
    else:
        auth_provider = AzureIdentityAuthenticationProvider(credentials=credentials)

    client = GraphClientFactory.create_with_default_middleware(
        options=options, 
        client=KiotaClientFactory.get_default_client(),
    )
    # Workaround: Explicitly set the base URL using enum values
    client.base_url = f"{NationalClouds.Global.value}/{APIVersion.v1.value}"
    
    return GraphRequestAdapter(
        auth_provider=auth_provider,
        client=client,
    )

[svrooij]

If this issue suddenly appeared (code worked previously, and stopped when you updated something), it might be related to this pr in kiota

[pl4nty]

this impacted me after a dependency cache expired, why aren't kiota dependencies using pinned versions?

msgraph-sdk-python/pyproject.toml

Lines 13 to 21 in 25d6c6c

	dependencies = [
	"azure-identity >=1.12.0",
	"microsoft-kiota-serialization-json >=1.8.0,<2.0.0",
	"microsoft-kiota-serialization-text >=1.8.0,<2.0.0",
	"microsoft-kiota-serialization-form >=1.8.0,<2.0.0",
	"microsoft-kiota-serialization-multipart >=1.8.0,<2.0.0",
	"msgraph_core >=1.3.1"
	]

[andrueastman]

Hey everyone,
After investigations, this issue is related to microsoftgraph/msgraph-sdk-python-core#818 together with changes made in microsoft/kiota-python#459.

In python 3.11, the formating of enum values to strings does not result in the enum value being set but the property name being used. This resulted in the httpclient.baseurl being set to something like NationalClouds.Global/APIVersion.v1/ rather than https://graph.microsoft.com/v1.0/. This change was unnoticed/inconsequenstial till the fix made in microsoft/kiota-python#459 that made the adapter use the base_url from the http client.

This issue was fortunately already been resolved in microsoftgraph/msgraph-sdk-python-core#820 and therefore the solution here it to ensure that your dependency tree has msgraph-core version 1.2.1 or greater.

https://github.com/microsoftgraph/msgraph-sdk-python-core/releases/tag/v1.2.1

[baywet]

Closing since the issue seems to be resolved