Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API usage issues - TOP being ignored, timeout, etc. #22823

Closed
OTDani opened this issue Jan 28, 2022 · 9 comments
Closed

API usage issues - TOP being ignored, timeout, etc. #22823

OTDani opened this issue Jan 28, 2022 · 9 comments
Assignees
Labels
Consumption - Billing All issues in consumption where billing/price-related data is shown. customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-author-feedback Workflow: More information is needed from author to address the issue. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team no-recent-activity There has been no recent activity on this issue. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@OTDani
Copy link

OTDani commented Jan 28, 2022

using version azure-mgmt-consumption==9.0.0

The API .usage_details.list(
scope=f'/providers/Microsoft.Billing/billingAccounts/{billingAccountIdLong}/billingProfiles/{billingProfileId}'
,metric='amortizedcost',
top=100)

Completely ignores the top clause, also if you use it '100'.

The API also throws (429) Too many requests. Please retry after 60 seconds.
when you try to consume it.

Could you please advise on what is the correct way to use this API to get the current months all charges from Azure, to all subscriptions in our billing account / billing profile?

Is there a way to govern to get back e.g. last 3 days only?

The closest we get if we start to loop over each subscription separately - but this both seems a bad approach, as we have billing profiles and consolidated accounts - while also it is VERY slow - 6 accounts data jan 1 to 15 takes 13 minutes, and these are not even our big accounts.

Thank you in advance


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 28, 2022
@tjprescott tjprescott added the Mgmt This issue is related to a management-plane library. label Jan 28, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jan 28, 2022
@tjprescott tjprescott added CXP Attention needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Jan 28, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jan 28, 2022
@tjprescott tjprescott added Consumption - Billing All issues in consumption where billing/price-related data is shown. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Jan 28, 2022
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jan 28, 2022
@tjprescott
Copy link
Member

@msyyc can you take a look?

@kazrael2119
Copy link
Contributor

kazrael2119 commented Feb 8, 2022

Hello @OTDani ,I tried to solve your problem. Here is the code. I hope it will help you

import os
import datetime

from azure.identity import DefaultAzureCredential
from azure.mgmt.consumption import ConsumptionManagementClient

credentials = DefaultAzureCredential()
subscription = os.getenv('SUBSCRIPTION_ID')

consumption_client = ConsumptionManagementClient(DefaultAzureCredential(), subscription_id=subscription)
billing_account_id = 'your billing account id'

#Get today's date and the date three days ago
current_date = datetime.datetime.now()
offset = datetime.timedelta(days=-3)
past_date = (current_date + offset).strftime('%Y-%m-%dT%H:%M:%SZ')
filter_expression = "{} and {}".format(current_date.strftime('%Y-%m-%dT%H:%M:%SZ'), past_date)

res=consumption_client.usage_details.list(
    scope=f'/providers/Microsoft.Billing/billingAccounts/{billing_account_id}/billingProfiles/{billing_account_id}'
    ,metric='amortizedcost',
    top=100,filter=filter_expression)

@BigCat20196 BigCat20196 added the needs-author-feedback Workflow: More information is needed from author to address the issue. label Feb 17, 2022
@PramodValavala-MSFT PramodValavala-MSFT added Service Attention Workflow: This issue is responsible by Azure service team. and removed CXP Attention labels Feb 21, 2022
@ghost
Copy link

ghost commented Feb 21, 2022

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @ccmbpxpcrew.

Issue Details

using version azure-mgmt-consumption==9.0.0

The API .usage_details.list(
scope=f'/providers/Microsoft.Billing/billingAccounts/{billingAccountIdLong}/billingProfiles/{billingProfileId}'
,metric='amortizedcost',
top=100)

Completely ignores the top clause, also if you use it '100'.

The API also throws (429) Too many requests. Please retry after 60 seconds.
when you try to consume it.

Could you please advise on what is the correct way to use this API to get the current months all charges from Azure, to all subscriptions in our billing account / billing profile?

Is there a way to govern to get back e.g. last 3 days only?

The closest we get if we start to loop over each subscription separately - but this both seems a bad approach, as we have billing profiles and consolidated accounts - while also it is VERY slow - 6 accounts data jan 1 to 15 takes 13 minutes, and these are not even our big accounts.

Thank you in advance


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

Author: OTDani
Assignees: msyyc, kazrael2119
Labels:

question, Service Attention, Mgmt, customer-reported, needs-author-feedback, Consumption - Billing

Milestone: -

@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label Feb 28, 2022
@ghost
Copy link

ghost commented Feb 28, 2022

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@OTDani
Copy link
Author

OTDani commented Feb 28, 2022

We are waiting on @ccmbpxpcrew
Please let me know what is the information you need from me?

@ghost ghost added needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team and removed needs-author-feedback Workflow: More information is needed from author to address the issue. no-recent-activity There has been no recent activity on this issue. labels Feb 28, 2022
@BigCat20196
Copy link
Contributor

Hi @OTDani, For some reason, the top parameter is indeed invalid in python sdk. @kazrael2119 provides an sample on how to use it to return only the results of the last three days. Hope it can help you.

@OTDani
Copy link
Author

OTDani commented Mar 2, 2022

Thanks @BigCat20196 @kazrael2119 !!

  1. in the @kazrael2119 code the scope is
scope=f'/providers/Microsoft.Billing/billingAccounts/{billing_account_id}/billingProfiles/{billing_account_id}'

Could you please double check this? As AFAIK it should be a billing profile id at the end, this throws an error 500.

  1. Using what i think was intended, the code times out with (429) Too many requests. Please retry after 60 seconds.
scope=f'/providers/Microsoft.Billing/billingAccounts/{billingAccountIdLong}/billingProfiles/{billingProfileId}'

but it runs, but not fine for just a subscription scope:

scope=f'/subscriptions/{azu_subscription_id}/'

Returns 1 small account taking over a minute - 4474 elements - (I'm sitting on gigabit net)
It completely ignores the date filter.

Expected data from Past 3 days, actual: the API gives back data from month's start as it would without the filter 😢

  1. Back to the other two issues:
  • How to overcome (429) Too many requests. Please retry after 60 seconds. ?
  • How to speed this up?

@msyyc
Copy link
Member

msyyc commented Apr 19, 2022

Hi @OTDani , Python SDK is just tools to send/receive request/response. According to the context, it seems that server does not take the filter effect. Please follow https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/debug_guide.md to provide more detailed logs. With the logs, you could see the raw request and response from server. If the request is right with filters while server still return more data, it proves that there may be problems in server and you could make issue in https://github.com/Azure/azure-rest-api-specs/issues

@xiangyan99 xiangyan99 added the needs-author-feedback Workflow: More information is needed from author to address the issue. label May 20, 2022
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label May 28, 2022
@ghost
Copy link

ghost commented May 28, 2022

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@ghost ghost closed this as completed Jun 11, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2023
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Consumption - Billing All issues in consumption where billing/price-related data is shown. customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-author-feedback Workflow: More information is needed from author to address the issue. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team no-recent-activity There has been no recent activity on this issue. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

7 participants