Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report LLM token usage #991

Merged
merged 2 commits into from
Feb 6, 2025
Merged

Report LLM token usage #991

merged 2 commits into from
Feb 6, 2025

Conversation

clavedeluna
Copy link
Contributor

Any codemod which may use an LLM and spends prompt and completion tokens will now be reported, per each codemod and at the end of codemodder run.

Looks something like this (redacted)

Codemod REDACTED
	completion_tokens = 2309
	prompt_tokens = 13742
...
running codemod SOME OTHER CODEMOD
...
...
[report]
scanned: 8 files
failed: 0 files (0 unique)
changed: 8 files (4 unique)
report file: here.txt
All token usage
	completion_tokens = 5176
	prompt_tokens = 40960

@pixeebot pixeebot bot mentioned this pull request Feb 6, 2025
Closed
andrecsilva
andrecsilva approved these changes Feb 6, 2025
View reviewed changes
src/codemodder/llm.py Outdated
return None
logger.info("Azure OpenAI API client not available")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this is a mistake, this log is unreacheable and will never execute.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wooops yes very much

@clavedeluna clavedeluna enabled auto-merge February 6, 2025 18:03
@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Feb 6, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@clavedeluna clavedeluna added this pull request to the merge queue Feb 6, 2025
Merged via the queue into main with commit b2baa38 Feb 6, 2025
15 checks passed
@clavedeluna clavedeluna deleted the report-token-usage branch February 6, 2025 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@drdavella drdavella drdavella approved these changes

@andrecsilva andrecsilva andrecsilva approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@clavedeluna @drdavella @andrecsilva