Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release/4.0] Support O3 OpenAI model mapping #7395

Merged
merged 2 commits into from
Feb 21, 2025

Conversation

github-actions[bot]
Copy link

@github-actions github-actions bot commented Feb 21, 2025

Backport of #7394 to release/4.0

/cc @tarekgh

Customer Impact

Users will be able to create a tokenizer object for the OpenAI o1 and o3 models.

Testing

Manually tested it and added a test specific to these models.

Risk

Very low. No real code change, only allow accepting the model names "o1" and "o3" to create the tokenizer instance.

tarekgh and others added 2 commits February 21, 2025 01:22
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link

codecov bot commented Feb 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (release/4.0@208651a). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             release/4.0    #7395   +/-   ##
==============================================
  Coverage               ?   68.89%           
==============================================
  Files                  ?     1470           
  Lines                  ?   274081           
  Branches               ?    28405           
==============================================
  Hits                   ?   188821           
  Misses                 ?    77944           
  Partials               ?     7316           
Flag Coverage Δ
Debug 68.89% <100.00%> (?)
production 63.29% <100.00%> (?)
test 89.42% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...Microsoft.ML.Tokenizers/Model/TiktokenTokenizer.cs 78.34% <100.00%> (ø)
...est/Microsoft.ML.Tokenizers.Tests/TiktokenTests.cs 98.98% <ø> (ø)

@ericstj
Copy link
Member

ericstj commented Feb 21, 2025

/ba-g all failures are #6978

@ericstj ericstj merged commit 867a30f into release/4.0 Feb 21, 2025
22 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants