Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support whisper for openvino #470

Merged
merged 10 commits into from
Dec 14, 2023
Merged

Conversation

eaidova
Copy link
Collaborator

@eaidova eaidova commented Nov 3, 2023

What does this PR do?

Adding support for inference whisper models family with openvino.
Verified on distil-whisper-large-v2

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@eaidova
Copy link
Collaborator Author

eaidova commented Nov 3, 2023

@AlexKoff88 @echarlaix @helena-intel could you please take a look?

@eaidova eaidova force-pushed the ea/whisper branch 4 times, most recently from bbc39bd to f73f469 Compare November 3, 2023 13:36
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@eaidova
Copy link
Collaborator Author

eaidova commented Nov 3, 2023

@echarlaix I pin transformers<4.35 version as workaround for incompatibility issues to be able pass ci checks (removing SSDP, refactoring models that lead to previously used patching approach does not work anymore for llama, opt, and other models).
Is there any plans to fix these issues and alignment with optimum package?

@eaidova eaidova force-pushed the ea/whisper branch 3 times, most recently from d66bb87 to 169e11c Compare November 7, 2023 06:45
@adrianboguszewski
Copy link

@echarlaix @helena-intel could we prioritize this PR? We need it very much. It will simplify using OpenVINO for Whisper a lot :)

Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice PR @eaidova, thanks!

optimum/intel/openvino/modeling_seq2seq.py Outdated Show resolved Hide resolved
optimum/intel/openvino/modeling_seq2seq.py Outdated Show resolved Hide resolved
optimum/intel/openvino/modeling_seq2seq.py Outdated Show resolved Hide resolved
optimum/intel/openvino/modeling_seq2seq.py Outdated Show resolved Hide resolved
@eaidova
Copy link
Collaborator Author

eaidova commented Dec 14, 2023

@echarlaix looks like transformers 4.36 break some model export, including whisper support. I added workaround similar to huggingface/optimum#1590, please take a look

Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great 🔥 , thanks a lot for the addition @eaidova

@echarlaix echarlaix merged commit b056177 into huggingface:main Dec 14, 2023
6 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants