Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create script to export BART encoder and decoder for use with custom beam search op #11210

Closed
BowenBao opened this issue Apr 13, 2022 · 5 comments · Fixed by #12779
Closed

Create script to export BART encoder and decoder for use with custom beam search op #11210

BowenBao opened this issue Apr 13, 2022 · 5 comments · Fixed by #12779
Assignees
@BowenBao @titaiwangms
Labels
converter related to ONNX converters model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.

Comments

@BowenBao
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Under https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/transformers/models, a set of tools to enable users to export BART encoder and decoder to ONNX, for use with custom beam search op.
@BowenBao BowenBao added converter related to ONNX converters model:huggingface labels Apr 13, 2022
@BowenBao BowenBao self-assigned this Apr 13, 2022
@mfuntowicz
Copy link
Contributor

ORT folks contributed an example for this on the transformers repository if it can be useful: https://github.com/huggingface/transformers/tree/main/examples/research_projects/onnx/summarization

@tianleiwu
Copy link
Contributor

@BowenBao, thanks for raising the feature request. we are working on encoder and decoder support in beam search op (currently using T5 as example). After that is ready, we will work on BART integration.

@BowenBao
Copy link
Contributor Author

@tianleiwu thanks for following up. I have created a simple initial version for helping Ye with the implementation. Will create a PR afterwards.

@BowenBao
Copy link
Contributor Author

BowenBao commented May 25, 2022 •
edited
Loading

PR work in progress. #11629

@BowenBao BowenBao linked a pull request May 25, 2022 that will close this issue
[WIP] Initial Bart beam search export example #11629
Closed
@titaiwangms
Copy link
Contributor

8/11 Update:

  1. Model has been exported successfully.
  2. Model has mismatched performance between Pytorch and ONNX
  3. Validated Encoder and Decoder parts of model, and the results match Pytorch
  4. Now, Having ONNXRUNTIME team checks beam search op result

@sophies927 sophies927 added model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc. and removed model:huggingface labels Aug 12, 2022
@titaiwangms titaiwangms mentioned this issue Aug 29, 2022
Merged
titaiwangms added a commit that referenced this issue Oct 6, 2022
@titaiwangms
[ONNX] Support huggingface BART to ONNX (#12779)
87f5550 
Add BART into transformer support, specificalyy for
`BartForConditionalGeneration`

**Motivation and Context**
- fixes #11210 

Currently, the custom op beam search is not working in nightly, this PR
should be run with a [custom
commit](10f3d46)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BowenBao BowenBao

@titaiwangms titaiwangms

Labels
converter related to ONNX converters model:transformer issues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.
Projects
None yet
Milestone
No milestone
5 participants
@mfuntowicz @BowenBao @titaiwangms @tianleiwu @sophies927