add parakeet finetuning tutorials #201
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
82996de to
76018fd
Compare
|
rebased and added a minor fix found in other thread, @nv-uvaidya could you review the new tutorial as well? |
|
I think it is better to replace the finetuning tutorial file name from asr_finetune_parakeet_nemo.ipynb to asr-finetune-parakeet-nemo.ipynb. |
myungjongk
reviewed
Nov 12, 2024
asr_finetune_parakeet_nemo.ipynb
Outdated
| "# How to Fine-Tune a Riva ASR Acoustic Model with NVIDIA NeMo\n", | ||
| "This tutorial walks you through how to fine-tune an NVIDIA Riva ASR acoustic model with NVIDIA NeMo.\n", | ||
| "\n", | ||
| "**Important**: If you plan to fine-tune an ASR acoustic model using the same tokenizer with which the model was trained, skip this tutorial and refer to the \"Sub-word Encoding CTC Model\" section (starting with the \"Load pre-trained model\" subsection) of the [NeMo ASR Language Finetuning tutorial](https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_CTC_Language_Finetuning.ipynb)." |
There was a problem hiding this comment.
We can remove this line as we already handle finetuning with same tokenizer in this tutorial. Or we can just refer to NeMo's tutorial as additional finetuning resources.
asr_finetune_parakeet_nemo.ipynb
Outdated
| "\n", | ||
| "Hybrid RNNT-CTC models is a group of models with both the RNNT and CTC decoders. Training a hybrid model would speedup the convergence for the CTC models and would enable the user to use a single model which works as both a CTC and RNNT model. This category can be used with any of the ASR models. Hybrid models uses two decoders of CTC and RNNT on the top of the encoder.\n", | ||
| "\n", | ||
| "NeMo uses `.yml` files to configure the training parameters. You may update them directly by editing the configuration file or from the command-line interface. For example, if the number of epochs needs to be modified, along with a change in the learning rate, you can add `trainer.max_epochs=100` and `optim.lr=0.02` and train the model.\n", |
asr_finetune_parakeet_nemo.ipynb
Outdated
| "\n", | ||
| "NeMo uses `.yml` files to configure the training parameters. You may update them directly by editing the configuration file or from the command-line interface. For example, if the number of epochs needs to be modified, along with a change in the learning rate, you can add `trainer.max_epochs=100` and `optim.lr=0.02` and train the model.\n", | ||
| "\n", | ||
| "The following sample command uses the `speech_to_text_hybrid_rnnt_ctc_bpe.py` script in the `examples` folder to train/fine-tune a Parakeet-Hybrid ASR model for 1 epoch. For other ASR models like Citrinet, Conformer, you may find the appropriate config files in the NeMo GitHub repo under [examples/asr/conf/](https://github.com/NVIDIA/NeMo/tree/main/examples/asr/conf).\n" |
There was a problem hiding this comment.
speech_to_text_hybrid_rnnt_ctc_bpe.py needs to be replaced with speech_to_text_finetune.py.
asr_finetune_parakeet_nemo.ipynb
Outdated
| "source": [ | ||
| "#### Convert to Riva\n", | ||
| "\n", | ||
| "Convert the downloaded model to the `.riva` format. We will set the encryption key with `--key=nemotoriva`. Choose a different encryption key value when generating `.riva` models for production.\n", |
There was a problem hiding this comment.
I think we can mention --onnx-opset 18 is needed for the Riva version 2.15.0 and above.
There was a problem hiding this comment.
not needed for RNNT models
asr_finetune_parakeet_nemo.ipynb
Outdated
| "outputs": [], | ||
| "source": [ | ||
| "riva_file_path = ctc_model_path[:-5]+\".riva\"\n", | ||
| "!nemo2riva --key=nemotoriva --out $riva_file_path $ctc_model_path" |
rmittal-github
approved these changes
Nov 14, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.