Support finishing PP inference once eos_token_id is found (#11336)
#26
| Job | Run time |
|---|---|
| 1s | |
| 2s | |
| 2s | |
| 3s | |
| 2s | |
| 2s | |
| 2s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 0s | |
| 0s | |
| 20s |
eos_token_id is found (#11336)
#26
| Job | Run time |
|---|---|
| 1s | |
| 2s | |
| 2s | |
| 3s | |
| 2s | |
| 2s | |
| 2s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 1s | |
| 0s | |
| 0s | |
| 20s |