Added the logic to fix the warmup phase for spec decoding when enforce_eager is not used #1017
Job | Run time |
---|---|
4s | |
3m 1s | |
5s | |
3s | |
2s | |
4s | |
4s | |
3s | |
4s | |
2s | |
3s | |
3s | |
2s | |
4s | |
3s | |
3s | |
3s | |
3s | |
4s | |
3s | |
4s | |
4s | |
2s | |
2s | |
4s | |
4m 19s |
Job | Run time |
---|---|
4s | |
3m 1s | |
5s | |
3s | |
2s | |
4s | |
4s | |
3s | |
4s | |
2s | |
3s | |
3s | |
2s | |
4s | |
3s | |
3s | |
3s | |
3s | |
4s | |
3s | |
4s | |
4s | |
2s | |
2s | |
4s | |
4m 19s |