[CherryPick][0.27.0] add TGI compat feature for rollingbatch (#1866)#1880

Merged

lanking520 merged 3 commits intodeepjavalibrary:0.27.0-dlcfrom lanking520:0.27.0-dlc

May 7, 2024

+78-69

Commits on May 7, 2024

add TGI compat feature for rollingbatch (deepjavalibrary#1866 )
Qing Lan
committed
[wlm] Avoid using gated model for testing (deepjavalibrary#1791 )

frankfliu
authored and
Qing Lan
committed
fix check style issue
Qing Lan
committed