Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add encoder model support in trt-llm (#1294)
* add encoder * update: port * update: serving image builder * fmt * fmt * fmt * add variables for encoder * update default config settings * update: constants, serving image builder * update: serving image builder * update start command * update truss serving builder * add clearer separation between encoder and decoder paths * remove concurrenct overwrite * serving image builder: update predict concurrency * add: comments for max batch size * make serving image builder asserion * add predict endpopint as /v1/embeddings for faster testing
- Loading branch information