Skip to content

Commit

Permalink
Add encoder model support in trt-llm (#1294)
Browse files Browse the repository at this point in the history
* add encoder

* update: port

* update: serving image builder

* fmt

* fmt

* fmt

* add variables for encoder

* update default config settings

* update: constants, serving image builder

* update: serving image builder

* update start command

* update truss serving builder

* add clearer separation between encoder and decoder paths

* remove concurrenct overwrite

* serving image builder: update predict concurrency

* add: comments for max batch size

* make serving image builder asserion

* add predict endpopint as /v1/embeddings for faster testing
  • Loading branch information
michaelfeil authored Jan 9, 2025
1 parent 9a029be commit c07100b
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 262 deletions.
Loading

0 comments on commit c07100b

Please sign in to comment.