Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added int4 compression to LLM chatbot notebook (#1428)
* Added compression code * Description changes * Tweaks * Disabled int4 compression for red-pajama model * Tweak * Readme tweak * Made mpt model a default one * Fix linter issues * Addressed comments * Tweaks * Changed int4/int8 to 4/8 bit data types * Included int8 in a note about dgpu * Tweaked notes * Added device * Added selecting model to run * Tweaked note
- Loading branch information