Skip to content

Commit

Permalink
Added int4 compression to LLM chatbot notebook (#1428)
Browse files Browse the repository at this point in the history
* Added compression code

* Description changes

* Tweaks

* Disabled int4 compression for red-pajama model

* Tweak

* Readme tweak

* Made mpt model a default one

* Fix linter issues

* Addressed comments

* Tweaks

* Changed int4/int8 to 4/8 bit data types

* Included int8 in a note about dgpu

* Tweaked notes

* Added device

* Added selecting model to run

* Tweaked note
  • Loading branch information
nikita-savelyevv authored Nov 8, 2023
1 parent 7a79ccb commit a6b3b36
Show file tree
Hide file tree
Showing 2 changed files with 372 additions and 186 deletions.
Loading

0 comments on commit a6b3b36

Please sign in to comment.