New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

quantization: instantiate model on CPU #163

Merged

lantiga merged 1 commit into Lightning-AI:main from t-vi:quantize-save-gpu-ram

Apr 19, 2023

Contributor

t-vi commented Apr 19, 2023

Initialize the mode on CPU to save GPU RAM.

Fixes #148


          quantization: instantiate model on CPU

617849a

t-vi requested review from awaelchli, carmocca and lantiga as code owners

April 19, 2023 05:58

lantiga approved these changes

View reviewed changes

Collaborator

lantiga left a comment

Stamped!

lantiga merged commit d77eb23 into Lightning-AI:main

timothylimyl referenced this pull request in timothylimyl/lit-llama-qa


          quantization: instantiate model on CPU (#163)

64636f1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet