Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chatllama] Add support to other open-source models #218

Open
8 of 13 tasks
diegofiori opened this issue Mar 8, 2023 · 0 comments
Open
8 of 13 tasks

[Chatllama] Add support to other open-source models #218

diegofiori opened this issue Mar 8, 2023 · 0 comments
Labels
chatllama Issue related to the ChatLLaMA module good first issue Good for newcomers

Comments

@diegofiori
Copy link
Collaborator

diegofiori commented Mar 8, 2023

Description

LLaMA is a great model, however the weights license can be a huge barrier for users interested in the open-source LLM capabilities. The actual RLHF is designed around the LLaMA model, but it can be extended to other open-source models, like Flan-T5 or OPT.

TODO

  • Extend support of ChatLLaMA to HuggingFace models
  • Add support for the encoder-decoder architectures (AutoModelsForSeq2Seq)
  • Add support for LLaMA from HF.
  • Add support for Cerebras GPT.
  • Test the support for
    • FlanT5
    • GPTJ
    • GPTNeoX
    • OPT
    • BLOOM
    • BLOOMZ
    • Galactica
  • Implement unittest for the code
@diegofiori diegofiori added the chatllama Issue related to the ChatLLaMA module label Mar 8, 2023
@nebuly-ai nebuly-ai moved this to Requested Features in ChatLLaMA Roadmap Mar 8, 2023
@nebuly-ai nebuly-ai moved this from Requested Features to Todo in ChatLLaMA Roadmap Mar 8, 2023
@nebuly-ai nebuly-ai moved this from Todo to In Progress in ChatLLaMA Roadmap Mar 8, 2023
@diegofiori diegofiori added the good first issue Good for newcomers label Mar 9, 2023
@PierpaoloSorbellini PierpaoloSorbellini changed the title Add support to other open-source models [Chatllama] Add support to other open-source models Mar 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chatllama Issue related to the ChatLLaMA module good first issue Good for newcomers
Projects
Status: In Progress
Development

No branches or pull requests

1 participant