[Tracking] [Docs] Model Support Streamlining

[CharlieFRuan]

Overview

This task focuses on streamlining the process of adding new models and keeping track of the existing prebuilt model libraries and model weights we offer. Specifically, there are three main goals:

1. Make the prebuilt model page more intuitive and well-structured
2. Create an end-to-end instruction for adding a new model, (as currently, “supporting a model” is relatively vague)
3. A better way to track the community’s model requests and contributions

For more on the design/layout, please see the doc here. Feel free to offer your suggestions, any insights would be greatly appreciated.

Co-author: @rickzx

Action Items

• Revamp the current prebuilt model page
  • [Docs] Model prebuilts tracking page revamp #1000
  • [Docs] Iterate model prebuilts docs #1043
• Create an end-to-end instruction/checklist on how to support a model, and where to update the documentation accordingly. (Edit: Perhaps not needed for now; include some pointers in the model prebuilt page to direct how to contribute to the tables)
  • Draft preview in pdf; corresponding to this branch. Need to wait for the model revamp PR to merge first. Also need to add concrete examples to make the page easier to follow.
  • Contribution log preview: so that contributors have a centralized place to record the models they compiled. We sync to the prebuilt model page once in a while (avoid and batchify cumbersome documentation updates).
• Create a dashboard for new model requests (perhaps use Phi or Falcon for trial to see how viable the workflow is)
  • https://github.com/orgs/mlc-ai/projects/2

Edit: move the below TODOs to the future when the new compile workflow lands

• Add some form of automation for adding model libraries and prebuilt weights (suggested by @junrushao)
• Add some form of automation that checks whether all the prebuilts we support are functioning (run weekly perhaps)

Links to Related Issues and PRs

There are various past efforts to improve this aspect of MLC-LLM's workflow, e.g.:

• [Tracking] New Model Requests #692
• https://github.com/mlc-ai/notebooks/blob/main/mlc-llm/tutorial_extensions_to_more_model_variants.ipynb

We would hope this attempt to be more future-proof and long-lasting.

[junrushao]

Hey @CharlieFRuan I love the work you've been leading on model streamlining! One additional question I'd love to discuss with you is that we will need not only a streamlined process, but also a transparent and reproducible workflow, preferably, say, a bash script or so, to release new models on all platforms, pushing them to GitHub and weights to HuggingFace.

So far, @MasterJH5574 has been in charge of new model releasing process, while knowing it's theoretically feasible, non-CMU contributors like me usually don't have sufficient incentive to push through in case of potential breaking change. Having a transparent bash script will help me personally to add new models without having to bother Ruihang all the time :)

[CharlieFRuan]

Hi @junrushao, thanks for bringing this up! We agree that some automation would make the process a lot less cumbersome, especially given that we have so many degrees of freedom (mode size, quantization scheme, and most importantly, platforms). I added this point to the task; we will follow up on this! cc @rickzx

[junrushao]

There are three aspects of releasing a model:

• A1. The model weights, which we usually update to HuggingFace; A model architecture could correspond to multiple model weights, for example, Llama2 for Llama2 and WizardLM;
• A2. The model lib, or the compiled execution of the model; A model architecture usually corresponds to a single model lib, unless there's some minor tweaks in the architecture;
• A3. The default mlc-chat-configuration.

Likely A1 and A3 could be bundled together, but I was curious if we have a plan displaying (A1 + A3) and A2 in a table?

[CharlieFRuan]

The PR here #1000 displays A1 and A2 into a set of tables. In the comment, there is a pdf for easier review of the page. There are three levels from high to low:

1. An all-in-one table, displaying all architectures and variants we support
2. Model lib table (A2 here), one for each architecture
3. Model weights (A1 here), one for each model variant

A3 is indeed missing, and we will add it to the level 3 tables! cc @rickzx

[junrushao]

Thanks! Let's move the discussion to that PR