Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add documentation #1567

Merged
merged 11 commits into from
Dec 16, 2024
Merged

fix: Add documentation #1567

merged 11 commits into from
Dec 16, 2024

Conversation

KennethEnevoldsen
Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen commented Dec 7, 2024

I had some time on a plane so I made a documentation page!! I initially was thinking of moving everything there, but there were too many problems (especially with circular imports encountered when using the autodoc function) so I decided to work only on the API documentation which we currently don't have.

Therefore there are a lot of changes that are currently not used in the docs. The hope is that after this PR we can resolve some of these issues.

Here are some examples of the docs:

Screenshot 2024-12-07 at 13 07 36 Screenshot 2024-12-07 at 13 06 45

@KennethEnevoldsen KennethEnevoldsen requested review from x-tabdeveloping, Samoed and isaac-chung and removed request for Samoed December 7, 2024 21:10
Copy link
Collaborator

@Samoed Samoed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

docs/adding_a_benchmark.md Outdated Show resolved Hide resolved
docs/getting_started.md Show resolved Hide resolved
Copy link
Collaborator

@x-tabdeveloping x-tabdeveloping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work so far, I really like it

Copy link
Collaborator

@isaac-chung isaac-chung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff! Added some suggested edits.

docs/getting_started.md Show resolved Hide resolved
docs/api/task.md Outdated Show resolved Hide resolved
docs/api/task.md Outdated Show resolved Hide resolved
docs/api/model.md Outdated Show resolved Hide resolved
docs/api/benchmark.md Show resolved Hide resolved
docs/adding_a_benchmark.md Outdated Show resolved Hide resolved
Comment on lines +31 to +36
1. Open a PR at https://huggingface.co/spaces/mteb/leaderboard modifying app.py to add your tab:
- Add any new models & their specs to the global lists
- Add your tab, credits etc to where the other tabs are defined
- If you're adding new results to existing models, remove those models from `EXTERNAL_MODEL_RESULTS.json` such that they can be reloaded with the new results and are not cached.
- You may also have to uncomment `, download_mode='force_redownload', verification_mode="no_checks")` where the datasets are loaded to experiment locally without caching of results
- Test that it runs & works locally as you desire with python app.py, **please add screenshots to the PR**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it only for adding a new tab to the leaderboard, or every time a new model is added?
If it's the latter, then 3 PRs are needed to add a new model: MTEB repo, results repo, and leaderboard repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing should be needed to the leaderboard repo

CONTRIBUTING.md Outdated Show resolved Hide resolved
CONTRIBUTING.md Outdated Show resolved Hide resolved
CONTRIBUTING.md Outdated Show resolved Hide resolved
@KennethEnevoldsen KennethEnevoldsen mentioned this pull request Dec 9, 2024
2 tasks
@KennethEnevoldsen
Copy link
Contributor Author

Updated docs and merged everything. I merge this into the branch now

@KennethEnevoldsen KennethEnevoldsen merged commit 06fc13f into v2.0.0 Dec 16, 2024
10 checks passed
@KennethEnevoldsen KennethEnevoldsen deleted the add-docs branch December 16, 2024 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants