Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: (web UI) Add samplers sequence customization #10255

Merged
merged 9 commits into from
Nov 16, 2024

Conversation

MaggotHATE
Copy link
Contributor

This is a simple way to add samplers sequence into the new server UI - an input field with simplified sequence and an explanation for it. Since server doesn't support simplified version for now, I've added it too.

image

@@ -227,6 +227,15 @@ <h3 class="text-lg font-bold mb-6">Settings</h3>
<summary class="collapse-title font-bold">Advanced config</summary>
<div class="collapse-content">
<label class="form-control mb-2">
<!-- Samplers queue -->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be inside "Other sampler settings"

The "advanced" is mostly for non-sampler stuff I think, like pinning slot number or stop sequence

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw if you merge with latest upstream master branch, you can re-use the settings-modal-short-input component

Copy link
Contributor Author

@MaggotHATE MaggotHATE Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can re-use the settings-modal-short-input component

Thanks, it works! However, the name is automatically set to samplers, which is not very descriptive.

Should I modify settings-modal-short-input to have a separate value for name? UPD: seems to work just fine. It would also allow setting prettier names for sampler parameters later.

@MaggotHATE
Copy link
Contributor Author

MaggotHATE commented Nov 16, 2024

Tested again, works correctly so far.

The next (and probably the last) step is adding customization to chat templates: for example, suffix is quite useful in many cases as an additional guidance, and in some instances there might be interest in customizing default/chosen template (which, for example, is suggested my Mistral).

This PR was initially supposed to be a list of cards to choose chat templates from, but I'm not sure it should be in UI - seems like it's only needed on server launch.

@ngxson
Copy link
Collaborator

ngxson commented Nov 16, 2024

In fact, I intentionally leave out chat template stuff because in the old UI, it's kinda hacky to support it via /completion endpoint. My main goal of this new UI is rely solely on /chat/completion which simplify a lot of things.

We could add the notion of chat_template in /chat/completion, as well as in_prefix/suffix for custom chat template. But this can be quite messy so I think I'll need to consider it later. Ideally, we should not spend too many LOCs on this, because most users don't care about it.

@ngxson ngxson merged commit bcdb7a2 into ggerganov:master Nov 16, 2024
54 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
* Samplers sequence: simplified and input field.

* Removed unused function

* Modify and use `settings-modal-short-input`

* rename "name" --> "label"

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants