Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Refactor llumlet to better adapt to different backend engines #4

Merged
merged 2 commits into from
Jul 26, 2024
Merged

[Refactor] Refactor llumlet to better adapt to different backend engines #4

merged 2 commits into from
Jul 26, 2024

Conversation

s5u13b
Copy link
Contributor

@s5u13b s5u13b commented Jul 23, 2024 •
edited
Loading

Refactor Llumlet, move functions and data such as run_engine_loop, step, and free_request_states to the LLMEngineLlumnix. There is no longer any need to make assumptions about step returns and loop ways of the backend engine.

@s5u13b s5u13b requested review from zhypku and ZeldaHuang July 23, 2024 11:30
@CLAassistant
Copy link

CLAassistant commented Jul 23, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@ZeldaHuang
Copy link
Contributor

It seems that we don't need BackendVLLM anymore, LLMEngineLlumnix implements all of its functions.

zhypku
zhypku approved these changes Jul 25, 2024
View reviewed changes
Copy link
Collaborator

@zhypku zhypku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ZeldaHuang
ZeldaHuang approved these changes Jul 26, 2024
View reviewed changes
Copy link
Contributor

@ZeldaHuang ZeldaHuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, We can remove BackendVLLM later.

@s5u13b s5u13b changed the title [Refactor] Refactor llumlet to adapt to different backend engines [Refactor] Refactor llumlet to better adapt to different backend engines Jul 26, 2024
@s5u13b s5u13b merged commit 986a617 into main Jul 26, 2024
4 checks passed
@s5u13b s5u13b deleted the llumlet-refactor branch July 26, 2024 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhypku zhypku zhypku approved these changes

@ZeldaHuang ZeldaHuang ZeldaHuang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@s5u13b @CLAassistant @ZeldaHuang @zhypku