Code

Disaggregate prefill decode with zmq#1

Code

Draft

panf2333 wants to merge 20 commits intomainyottalabsai/vllm:mainfrom disaggregate_prefill_decode_with_zmqyottalabsai/vllm:disaggregate_prefill_decode_with_zmqCopy head branch name to clipboard

+434-10

Commits

Commits on Feb 18, 2025

• add test py

  Show description for 0f5d505
  panf2333

  committed
  0f5d505

  Copy full SHA for 0f5d505

• add vllm connect cmd

  Show description for b80f7b5
  panf2333

  committed
  b80f7b5

  Copy full SHA for b80f7b5

• add identity url headers

  Show description for fb54d40
  panf2333

  committed
  fb54d40

  Copy full SHA for fb54d40

• add /v1/completions stream support

  Show description for 89e11da
  panf2333

  committed
  89e11da

  Copy full SHA for 89e11da

• 1. connect_parser set --prefill-addr and --decode-addr are required

  Show description for 24efce2
  panf2333

  committed
  24efce2

  Copy full SHA for 24efce2

• update disagg_connect test_request.py

  Show description for 604f6c3
  panf2333

  committed
  604f6c3

  Copy full SHA for 604f6c3

• Replace zmq.asyncio.Context().term() with zmq.asyncio.Context().destroy(linger=0) for immediate termination

  Show description for ac26495
  panf2333

  committed
  ac26495

  Copy full SHA for ac26495

• 1. fix mypy issue

  Show description for 2e9615a
  panf2333

  committed
  2e9615a

  Copy full SHA for 2e9615a

• Run yapf and ruff

  Show description for e732cdd
  panf2333

  committed
  e732cdd

  Copy full SHA for e732cdd

• 1. replace tpc:// with ipc:// \n 2. fix json response

  Show description for 2cc7234
  panf2333

  committed
  2cc7234

  Copy full SHA for 2cc7234

• create proxy sockets in the proxy function for thread safety

  Show description for d588f6a
  panf2333

  committed
  d588f6a

  Copy full SHA for d588f6a

• fix ThreadProxy

  Show description for cdf0d12
  panf2333

  committed
  cdf0d12

  Copy full SHA for cdf0d12

• run format.sh

  Show description for 95d2125
  panf2333

  committed
  95d2125

  Copy full SHA for 95d2125

• add benchmark shell

  Show description for ec2dbb5
  panf2333

  committed
  ec2dbb5

  Copy full SHA for ec2dbb5

• run format

  Show description for 4dc8b1a
  panf2333

  committed
  4dc8b1a

  Copy full SHA for 4dc8b1a

• remove invalid zmq benchmark code

  Show description for 0882aff
  panf2333

  committed
  0882aff

  Copy full SHA for 0882aff

• change disagg_prefill example to use zmq

  Show description for a00e630
  panf2333

  committed
  a00e630

  Copy full SHA for a00e630

• change log level and fix some comments

  Show description for 1ebfcaa
  panf2333

  committed
  1ebfcaa

  Copy full SHA for 1ebfcaa

• remove default socket address value

  Show description for 4394ec3
  panf2333

  committed
  4394ec3

  Copy full SHA for 4394ec3

• fix SIM105

  Show description for cc7c582
  panf2333

  committed
  cc7c582

  Copy full SHA for cc7c582