Skip to content

Commit

Permalink
[serve] Fix failing test pow 2 scheduler on windows (ray-project#47975)
Browse files Browse the repository at this point in the history
## Why are these changes needed?

Fix `test_pow_2_replica_scheduler.py` on windows. Best guess is asyncio
is slower on windows, so the shortened timeouts for some tests cause the
tests to fail because tasks didn't get a chance to start/finish
executing.

Failing tests on windows:
- `test_multiple_queries_with_different_model_ids`
- `test_queue_len_cache_replica_at_capacity_is_probed`
- `test_queue_len_cache_background_probing`

## Related issue number

Closes ray-project#47950

Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
Signed-off-by: ujjawal-khare <ujjawal.khare@dream11.com>
  • Loading branch information
zcin authored and ujjawal-khare committed Oct 15, 2024
1 parent 10b8be1 commit d0b6bfc
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions python/ray/serve/tests/unit/test_pow_2_replica_scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1253,7 +1253,7 @@ async def test_multiple_queries_with_different_model_ids(self, pow_2_scheduler):
),
]

done, _ = await asyncio.wait(tasks, timeout=0.01)
done, _ = await asyncio.wait(tasks, timeout=0.1)
assert len(done) == len(tasks)

assert all(
Expand Down Expand Up @@ -1600,15 +1600,15 @@ async def test_queue_len_cache_replica_at_capacity_is_probed(pow_2_scheduler):
s.replica_queue_len_cache.update(r1.replica_id, DEFAULT_MAX_ONGOING_REQUESTS)

task = loop.create_task(s.choose_replica_for_request(fake_pending_request()))
done, _ = await asyncio.wait([task], timeout=0.01)
done, _ = await asyncio.wait([task], timeout=0.1)
assert len(done) == 0
# 1 probe from scheduling requests
# + 1 probe from when the replica set was updated with replica r1
assert len(r1.queue_len_deadline_history) - 1 == 1

# Now let the replica respond and accept the request, it should be scheduled.
r1.set_queue_len_response(DEFAULT_MAX_ONGOING_REQUESTS - 1)
done, _ = await asyncio.wait([task], timeout=0.01)
done, _ = await asyncio.wait([task], timeout=0.1)
assert len(done) == 1
assert (await task) == r1

Expand Down Expand Up @@ -1636,7 +1636,7 @@ async def test_queue_len_cache_background_probing(pow_2_scheduler):
s.replica_queue_len_cache.update(r1.replica_id, 0)

task = loop.create_task(s.choose_replica_for_request(fake_pending_request()))
done, _ = await asyncio.wait([task], timeout=0.01)
done, _ = await asyncio.wait([task], timeout=0.1)
assert len(done) == 1
assert (await task) == r1
# 0 probes from scheduling requests
Expand Down

0 comments on commit d0b6bfc

Please sign in to comment.