[Core] Cherry pick streaming generator PRs

[rkooo567]

What happened + What you expected to happen

We want to support the streaming generator for ray serve by Ray 2.5. We need to cherry-pick all these commits.

• [1/N] Streaming Generator. Cpp interfaces and implementation #35291
  • Cherry-pick [1/N] Streaming Generator. Cpp interfaces and implementation (#35291) #35656
• [2/N] Streaming Generator. Support core worker APIs + cython generator interface. #35324
  • [Cherry pick][2/N] Streaming Generator. Support core worker APIs + cython generat… #35682
• [3/N] Streaming Generator. E2e integration #35325
  • [Cherry-pick][3/N] Streaming Generator. E2e integration (#35325) #35708
• [4/N] Support async actor and async generator interface.  #35584
  • [Cherry pick] [4/N] Support async actor and async interface.  #35752
• [Streaming Generator] Fix a reference leak when a stream is deleted with out of order writes. #35591
  • [Cherry-pick][Streaming Generator] Fix a reference leak when a stream is deleted w… #35707
• [Test] Move generators test to large #35747
  • [Cherry pick] [Test] Move generators test to large (#35747) #35772
• [Streaming Generator] Fix a reference leak when pinning requests are received after refs are consumed. #35712
  • [Cherry-pick][Streaming Generator] Fix a reference leak when pinning requests are … #35794
• [Streaming Generator] Allow to raise an exception to avoid check failures. #35766
  • [Cherry pick][Streaming Generator] Allow to raise an exception to avoid check fail… #35803
• [serve] Add experimental support for StreamingResponse using RayObjectRefGenerator #35720
  • [serve] Cherry pick Ray Serve experimental StreamingResponse support #35811

Versions / Dependencies

master

Reproduction script

n/a

Issue Severity

None

[can-anyscale]

This is an additional feature instead of a a release-blocker fix. Can this wait until 2.6?

[xieus]

@can-anyscale This streaming feature is a top priority in collaboration with Serve team. If it doesn't ship, we may have a delayed delivery of LLM serving feature. We should prioritize this.

[can-anyscale]

@xieus ack!

[zhe-thoughts]

cc @waleedkadous @edoakes

Yes, @can-anyscale I added you to an email thread

[edoakes]

There will be additional Ray Serve PRs to pick, should all be ready by EOW

[ArturNiederfahrenhorst]

@zhe-thoughts @edoakes @rkooo567
As far as I can tell, all the PRs listed here have been merged.
If no tests fail, is this release-blocker resolved?

[zhe-thoughts]

I agree it can be resolved.

[rkooo567]

Should be good to go!