Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More donation take 2 #1923

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

More donation take 2 #1923

wants to merge 20 commits into from

Conversation

awni
Copy link
Member

@awni awni commented Mar 4, 2025

This is a redux of #1858

  • The race condition of checking use counts is mitigated since the checks are done on the main thread now
  • Use weak pointers to hold references to other inputs to avoid breaking async eval. Another option is to just use a more restrictive policy when doing async eval and avoid this complexity.
  • Also moves all the buffer retention logic from the CPU / GPU task submission to the main thread since it's all duplicated now.

Comment from #1858 still applies (copied below):

Previously if an input didn't get donated it was held by the command buffer and no longer eligible for donation.

So sequences like:

a = x + y
b = mx.abs(x)

Assuming x didn't get donated to a then it is impossible for it to be donated to b.

This changes that by removing buffers that are safe to not hold in the command buffer.

@awni awni force-pushed the faster_cpu_gpu_synch branch from 2511507 to 05167ba Compare March 4, 2025 23:34
@awni awni force-pushed the faster_cpu_gpu_synch branch from 05167ba to 9073755 Compare March 6, 2025 22:22
Base automatically changed from faster_cpu_gpu_synch to main March 7, 2025 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant