CPU-GPU sync contrary to comment

[Ilykuleshov]

Hi, sorry to bother you again. Seems that this line shouldn't be there, at least according to the comment above. I was able to achieve a significant speed-up by removing this if.

[martenlienen]

You are right that this is inconsistent with the comment. If I remember correctly, this might come down to the problem being solved. Timing this is also non-trivial. If you come to a conclusion about which way is faster, we can either fix the comment or make the code behave as described again. Thanks for looking into this!

[martenlienen]

I have added you as a collaborator to the repository :)

[Ilykuleshov]

I think I know what's going on here: to_be_evaluated.any() indeed causes a blocking CPU-GPU sync point, but:

1. A sync is necessary at each evaluation point anyway, since we can't know how many evaluation points have been passed (it happens at nonzero, see Certain operations cause implicity sync-points pytorch/pytorch#12461).
2. Without evaluation points, this code is unreachable, so no syncs happen.

Theoretically, this could be an issue for sparse evaluation points, but in my experiments I couldn't achieve a speed-up by deferring this sync, so I'm just going to sum all this up in the comment, referencing this issue, and leave things as-is.

[martenlienen]

Good catch that there is a sync point happening anyway!