Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move passthrough for unknown types from dispatchers to transforms #7804

Merged
merged 16 commits into from
Aug 10, 2023

Conversation

pmeier
Copy link
Collaborator

@pmeier pmeier commented Aug 7, 2023

Addresses 2. / 3. of #7747 (comment). This is done by making the noop behavior of _get_kernel optional (turned off by default) and add a new method on the transforms def _call_or_noop(dispatcher, inpt, *args, **kwargs). Instead of calling any dispatcher directly inside _transform, we now go through the new method to retain the noop behavior for the transforms. For example

F.resize(inpt, ...)

changes to

self._call_or_noop(F.resize, inpt, ...)

cc @vfdev-5

@pytorch-bot
Copy link

pytorch-bot bot commented Aug 7, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7804

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 1 New Failure

As of commit 8932f74:

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pmeier pmeier changed the title [PoC] move passthrough for unknown types from dispatchers to transforms move passthrough for unknown types from dispatchers to transforms Aug 7, 2023
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Philip, looks good

Comment on lines +83 to +84
if datapoint_cls in _BUILTIN_DATAPOINT_TYPES:
raise ValueError(f"Kernels cannot be registered for the builtin datapoint classes, but got {datapoint_cls}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is really needed. The previous error message was fine, but anyway.

Note that this means users can't override e.g. erase for bboxes, but this could be a desirable feature.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that exactly what we wanted? We don't want users to register custom kernels for our builtin datapoints. Otherwise

from torchvision.transforms.v2 import functional as F
@F.register_kernel(F.adjust_brightness, datapoints.BoundingBox)
def lol(...):
...

would be valid again.

We could special case F.erase and the other outliers, but I feel users could come to us if they see the warning. We could need the input, since we ourselves are not 100% sure what we should actually do there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just that erase is a bit special in that we probably could have a kernel for it, we just haven't implemented it yet. So it's reasonable for users to want to have their own. But that's not super important.

@pmeier pmeier requested a review from NicolasHug August 10, 2023 10:24
@@ -187,66 +184,6 @@ def test__get_params(self, mocker):
assert params["needs_pad"]
assert any(pad > 0 for pad in params["padding"])

@pytest.mark.parametrize("needs", list(itertools.product((False, True), repeat=2)))
def test__transform(self, mocker, needs):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We remove this test, because it never checked correctness, but rather if the specific dispatchers were being called. Due to our new architecture, they are never called and thus this test fails.

The same argument applies to the removed tests in test/test_transforms_v2.py.

("dispatcher", "registered_input_types"),
[(dispatcher, set(registry.keys())) for dispatcher, registry in _KERNEL_REGISTRY.items()],
)
def test_exhaustive_kernel_registration(dispatcher, registered_input_types):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer have an exhaustive registration and thus this test needs to be removed.

Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments above but non-blocking (we can address in follow-up PRs). Thanks Philip

@pmeier pmeier merged commit db31063 into pytorch:main Aug 10, 2023
@pmeier pmeier deleted the no-passthrough-dispatchers branch August 10, 2023 13:02
@github-actions
Copy link

Hey @pmeier!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

facebook-github-bot pushed a commit that referenced this pull request Aug 25, 2023
…forms (#7804)

Reviewed By: matteobettini

Differential Revision: D48642257

fbshipit-source-id: c562182ae2f6bcf344bf7adc7f5d07cd1da627d3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants