Add tuto for custom transforms and custom datapoints in gallery example #7795

NicolasHug · 2023-08-02T16:50:39Z

Bunch of docs.

pytorch-bot · 2023-08-02T16:50:42Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7795

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ 3 Unrelated Failures

As of commit 7be27e8:

BROKEN TRUNK - The following jobs failed but were present on the merge base bf03f4e:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

gallery/plot_datapoints.py

NicolasHug · 2023-08-02T17:13:37Z

gallery/plot_datapoints.py

 # Do I have to wrap the output of the datasets myself?
 # ----------------------------------------------------
 #
+# TODO: Move this in another guide - this is user-facing, not dev-facing.


Will address eventually and open an issue so we don't forget. Just LMK if this is OK in principle.

Maybe

vision/gallery/plot_transforms_v2.py

Line 3 in 9ebf10a

Getting started with transforms v2

?

docs/source/conf.py

torchvision/datapoints/_bounding_box.py

NicolasHug · 2023-08-03T11:06:11Z

torchvision/datapoints/_datapoint.py

@@ -25,9 +32,13 @@ def _to_tensor(
            requires_grad = data.requires_grad if isinstance(data, torch.Tensor) else False
        return torch.as_tensor(data, dtype=dtype, device=device).requires_grad_(requires_grad)

+    @classmethod
+    def _wrap(cls: Type[D], tensor: torch.Tensor) -> D:
+        return tensor.as_subclass(cls)


I ended up moving _wrap and wrap_like in the base class. They were all implemented exactly in the same way for all datapoints except for BoundingBoxes, because that one requires meta-data.

I think it's OK to assume that these are a good default implementations for most datapoints, i.e. those that don't require meta-data. What can go wrong anyway?

It makes the example much easier to write and explain, and more importantly it makes the Datapoint authoring experience much smoother. So I think we should do that. I can do it in another PR if you prefer though.

I'm ok leaving it here.

NicolasHug · 2023-08-03T12:15:23Z

torchvision/datapoints/_datapoint.py

    @classmethod
    def wrap_like(cls: Type[D], other: D, tensor: torch.Tensor) -> D:
-        raise NotImplementedError
+        return cls._wrap(tensor)


Wait, so now I'm a bit confused.
We clearly don't use other so we should remove it from there.

BUT then:

don't we want to allow wrap_like(other, t) so that t gets the same device (and dtype??) as other?? we don't seem to do that

do we eve need wrap_like for those datapoints that don't have meta-data? What's the point of doing Image.wrap_like(other, t) or even Image.wrap_like(t) (not supported) when one can just do Image(t)?

Anyway, I'd suggest to keep things as-is for now, and to address that stuff after this PR is merged.

The reason we have wrap_like the way it is, is that we use it inside __torch_function__:

vision/torchvision/datapoints/_datapoint.py

Lines 32 to 39 in 9ebf10a

_NO_WRAPPING_EXCEPTIONS = {

torch.Tensor.clone: lambda cls, input, output: cls.wrap_like(input, output),

torch.Tensor.to: lambda cls, input, output: cls.wrap_like(input, output),

torch.Tensor.detach: lambda cls, input, output: cls.wrap_like(input, output),

# We don't need to wrap the output of `Tensor.requires_grad_`, since it is an inplace operation and thus

# retains the type automatically

torch.Tensor.requires_grad_: lambda cls, input, output: output,

}

Basically, we need a generic way for "Here is a datapoint input possibly with some metadata. Given a plain tensor, make it the same type as the datapoint and copy the metadata if available".

So the reason we have other in all of them is because if we need metadata and pass nothing explicitly, we need to take it from other. This also means that on datapoints that have no metadata, we never use other.

OK. I understand this is useful for us internally but I'm still questioning whether we should expose that as public for datapoints that don't have meta-data. Let's just keep things as-is in this PR so we can merge it soon, and revisit just after.

pmeier

Looking good, thanks Nicolas! I left a bunch of comments.

docs/source/datapoints.rst

gallery/plot_custom_datapoints.py

torchvision/datapoints/_bounding_box.py

pmeier · 2023-08-03T22:05:27Z

torchvision/datapoints/_datapoint.py

@@ -25,9 +32,13 @@ def _to_tensor(
            requires_grad = data.requires_grad if isinstance(data, torch.Tensor) else False
        return torch.as_tensor(data, dtype=dtype, device=device).requires_grad_(requires_grad)

+    @classmethod
+    def _wrap(cls: Type[D], tensor: torch.Tensor) -> D:
+        return tensor.as_subclass(cls)


I'm ok leaving it here.

torchvision/datapoints/_datapoint.py

gallery/plot_datapoints.py

torchvision/transforms/v2/functional/_utils.py

gallery/plot_datapoints.py

NicolasHug

Thanks Philip, I addressed most comments or left some thoughts on the rest. LMK.

gallery/plot_custom_datapoints.py

gallery/plot_datapoints.py

NicolasHug · 2023-08-04T09:04:15Z

gallery/plot_datapoints.py

+        target["bboxes"] = datapoints.BoundingBoxes(
+            bboxes,


I'm not sure, I think we want to be consistent with the names used in this tutorial. What's being used in another tutorial is lower in priority when it comes to consistency

torchvision/datapoints/_datapoint.py

torchvision/transforms/v2/functional/_utils.py

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

pmeier

Thanks Nicolas, awesome to have better docs now!

github-actions · 2023-08-04T16:13:35Z

Hey @NicolasHug!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

…ery example (#7795) Summary: Co-authored-by: Philip Meier <github.pmeier@posteo.de> Reviewed By: matteobettini Differential Revision: D48642308 fbshipit-source-id: efe98a8405393c0bf0ca84141b895265d93636d0

NicolasHug added 3 commits August 2, 2023 15:30

Use syntax for gallery examples

270f4f0

Add gallery tuto for custom transforms

d1d8aa9

Merge branch 'main' of github.com:pytorch/vision into moaeinfoaenf

24ca6cc

facebook-github-bot added the cla signed label Aug 2, 2023

NicolasHug commented Aug 2, 2023

View reviewed changes

gallery/plot_datapoints.py Show resolved Hide resolved

More stuff

937069d

NicolasHug commented Aug 2, 2023

View reviewed changes

NicolasHug added 3 commits August 2, 2023 18:15

not tensor-like

5780fd7

Some more

cbff036

Allow register_kernel to take name as input

43a71e7

NicolasHug mentioned this pull request Aug 2, 2023

Allow register_kernel() to take dispatcher name as input #7796

Merged

pmeier linked an issue Aug 2, 2023 that may be closed by this pull request

Transforms V2 proposal: Make Datapoint public and open an API for 3rd party function injection/dipatch #7028

Closed

NicolasHug added 6 commits August 3, 2023 09:54

Better _name_to_dispatcher

99ea401

Merge branch 'main' of github.com:pytorch/vision into register_str

12fb8e1

Merge branch 'main' of github.com:pytorch/vision into moaeinfoaenf

abe4f05

Merge branch 'register_str' into moaeinfoaenf

24036da

More docs

88bf09b

Merge branch 'main' of github.com:pytorch/vision into moaeinfoaenf

1ad2465

NicolasHug commented Aug 3, 2023

View reviewed changes

docs/source/conf.py Show resolved Hide resolved

NicolasHug commented Aug 3, 2023

View reviewed changes

torchvision/datapoints/_bounding_box.py Show resolved Hide resolved

oops

bedb858

NicolasHug commented Aug 3, 2023

View reviewed changes

NicolasHug marked this pull request as ready for review August 3, 2023 11:06

NicolasHug requested review from vfdev-5 and pmeier August 3, 2023 11:07

NicolasHug commented Aug 3, 2023

View reviewed changes

pmeier reviewed Aug 3, 2023

View reviewed changes

gallery/plot_datapoints.py Outdated Show resolved Hide resolved

Address comments

8178376

NicolasHug commented Aug 4, 2023

View reviewed changes

NicolasHug and others added 2 commits August 4, 2023 14:02

Update torchvision/transforms/v2/functional/_utils.py

5f58689

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

Merge branch 'main' into moaeinfoaenf

7be27e8

pmeier approved these changes Aug 4, 2023

View reviewed changes

NicolasHug merged commit 84db2ac into pytorch:main Aug 4, 2023

NicolasHug mentioned this pull request Aug 4, 2023

Move v2 dataset example #7802

Closed

NicolasHug added enhancement module: documentation labels Aug 4, 2023

NicolasHug mentioned this pull request Aug 4, 2023

re-think _wrap and wrap_like #7803

Closed

pmeier mentioned this pull request Sep 25, 2023

fix weight docs #7993

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add tuto for custom transforms and custom datapoints in gallery example #7795

Add tuto for custom transforms and custom datapoints in gallery example #7795

NicolasHug commented Aug 2, 2023 •

edited

Loading

pytorch-bot bot commented Aug 2, 2023 •

edited

Loading

NicolasHug Aug 2, 2023

pmeier Aug 3, 2023

NicolasHug Aug 3, 2023 •

edited

Loading

pmeier Aug 3, 2023

NicolasHug Aug 3, 2023

pmeier Aug 3, 2023

NicolasHug Aug 3, 2023

pmeier left a comment

pmeier Aug 3, 2023

NicolasHug left a comment •

edited

Loading

NicolasHug Aug 4, 2023

pmeier left a comment

github-actions bot commented Aug 4, 2023

	_NO_WRAPPING_EXCEPTIONS = {
	torch.Tensor.clone: lambda cls, input, output: cls.wrap_like(input, output),
	torch.Tensor.to: lambda cls, input, output: cls.wrap_like(input, output),
	torch.Tensor.detach: lambda cls, input, output: cls.wrap_like(input, output),
	# We don't need to wrap the output of `Tensor.requires_grad_`, since it is an inplace operation and thus
	# retains the type automatically
	torch.Tensor.requires_grad_: lambda cls, input, output: output,
	}

Add tuto for custom transforms and custom datapoints in gallery example #7795

Add tuto for custom transforms and custom datapoints in gallery example #7795

Conversation

NicolasHug commented Aug 2, 2023 • edited Loading

pytorch-bot bot commented Aug 2, 2023 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7795

✅ 3 Unrelated Failures

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasHug Aug 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmeier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NicolasHug left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmeier left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 4, 2023

NicolasHug commented Aug 2, 2023 •

edited

Loading

pytorch-bot bot commented Aug 2, 2023 •

edited

Loading

NicolasHug Aug 3, 2023 •

edited

Loading

NicolasHug left a comment •

edited

Loading