[proto] Ported SimpleCopyPaste transform #6451
Conversation
| # fetch all images, bboxes, masks and labels from unstructured input | ||
| # with List[image], List[BoundingBox], List[SegmentationMask], List[Label] | ||
| images, bboxes, masks, labels = [], [], [], [] | ||
| for obj in flat_sample: | ||
| if isinstance(obj, PIL.Image.Image, features.Image) or is_simple_tensor(obj): | ||
| images.append(F.to_image_tensor(obj)) | ||
| elif isinstance(obj, features.BoundingBox): | ||
| bboxes.append(obj) | ||
| elif isinstance(obj, features.SegmentationMask): | ||
| masks.append(obj) | ||
| elif isinstance(obj, (features.Label, features.OneHotLabel)): | ||
| labels.append(obj) |
There was a problem hiding this comment.
As our input is a non-collated batch of datapoint, we need to fetch images, bboxes, masks, labels.
We then transform them and finally put into their places.
I'm not a fan of this code but this helps to do what we want. Maybe we could specify a bit more the input structure and thus code forward in a more elegant way.
| output_targets.append(output_data) | ||
|
|
||
| # Insert updated images and targets into input flat_sample | ||
| c0, c1, c2, c3 = 0, 0, 0, 0 |
There was a problem hiding this comment.
Can't we save these indices the first time we iterate over the flat sample? If yes, maybe we can get away with only doing flat_sample[i] = output[i] and only looping over the output.
There was a problem hiding this comment.
yes, we can do that. I just found that passing indices everywhere would be a bit bulky...
|
@vfdev-5 The static typing job is complaining. Seems related. |
|
Seems like git merge has failed as on main we removed PIL and here I'm still using it. |
|
Hey @vfdev-5! You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py |
Summary: * WIP * [proto] Added SimpleCopyPaste transform * Refactored and cleaned the implementation and added tests * Fixing code * Fixed code formatting issue * Minor updates * Fixed merge issue Reviewed By: datumbox Differential Revision: D39013674 fbshipit-source-id: 212f54c4fd9cbbc72011dc331de61106842d0e99 Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Example with MSCoco (batchsize = 2):