Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Dataset Adapter: Avoid duplicated annotation and permit empty image #1873

Merged
merged 8 commits into from
Mar 16, 2023
Merged

[FIX] Dataset Adapter: Avoid duplicated annotation and permit empty image #1873

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
cf87bec
Add logic to add empty label image
sungmanc Mar 9, 2023
b2ef4a5
Pre-commit
sungmanc Mar 9, 2023
b2e3fec
Remove useless lines
sungmanc Mar 10, 2023
eb677ee
Merge branch 'develop' of https://github.com/openvinotoolkit/training…
sungmanc Mar 10, 2023
11254e5
Empty label should be included in validation
sungmanc Mar 10, 2023
871d855
Update otx/core/data/adapter/detection_dataset_adapter.py
sungmanc Mar 10, 2023
23ae3c7
Fix pre-commit
sungmanc Mar 10, 2023
044e2c1
Fix mistypo
sungmanc Mar 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion otx/core/data/adapter/detection_dataset_adapter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def get_otx_dataset(self) -> DatasetEntity:
if ann.label not in used_labels:
used_labels.append(ann.label)

if len(shapes) > 0 or subset == Subset.UNLABELED or len(datumaro_item.annotations) == 0:
if len(shapes) > 0 or subset == Subset.UNLABELED or (subset == Subset.VALIDATION and len(datumaro_item.annotations)) == 0:
sungmanc marked this conversation as resolved.
Show resolved Hide resolved
dataset_item = DatasetItemEntity(image, self._get_ann_scene_entity(shapes), subset=subset)
dataset_items.append(dataset_item)
self.remove_unused_label_entities(used_labels)
Expand Down
2 changes: 1 addition & 1 deletion otx/core/data/adapter/segmentation_dataset_adapter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def get_otx_dataset(self) -> DatasetEntity:
if d_polygon.label not in used_labels:
used_labels.append(d_polygon.label)

if len(shapes) > 0 or subset == Subset.UNLABELED:
if len(shapes) > 0 or subset == Subset.UNLABELED == 0:
dataset_item = DatasetItemEntity(image, self._get_ann_scene_entity(shapes), subset=subset)
dataset_items.append(dataset_item)

Expand Down
10 changes: 0 additions & 10 deletions tests/assets/car_tree_bug/annotations/instances_train.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,6 @@
"flickr_url": "",
"coco_url": "",
"date_captured": 0
},
{
"id": 7,
"width": 1280,
"height": 720,
"file_name": "Slide3.PNG",
"license": 0,
"flickr_url": "",
"coco_url": "",
"date_captured": 0
}
],
"annotations": [
Expand Down
12 changes: 11 additions & 1 deletion tests/assets/car_tree_bug/annotations/instances_val.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,17 @@
"id": 8,
"width": 1280,
"height": 720,
"file_name": "Slide20.PNG",
"file_name": "Slide4.PNG",
"license": 0,
"flickr_url": "",
"coco_url": "",
"date_captured": 0
},
{
"id": 9,
"width": 1280,
"height": 720,
"file_name": "Slide5.PNG",
"license": 0,
"flickr_url": "",
"coco_url": "",
Expand Down
0 ...ssets/car_tree_bug/images/val/Slide20.PNG → ...assets/car_tree_bug/images/val/Slide4.PNG
View file
Open in desktop
File renamed without changes
0 ...sets/car_tree_bug/images/train/Slide3.PNG → ...assets/car_tree_bug/images/val/Slide5.PNG
View file
Open in desktop
File renamed without changes
2 changes: 0 additions & 2 deletions tests/unit/core/data/test_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,6 @@ def generate_datumaro_dataset_item(
task: str,
image_shape: np.array = np.array((5, 5, 3)),
mask_shape: np.array = np.array((5, 5)),
empty_label: bool = False,
) -> dm.DatasetItem:
"""Generate Datumaro DatasetItem.

Expand All @@ -96,7 +95,6 @@ def generate_datumaro_dataset_item(
task (str): task type, e.g. "classification"
image_shape (np.array): the shape of image.
image_shape (np.array): the shape of mask.
empty_label (bool): make dataset item with no annotation.

Returns:
dm.DatasetItem: Datumaro DatasetItem
Expand Down