Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable h-label classification #2761

Merged
merged 18 commits into from
Jan 11, 2024
Merged

Conversation

sungmanc
Copy link
Contributor

@sungmanc sungmanc commented Jan 5, 2024

Summary

This PR introduces the H-label Classification for OTX2.0. The final benchmark table will be uploaded soon.
Also, this PR will be edited after migrating #2758.

Notes,

  • In this PR, the model metric is different from the OTX1.5. In the OTX1.5, the final metric is [top-1-acc. + mAP], however, I edited the metric to [top-1-acc + multilabel-acc] since it is more reasonable for me. Changing the model metric at the OTX side doesn't affect to the Geti.

The figure below represents the simple design of H-label data part
image

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added TEST Any changes in tests OTX 2.0 labels Jan 5, 2024
@sungmanc sungmanc marked this pull request as ready for review January 5, 2024 08:20
@sungmanc sungmanc requested a review from sovrasov January 5, 2024 08:23
@sungmanc
Copy link
Contributor Author

sungmanc commented Jan 8, 2024

@vinnamkim , thanks for the comments, I'll update this PR after merging this PR (#2758) since it brings the DataMetaInfo what I need.

@sovrasov
Copy link
Contributor

sovrasov commented Jan 8, 2024

When label resolving functionality is going to be added? As I see now, we have only metric computation, and therefore we don't need to produce human-readable predictions of the model, but in the future obtaining postprocessed results is also good to add (at least for Geti).

@sungmanc
Copy link
Contributor Author

sungmanc commented Jan 9, 2024

@jaegukhyun , @vinnamkim

This commit(fdc033e)

  • introduces the HLabelMetaInfo and changed the location of DataMetaInfo. For the h-label classification, HLabelClsDataset need the information of h-label, however, in the previous implementation it is located at the DataModule side. That's the reason why I changed the location.
  • Also I added the SubsetDataMetaInfo and DataMetaInfo to represent the information for all subsets (train, val, test). Currently, there are no cases that have different data classes between train and val. However, we need to care about the semi or self manner training.

@sungmanc
Copy link
Contributor Author

sungmanc commented Jan 9, 2024

When label resolving functionality is going to be added? As I see now, we have only metric computation, and therefore we don't need to produce human-readable predictions of the model, but in the future obtaining postprocessed results is also good to add (at least for Geti).

It seems great, I'll add that functionality at the later phase.

Copy link
Contributor

@vinnamkim vinnamkim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@sungmanc sungmanc merged commit 7843f3b into openvinotoolkit:v2 Jan 11, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants