Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CVAT corrupts annotations when importing #7498

Closed
2 tasks done
PMazarovich opened this issue Feb 20, 2024 · 7 comments · Fixed by #7615
Closed
2 tasks done

CVAT corrupts annotations when importing #7498

PMazarovich opened this issue Feb 20, 2024 · 7 comments · Fixed by #7615
Assignees
@ccinv
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@PMazarovich
Copy link
Contributor

PMazarovich commented Feb 20, 2024 •
edited
Loading

Actions before raising this issue

  • I searched the existing issues and did not find anything similar.
  • I read/searched the docs

Steps to Reproduce

How I did it on app.cvat.ai

  1. Create a task with 1 job
  2. Annotate one skeleton track
  3. Create a task with 2 jobs
  4. Export from 1 and import to 3. (cvat for video)
    You'll see that for first job in second task - coordinates of annotations are the same and not changing.
    Demo:
    https://github.com/opencv/cvat/assets/57707673/45018fd7-e70c-4f05-afb6-aa0ff4efa5da
    Annotations:
    DEMO.zip

Expected Behavior

cvat should not corrupt annotations

Possible Solution

No response

Context

No response

Environment

cvat 2.9.2 and app.cvat.ai
@PMazarovich PMazarovich added the bug Something isn't working label Feb 20, 2024
@PMazarovich PMazarovich changed the title CVAT corrupts annotations after import CVAT corrupts annotations when importing Feb 20, 2024
@bsekachev
Copy link
Member

bsekachev commented Feb 20, 2024 •
edited
Loading

Hello,

Only keyframes are imported when you upload annotations.
In your case in the first job of the second task only one keyframe was imported (the second one was ignored because it is out of the job).

@PMazarovich
Copy link
Contributor Author

Hello!
@bsekachev, thanks for the response.
Do you think this is a correct behavior, that when I export the data from one task to another, I receive wrong data in the second one? (regardless of the nature of the task (single job\multi job) and regardless the keyframes)
From my perspective, for the demo that I showed, the track should be divided into 2 (so to say, the second keyframe should be added for the first job) and that way, we'd import the correct annotations.
So, in my opinion, inside the CVAT the appropriate keyframes should be created when it sees that there are multiple jobs and only one track across these jobs.
But if you think this is a correct behavior... Then I can close this issue

@PMazarovich
Copy link
Contributor Author

The idea is that keyframes actually does matter comparing to annotation coordinates. That is why, as for me, it is better to add some keyframes rather then loose a bunch of annotations

@bsekachev
Copy link
Member

bsekachev commented Feb 23, 2024 •
edited
Loading

You are right, that looks weird

@nmanovic nmanovic added the good first issue Good for newcomers label Mar 5, 2024
@ccinv
Copy link
Contributor

ccinv commented Mar 10, 2024

Hello @nmanovic, I am interested in this issue, can you assign it to me?

@ccinv
Copy link
Contributor

ccinv commented Mar 14, 2024

i'm trying to solve this issue by a adding some interpolation to the improrted data patch part, for which I think it's best to inserted in slice function of tracks.

https://github.com/opencv/cvat/blob/f22322f4349a91bbb645dfcacf1e799d870a6f16/cvat/apps/dataset_manager/annotation.py#L131-L146

modify the for loop starting from 139 could work. But I can't figure out what track['shapes'] means and what's the difference with track['elements']. Could you please give me a hint? @bsekachev @nmanovic

@bsekachev
Copy link
Member

track['shapes'] contains positions data of the track on different frames
track['elements'] is about skeletons, which are complex shapes consisted of other shapes (points). This is array of other "children" tracks.

@ccinv ccinv mentioned this issue Mar 15, 2024
Merged
6 tasks
zhiltsov-max added a commit that referenced this issue Mar 28, 2024
@ccinv @zhiltsov-max
[GSoC2024] fix skeleton tracks slice between jobs (#7615)
dcbff88 
Fixes #7498

In #6968, sub tracks splitting has been fixed but not entirely, if the
user wants to import tracks between more than 2 jobs it can cause
malformed annotation like #7498.

In the pervious test case,
https://github.com/opencv/cvat/blob/d1a300f0189744bef01d2c60135ec474c5004fbc/tests/python/rest_api/test_tasks.py#L543-L563
skeleton track is static so it did not identify any issues.

<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
Pervious interpolate method will drop the last frame. That's correct for
most cases, but if slices and tracks are like this:

slice: -------------xxxxx---------
track: ---------xxxxxxxxxx-------

We have to keep the last frame of slice in the track. If not, this track
won't close.

### How has this been tested?
I rewrited the testcase `test_can_split_skeleton_tracks_on_jobs`
Now it test the split function with a longer input track and stricter
interpolate reference.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
~~- [ ] I have updated the documentation accordingly~~
- [x] I have added tests to cover my changes
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [x] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

---------

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
g-kartik pushed a commit to g-kartik/cvat that referenced this issue Mar 29, 2024
@ccinv @zhiltsov-max @g-kartik
[GSoC2024] fix skeleton tracks slice between jobs (cvat-ai#7615)
c53d69a 
Fixes cvat-ai#7498

In cvat-ai#6968, sub tracks splitting has been fixed but not entirely, if the
user wants to import tracks between more than 2 jobs it can cause
malformed annotation like cvat-ai#7498.

In the pervious test case,
https://github.com/opencv/cvat/blob/d1a300f0189744bef01d2c60135ec474c5004fbc/tests/python/rest_api/test_tasks.py#L543-L563
skeleton track is static so it did not identify any issues.

<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
Pervious interpolate method will drop the last frame. That's correct for
most cases, but if slices and tracks are like this:

slice: -------------xxxxx---------
track: ---------xxxxxxxxxx-------

We have to keep the last frame of slice in the track. If not, this track
won't close.

### How has this been tested?
I rewrited the testcase `test_can_split_skeleton_tracks_on_jobs`
Now it test the split function with a longer input track and stricter
interpolate reference.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
~~- [ ] I have updated the documentation accordingly~~
- [x] I have added tests to cover my changes
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [x] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

---------

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ccinv ccinv

Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[GSoC2024] fix skeleton tracks slice between jobs ccinv/cvat
4 participants
@ccinv @bsekachev @nmanovic @PMazarovich