Interpolation mode UI problems

[zhiltsov-max]

Old UI:

• interpolation mode is only allowed for boxes. Trying to select another type of shape produces the message that it is allowed for boxes and single points, but pressing on points only produces the same notification. [i0]
• tracks, however, can be created by merging shapes.
  • merging polygons and polylines is allowed, but no interpolation happens, however. [i1]
  • merged shapes have "outside" property set for intermediate frames. May be ok, but enabling again does not incur interpolaiton. [i2]
  • merged shapes have "outside" property set for the last shape. If shape type is points "outside" is not set, however. [i3]

New UI:

• [i0] Merge is allowed for points, but clicks on points conflict with frame dragging logic.
• [i1] After saving, closing and loading the task interpolation finally happens. It is true for polygons and points, but not for polylines.
• "Go to previous keyframe" highlights hidden ("outside") shape on an intermediate frame. "Go to next keyframe" does not.
• Interpolation for one to multiple points happens mostly for 1 point. It would probably be better if all the points were moved uniformly.
• [i2] Interpolation happens all the way long for polygons and points ignoring "outside".
• [i3] last "outside" is ignored for polylines. It looks like for polylines this logic is inverted.

Task 29256, representative test is a task with 4+ frames.

[bsekachev]

but pressing on points only produces the same notification.

It is need to set number of points to 1 before.

merging polygons and polylines is allowed, but no interpolation happens, however

Old CVAT UI (and CVAT server, by the way) does not have algorithm to interpolate polyshapes. We've never supported it before.

merged shapes have "outside" property set for intermediate frames. May be ok, but enabling again does not incur interpolaiton.

I am not sure what you mean enabling again does not incur interpolaiton

Speaking about the new UI, it has some "internal" interpolation algorithms (again, only on client), but they are only beta. To sum up, it is better do not use new UI to work with any tracked polyshapes (merged or drawn) at least because you aren't able to dump got interpolated annotations at all.

[zhiltsov-max]

Old CVAT UI (and CVAT server, by the way) does not have algorithm to interpolate polyshapes. We've never supported it before.

It is, actually, supported on the server for quite a while. It will probably work even for different shape types in the same track.

I am not sure what you mean enabling again does not incur interpolaiton

I mean that there is no way to get interpolated shapes after merging until task annotation window is reopened (to be more correct - until annotations are saved and loaded). For this specific case the situation was: frame 1 has shape, frame 3 has shape, they are merged into a track. The track then has a shape on the frames 2 and 4 with outside property set. One may expect that the track could be interpolated between 1 and 3rd frames if outside is unset, but no. However, after the task is reloaded, tracks are partially interpolated, as described above.

[bsekachev]

It is, actually, supported on the server for quite a while. It will probably work even for different shape types in the same track.

Yes, Nikita did something like that, but there are reasons why this server algorithm doesn't work in the most of cases. So, I wouldn't say server supports it.

What's more, behavior you described, probably happens because of incompatibility between server and client interpretation of poly tracks.

[bsekachev]

I see here some minor bugs in both old and new UI, and I will fix them.
But any interpolation algorithms, finally will be added only in the nearest future (release 1.0).

[bsekachev]

merged shapes have "outside" property set for the last shape. If shape type is points "outside" is not set, however.

Should be fixed in the PR above

Merge is allowed for points, but clicks on points conflict with frame dragging logic.

Should be fixed in the PR above

[i1] After saving, closing and loading the task interpolation finally happens. It is true for polygons and points, but not for polylines.
"Go to previous keyframe" highlights hidden ("outside") shape on an intermediate frame. "Go to next keyframe" does not.
Interpolation for one to multiple points happens mostly for 1 point. It would probably be better if all the points were moved uniformly.
[i2] Interpolation happens all the way long for polygons and points ignoring "outside".
[i3] last "outside" is ignored for polylines. It looks like for polylines this logic is inverted.

Was not able to reproduce. Will try later.

[zhiltsov-max]

Check out the task 29256, where all of this was tested.

[bsekachev]

I tried.

After saving, closing and loading the task interpolation finally happens. It is true for polygons and points, but not for polylines.
"Go to previous keyframe" highlights hidden ("outside") shape on an intermediate frame. "Go to next keyframe" does not.

1. I created three polygons (shapes) on frames 0, 1, 5
2. Merged them
3. Tried Go next/previous keyframe. In both directions it visits the same frames.
4. Saved, reloaded the page
5. No interpolation on frames 2,3,4, > 6 (track is invisible)

Also I could not reproduce:
[i2] Interpolation happens all the way long for polygons and points ignoring "outside". [i3] last "outside" is ignored for polylines. It looks like for polylines this logic is inverted.
Actually I am not sure what should I do to reproduce it. Merge works well.

[bsekachev]

@zhiltsov-max

Can we close the issue?