[GSoC2024] fix skeleton tracks slice between jobs (cvat-ai#7615)

Fixes cvat-ai#7498

In cvat-ai#6968, sub tracks splitting has been fixed but not entirely, if the
user wants to import tracks between more than 2 jobs it can cause
malformed annotation like cvat-ai#7498.

In the pervious test case,
https://github.com/opencv/cvat/blob/d1a300f0189744bef01d2c60135ec474c5004fbc/tests/python/rest_api/test_tasks.py#L543-L563
skeleton track is static so it did not identify any issues.

<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
Pervious interpolate method will drop the last frame. That's correct for
most cases, but if slices and tracks are like this:

slice: -------------xxxxx---------
track: ---------xxxxxxxxxx-------

We have to keep the last frame of slice in the track. If not, this track
won't close.

### How has this been tested?
I rewrited the testcase `test_can_split_skeleton_tracks_on_jobs`
Now it test the split function with a longer input track and stricter
interpolate reference.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
~~- [ ] I have updated the documentation accordingly~~
- [x] I have added tests to cover my changes
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [x] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

---------

Co-authored-by: Maxim Zhiltsov <zhiltsov.max35@gmail.com>

changelog.d/20240315_183623_avaicode.md

@@ -0,0 +1,5 @@
+### Fixed
+
+- Changed interpolation behavior in `annotation.py`, now correctly keep the last frame
+- Insert last frame if it is key to the track, fixes data corruption when tracks crossing more than 1 jobs
+  (<https://github.com/opencv/cvat/pull/7615>)

cvat/apps/dataset_manager/annotation.py

@@ -107,13 +107,16 @@ def filter_track_shapes(shapes):
 
         if len(segment_shapes) < len(track['shapes']):
             interpolated_shapes = TrackManager.get_interpolated_shapes(
-                track, start, stop, dimension)
+                track, start, stop + 1, dimension)
             scoped_shapes = filter_track_shapes(interpolated_shapes)
 
             if scoped_shapes:
+                last_key = sorted(track['shapes'], key=lambda s: s['frame'])[-1]['frame']
                 if not scoped_shapes[0]['keyframe']:
                     segment_shapes.insert(0, scoped_shapes[0])
-                if scoped_shapes[-1]['keyframe'] and \
+                if last_key >= stop and scoped_shapes[-1]['points'] != segment_shapes[-1]['points']:
+                    segment_shapes.append(scoped_shapes[-1])
+                elif scoped_shapes[-1]['keyframe'] and \
                         scoped_shapes[-1]['outside']:
                     segment_shapes.append(scoped_shapes[-1])
                 elif stop + 1 < len(interpolated_shapes) and \

tests/python/rest_api/test_tasks.py

@@ -557,9 +557,12 @@ def test_can_split_skeleton_tracks_on_jobs(self, jobs):
                             "label_id": 59,
                             "frame": 0,
                             "shapes": [
+                                # https://github.com/opencv/cvat/issues/7498
+                                # https://github.com/opencv/cvat/pull/7615
+                                # This shape covers frame 0 to 7,
+                                # We need to check if frame 5 is generated correctly for job#1
                                 {"type": "points", "frame": 0, "points": [1.0, 2.0]},
-                                {"type": "points", "frame": 2, "points": [1.0, 2.0]},
-                                {"type": "points", "frame": 7, "points": [1.0, 2.0]},
+                                {"type": "points", "frame": 7, "points": [2.0, 4.0]},
                             ],
                         },
                     ],
@@ -588,8 +591,22 @@ def test_can_split_skeleton_tracks_on_jobs(self, jobs):
             track = job_annotations["tracks"][0]
             assert track.get("elements", []), "Expected to see track with elements"
 
+            def interpolate(frame):
+                # simple interpolate from ([1, 2], 1) to ([2, 4], 7)
+                return [(2.0 - 1.0) / 7 * (frame - 0) + 1.0, (4.0 - 2.0) / 7 * (frame - 0) + 2.0]
+
             for element in track["elements"]:
                 element_frames = set(shape["frame"] for shape in element["shapes"])
+                assert all(
+                    [
+                        not DeepDiff(
+                            interpolate(shape["frame"]), shape["points"], significant_digits=2
+                        )
+                        for shape in element["shapes"]
+                        if shape["frame"] >= 0 and shape["frame"] <= 7
+                    ]
+                )
+                assert len(element["shapes"]) == 2
                 assert element_frames <= job_frame_range, "Track shapes get out of job frame range"