Unexpected behaviour with task `meta` #4179

Sigura · 2022-01-14T22:59:15Z

My actions before raising this issue

Read/searched the docs
Searched past issues

Steps to Reproduce (for bugs)

Make manifest.jsonl with own order (not abc) on aws (2D images)
Add manifest.jsonl to cloud storage
Create tasks with same manifest.jsonl
In task creation form files in expected order (as in manifest.jsonl)
After creation file names show in abc order in "player"
But files shown as it is in manifest.jsonl file

Export to COCO mess too – file names and order.

This endpoint shown files in expected order:
cloudstorages/{store_id}/content?manifest_path=manifest.jsonl

But tasks/{new-task-id}/data/meta return abc order. (looks like abc)

I checked folder /home/django/data/storages/{store_id}/{name}/manifest.jsonl it has expected order

Possible Solution

Let users make own order in manifest.jsonl and use it as it is

And restore it from manifest or form request.data['server_files']

Steps:

Remove all unexpected sorted from code

Found related:

Fix order after serializer creation

~~and I don't know why data['server_files'] income reversed?!!~~

now I know why it may come in any order, because find the root of this evil

https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/engine/views.py#L630

Because looks like serializer does not guaranties income order, is not it?

But it fixable there – just rebuild it if from request.data['server_files'].

Your Environment

Author: Maria Khrustaleva <maya17grd@gmail.com>
Date:   Wed Dec 1 16:57:31 2021 +0300

    Remove key_file_path field (#3959)

    * Remove key_file_path field
    * fix
    * Update comparison

The text was updated successfully, but these errors were encountered:

Sigura · 2022-01-15T07:42:54Z

I found place where file order is wrong

> docker exec -u root -it cvat_db sh
> psql -U root -d cvat
# select * from engine_image where path ilike '%{task/path}%' order by frame limit 12;

Sigura · 2022-01-15T07:57:53Z

For me it looks like /data/meta sorted in abc order and manifest.jsonl has own order

and order of server_files in POST request to tasks/{task-id}/data ignored

that is why annotation mess

Marishka17 · 2022-01-17T06:34:53Z

@Sigura, Hi, you can specify Sorting method in the advanced section of the task creation form. In your case, you can choose Predefined sorting.
See here for details.

Sigura · 2022-01-17T06:40:20Z

@Sigura, Hi, you can specify Sorting method in the advanced section of the task creation form. In your case, you can choose Predefined sorting. See here for details.

I know, but I think it's the wrong way.

CVAT can use a user-defined order in the manifest file, and then all those constructions do not need.

AND it's the matter! Bug with wrong frame order persists too.

Marishka17 · 2022-01-17T13:29:29Z

@Sigura, Hi, you can specify Sorting method in the advanced section of the task creation form. In your case, you can choose Predefined sorting. See here for details.

I know, but I think it's the wrong way.

CVAT can use a user-defined order in the manifest file, and then all those constructions do not need.

The manifest file in the current implementation plays the role of a list of files in the cloud storage. And I believe that we should not limit the order of the files to the order in the manifest file.

AND it's the matter! Bug with wrong frame order persists too.

I'll investigate it. Thanks.

gHerzmann · 2022-04-05T12:47:43Z

@Sigura, Hi, you can specify Sorting method in the advanced section of the task creation form. In your case, you can choose Predefined sorting. See here for details.

When I try to use the predefined sorting method, I receive the following error:
Could not create the task on the server. Exception: It isn't supported to upload manifest file and use random sorting.

See:
https://github.com/openvinotoolkit/cvat/blob/6fad1764efd922d99dbcda28c4ee72d071aa5a07/cvat/apps/engine/task.py#L374

It would be great if we could select a sorting method that would use the same order defined in the manifest file.

Sigura · 2022-04-05T14:00:15Z

@gHerzmann I removed all sorting from source code and fixed it in request.data['server_files'] and rebuild images

Because do not understand why anybody need to setup it in another place than manifest

Fixes #5061, #4179 - Added a way to declare custom file ordering for the local task data uploads via TUS protocol - Added an option to use a manifest to support the `predefined` sorting method - This file is required for the `predefined` sorting mode with image archives - Fixed file ordering when tasks are created from SDK or CLI in the `predefined` sorting mode - Added more tests for task data uploading API The uploading protocol is implemented: The user specifies `sorting_method=predefined` if the task creation request. Then the data is uploaded. 1. Client files uploading 1.1. The files are uploaded as separate files (using the TUS protocol) or grouped files (using the `Upload-Multiple` requests). 1.2. The `Upload-Finish` request comes (or its unlabeled legacy equivalent). The new optional field can be supplied: `upload_file_order` - a list of strings. It allows to override the input file order, if necessary, and is only valid with the `predefined` sorting method specified. 1.2.1. If the field is empty or missing, the client files in the data requests are considered ordered. 1.2.2. If the field is not empty, a list containing the file list in the required order is expected in the `upload_file_order` field. 1.2.2.1. If there are `client_files` in the request, the files are sorted 1.2.2.2. If file lists mismatch, an explanatory error is raised. 2. Data processing 2.1. At this point, all `*_files` are considered ordered as requested. 2.2. Require a metafile for zip uploads with predefined sorting. The file is expected to accompany the uploaded zip file, not to be inside of the archive. 2.3. If there is a metafile in the input data, files are ordered after the metafile. 2.3.1. If the data is extracted from cloud, only the specified subset of the files is kept in the manifest. 2.3.2. If the upload data doesn't exist in the metafile, an error is raised. 2.3.3. A `job_file_mapping` has higher priority than metafile, if specified. Co-authored-by: Roman Donchenko <roman@cvat.ai>

zhiltsov-max · 2023-06-08T12:55:14Z

@gHerzmann, @Sigura

It would be great if we could select a sorting method that would use the same order defined in the manifest file.

Possible Solution
Let users make own order in manifest.jsonl and use it as it is
And restore it from manifest or form request.data['server_files']

Added in #5083

Marishka17 added the question Further information is requested label Jan 17, 2022

Marishka17 added bug Something isn't working and removed question Further information is requested labels Jan 17, 2022

Marishka17 self-assigned this Jan 18, 2022

zhiltsov-max mentioned this issue Mar 14, 2023

Fix predefined sorting for task data #5083

Merged

7 tasks

bsekachev closed this as completed Jun 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unexpected behaviour with task `meta` #4179

Unexpected behaviour with task `meta` #4179

Sigura commented Jan 14, 2022 •

edited

Loading

Sigura commented Jan 15, 2022 •

edited

Loading

Sigura commented Jan 15, 2022 •

edited

Loading

Marishka17 commented Jan 17, 2022

Sigura commented Jan 17, 2022

Marishka17 commented Jan 17, 2022

gHerzmann commented Apr 5, 2022

Sigura commented Apr 5, 2022

zhiltsov-max commented Jun 8, 2023

Unexpected behaviour with task meta #4179

Unexpected behaviour with task meta #4179

Comments

Sigura commented Jan 14, 2022 • edited Loading

My actions before raising this issue

Steps to Reproduce (for bugs)

Possible Solution

Your Environment

Sigura commented Jan 15, 2022 • edited Loading

Sigura commented Jan 15, 2022 • edited Loading

Marishka17 commented Jan 17, 2022

Sigura commented Jan 17, 2022

Marishka17 commented Jan 17, 2022

gHerzmann commented Apr 5, 2022

Sigura commented Apr 5, 2022

zhiltsov-max commented Jun 8, 2023

Unexpected behaviour with task `meta` #4179

Unexpected behaviour with task `meta` #4179

Sigura commented Jan 14, 2022 •

edited

Loading

Sigura commented Jan 15, 2022 •

edited

Loading

Sigura commented Jan 15, 2022 •

edited

Loading