Annotation formats documentation

cvat/apps/annotation/README.md

@@ -131,7 +131,7 @@ It allows to download and upload annotations in different formats and easily add
 
         annotations.add_shape(shape)
     ```
-    Full examples can be found in [builtin](builtin) folder.
+    Full examples can be found in corrseponding *.py files (cvat.py, coco.py, yolo.py, etc.).
 1.  Add path to a new python script to the annotation app settings:
 
     ```python
@@ -150,3 +150,353 @@ It allows to download and upload annotations in different formats and easily add
   Possible solutions: install additional modules via pip call to a separate directory for each Annotation Format
   to reduce version conflicts, etc. Thus, custom code can be run in an extended environment, and core CVAT modules
   should not be affected. As well, this functionality can be useful for Auto Annotation module.
+
+## Format specifications
+
+### CVAT
+This is native CVAT annotation format.
+[Detailed format description](cvat/apps/documentation/xml_format.md)
+
+#### CVAT XML for images dumper
+- downloaded file: Single unpacked XML
+- supported shapes - Rectangles, Polygons, Polylines, Points
+
+#### CVAT XML for videos dumper
+- downloaded file: Single unpacked XML
+- supported shapes - Rectangles, Polygons, Polylines, Points
+
+#### CVAT XML Loader
+- uploaded file: Single unpacked XML
+- supported shapes - Rectangles, Polygons, Polylines, Points
+
+### [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/)
+
+#### Pascal dumper description
+- downloaded file: a zip archive with following structure:
+  ```bash
+  taskname.zip
+  ├── frame_000001.xml
+  ├── frame_000002.xml
+  ├── frame_000003.xml
+  └── ...
+  ```
+  Each annotation `*.xml` file has a name that corresponds to the name of the image file
+  (e.g. `frame_000001.xml` is the annotation for the `frame_000001.jpg` image).
+  Detailed structure specification of the `*.xml` file can be found
+  [here](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/devkit_doc.pdf).
+- supported shapes - Rectangles
+- additional comments: If you plan to use 'truncated' and 'difficult' attributes please add the corresponding
+  items to the CVAT label attributes:
+  `~checkbox=difficult:false ~checkbox=truncated:false`
+
+#### Pascal loader description
+-   uploaded file: a zip archive with following structure:
+    ```bash
+    taskname.zip
+    ├── frame_000001.xml
+    ├── frame_000002.xml
+    ├── frame_000003.xml
+    └── ...
+    ```
+    It should be possible to match the CVAT frame(imagename) and image filename from the annotation \*.xml
+    file (the tag filename, e.g. `<filename>2008_004457.jpg</filename>`). There are 2 options:
+    1. full match between image name and filename from annotation *.xml
+       file (in case of a task was created from images or archive of images).
+    1. match by frame number (if CVAT cannot match by name). File name should be in the following format `frame_%6d.jpg`.
+       It will be used when task was created from a video.
+
+-   supported shapes: Rectangles
+-   limitations: Support of Pascal VOC object detection format
+-   additional comments: the CVAT task should be created with the full label set that may be in the annotation files
+
+#### How to create a task from Pascal VOC dataset
+1.  Download the Pascal Voc dataset (Can be downloaded from the
+    [PASCAL VOC website](http://host.robots.ox.ac.uk/pascal/VOC/))
+1.  Create a CVAT task with the following labels:
+    ```bash
+    aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
+    ```
+    You can add `~checkbox=difficult:false ~checkbox=truncated:false` attributes for each label if you want to use them.
+
+    Select interesting image files
+    (See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
+    guide for details)
+1.  zip the corresponding annotation files
+1.  click `Upload annotation` button, choose `Pascal VOC ZIP 1.0`
+and select the *.zip file with annotations from previous step.
+It may take some time.
+
+### [YOLO](https://pjreddie.com/darknet/yolo/)
+#### Yolo dumper description
+- downloaded file: a zip archive with following structure:
+  ```bash
+  taskname.zip
+  ├── frame_000001.txt
+  ├── frame_000002.txt
+  ├── ...
+  └── obj.names
+  ```
+  Each annotation `*.txt` file has a name that corresponds to the name of the image file
+  (e.g. `frame_000001.txt` is the annotation for the `frame_000001.jpg` image).
+  Short description of  `*.txt` file structure: each line describes label and bounding box
+  in the following format `label_id cx cy w h`.
+  `obj.names` contains the ordered list of label names.
+- supported shapes - Rectangles
+
+#### Yolo loader description
+-   uploaded file: a zip archive with following structure:
+    ```bash
+    taskname.zip
+    ├── frame_000001.txt
+    ├── frame_000002.txt
+    ├── frame_000003.txt
+    ├── ...
+    └──obj.names
+    ```
+    It should be possible to match the CVAT frame(imagename) and annotation filename
+    There are 2 options:
+    1. full match between image name and name of annotation `*.txt` file
+       (in case of a task was created from images or archive of images).
+    1. match by frame number (if CVAT cannot match by name). File name should be in the following format `frame_%6d.jpg`.
+       It will be used when task was created from a video.
+
+-   supported shapes: Rectangles
+-   additional comments: the CVAT task should be created with the full label set that may be in the annotation files
+
+#### How to create a task from YOLO formatted dataset (from VOC for example)
+1. Follow the official [guide](https://pjreddie.com/darknet/yolo/)(see Training YOLO on VOC section)
+   and prepare the YOLO formatted annotation files.
+1. Zip train images
+   ```bash
+   zip images.zip -j -@ < train.txt
+   ```
+1. Create a CVAT task with the following labels:
+   ```bash
+   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
+   ```
+   Select images.zip as data. Most likely you should use `share`
+   functionality because size of images.zip is more than 500Mb.
+   See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
+   guide for details.
+1. Create `obj.names` with the following content:
+   ```bash
+   aeroplane
+   bicycle
+   bird
+   boat
+   bottle
+   bus
+   car
+   cat
+   chair
+   cow
+   diningtable
+   dog
+   horse
+   motorbike
+   person
+   pottedplant
+   sheep
+   sofa
+   train
+   tvmonitor
+   ```
+1. Zip all label files together (we need to add only label files that correspond to the train subset)
+   ```bash
+   cat train.txt | while read p; do echo ${p%/*/*}/labels/${${p##*/}%%.*}.txt; done | zip labels.zip -j -@ obj.names
+   ```
+1. Click `Upload annotation` button, choose `YOLO ZIP 1.0` and select the *.zip file with labels from previous step.
+   It may take some time.
+
+### [MS COCO Object Detection](http://cocodataset.org/#format-data)
+#### COCO dumper description
+- downloaded file: single unpacked `json`. Detailed description of the MS COCO format can be found [here](http://cocodataset.org/#format-data)
+- supported shapes - Polygons, Rectangles (interpreted as polygons)
+
+#### COCO loader description
+- uploaded file: single unpacked `*.json`.
+- supported shapes: Polygons (the `segmentation` must not be empty)
+- additional comments: the CVAT task should be created with the full label set that may be in the annotation files
+
+#### How to create a task from MS COCO dataset
+1.  Download the [MS COCO dataset](http://cocodataset.org/#download).
+    For example [2017 Val images](http://images.cocodataset.org/zips/val2017.zip)
+    and [2017 Train/Val annotations](http://images.cocodataset.org/annotations/annotations_trainval2017.zip).
+1.  Create a CVAT task with the following labels:
+      ```bash
+      person bicycle car motorcycle airplane bus train truck boat "traffic light" "fire hydrant" "stop sign" "parking meter" bench bird cat dog horse sheep cow elephant bear zebra giraffe backpack umbrella handbag tie suitcase frisbee skis snowboard "sports ball" kite "baseball bat" "baseball glove" skateboard surfboard "tennis racket" bottle "wine glass" cup fork knife spoon bowl banana apple sandwich orange broccoli carrot "hot dog" pizza donut cake chair couch "potted plant" bed "dining table" toilet tv laptop mouse remote keyboard "cell phone" microwave oven toaster sink refrigerator book clock vase scissors "teddy bear" "hair drier" toothbrush
+      ```
+
+      Select val2017.zip as data
+      (See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
+      guide for details)
+1.  unpack annotations_trainval2017.zip
+1.  click `Upload annotation` button,
+    choose `COCO JSON 1.0` and select `instances_val2017.json.json` annotation file. It may take some time.
+
+### [TFRecord](https://www.tensorflow.org/tutorials/load_data/tf_records)
+TFRecord is a very flexible format, but we try to correspond the format that used in
+[TF object detection](https://github.com/tensorflow/models/tree/master/research/object_detection)
+with minimal modifications.
+Used feature description:
+```python
+image_feature_description = {
+    'image/filename': tf.io.FixedLenFeature([], tf.string),
+    'image/source_id': tf.io.FixedLenFeature([], tf.string),
+    'image/height': tf.io.FixedLenFeature([], tf.int64),
+    'image/width': tf.io.FixedLenFeature([], tf.int64),
+    # Object boxes and classes.
+    'image/object/bbox/xmin': tf.io.VarLenFeature(tf.float32),
+    'image/object/bbox/xmax': tf.io.VarLenFeature(tf.float32),
+    'image/object/bbox/ymin': tf.io.VarLenFeature(tf.float32),
+    'image/object/bbox/ymax': tf.io.VarLenFeature(tf.float32),
+    'image/object/class/label': tf.io.VarLenFeature(tf.int64),
+    'image/object/class/text': tf.io.VarLenFeature(tf.string),
+}
+```
+#### TFRecord dumper description
+- downloaded file: a zip archive with following structure:
+  ```bash
+  taskname.zip
+  ├── task2.tfrecord
+  └── label_map.pbtxt
+  ```
+- supported shapes - Rectangles
+
+#### TFRecord loader description
+- uploaded file: a zip archive with following structure:
+  ```bash
+  taskname.zip
+  └── task2.tfrecord
+  ```
+- supported shapes: Rectangles
+- additional comments: the CVAT task should be created with the full label set that may be in the annotation files
+
+#### How to create a task from TFRecord dataset (from VOC2007 for example)
+1. Create label_map.pbtxt file with the following content:
+```js
+item {
+	id: 1
+	name: 'aeroplane'
+}
+item {
+	id: 2
+	name: 'bicycle'
+}
+item {
+	id: 3
+	name: 'bird'
+}
+item {
+	id: 4
+	name: 'boat'
+}
+item {
+	id: 5
+	name: 'bottle'
+}
+item {
+	id: 6
+	name: 'bus'
+}
+item {
+	id: 7
+	name: 'car'
+}
+item {
+	id: 8
+	name: 'cat'
+}
+item {
+	id: 9
+	name: 'chair'
+}
+item {
+	id: 10
+	name: 'cow'
+}
+item {
+	id: 11
+	name: 'diningtable'
+}
+item {
+	id: 12
+	name: 'dog'
+}
+item {
+	id: 13
+	name: 'horse'
+}
+item {
+	id: 14
+	name: 'motorbike'
+}
+item {
+	id: 15
+	name: 'person'
+}
+item {
+	id: 16
+	name: 'pottedplant'
+}
+item {
+	id: 17
+	name: 'sheep'
+}
+item {
+	id: 18
+	name: 'sofa'
+}
+item {
+	id: 19
+	name: 'train'
+}
+item {
+	id: 20
+	name: 'tvmonitor'
+}
+```
+1. Use [create_pascal_tf_record.py](https://github.com/tensorflow/models/blob/master/research/object_detection/dataset_tools/create_pascal_tf_record.py)
+to convert VOC2007 dataset to TFRecord format.
+As example:
+```bash
+python create_pascal_tf_record.py --data_dir <path to VOCdevkit> --set train --year VOC2007 --output_path pascal.tfrecord --label_map_path label_map.pbtxt
+```
+1. Zip train images
+   ```bash
+   cat <path to VOCdevkit>/VOC2007/ImageSets/Main/train.txt | while read p; do echo <path to VOCdevkit>/VOC2007/JPEGImages/${p}.jpg  ; done | zip images.zip -j -@
+   ```
+1. Create a CVAT task with the following labels:
+   ```bash
+   aeroplane bicycle bird boat bottle bus car cat chair cow diningtable dog horse motorbike person pottedplant sheep sofa train tvmonitor
+   ```
+   Select images.zip as data.
+   See [Creating an annotation task](cvat/apps/documentation/user_guide.md#creating-an-annotation-task)
+   guide for details.
+1. Zip pascal.tfrecord and label_map.pbtxt files together
+   ```bash
+   zip anno.zip -j <path to pascal.tfrecord> <path to label_map.pbtxt>
+   ```
+1. Click `Upload annotation` button, choose `TFRecord ZIP 1.0` and select the *.zip file
+   with labels from previous step. It may take some time.
+
+### PNG mask
+#### Mask dumper description
+- downloaded file: a zip archive with the following structure:
+  ```bash
+  taskname.zip
+  ├── frame_000001.png
+  ├── frame_000002.png
+  ├── frame_000003.png
+  ├── ...
+  └── colormap.txt
+  ```
+  Mask is a png image with several (RGB) channels where each pixel has own color which corresponds to a label.
+  Color generation correspond to the Pascal VOC color generation
+  [algorithm](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/devkit_doc.html#sec:voclabelcolormap).
+  (0, 0, 0) is used for background.
+  `colormap.txt` file contains the values of the used colors in RGB format.
+- supported shapes - Rectangles, Polygons
+
+#### Mask loader description
+Not supported

cvat/apps/annotation/annotation.py

@@ -118,6 +118,8 @@ def __init__(self, annotation_ir, db_task, scheme='', host='', create_callback=N
         self._host = host
         self._create_callback=create_callback
         self._MAX_ANNO_SIZE=30000
+        self._frame_info = {}
+        self._frame_mapping = {}
 
         db_labels = self._db_task.label_set.all().prefetch_related('attributespec_set').order_by('pk')
 
@@ -189,6 +191,10 @@ def _init_frame_info(self):
                 "height": db_image.height,
             } for db_image in self._db_task.image_set.all()}
 
+        self._frame_mapping = {
+            self._get_filename(info["path"]): frame for frame, info in self._frame_info.items()
+        }
+
     def _init_meta(self):
         db_segments = self._db_task.segment_set.all().prefetch_related('job_set')
         self._meta = OrderedDict([
@@ -424,3 +430,15 @@ def _len(self):
     @property
     def frame_info(self):
         return self._frame_info
+
+    @staticmethod
+    def _get_filename(path):
+        return os.path.splitext(os.path.basename(path))[0]
+
+    def match_frame(self, filename):
+        # try to match by filename
+        _filename = self._get_filename(filename)
+        if _filename in self._frame_mapping:
+            return self._frame_mapping[_filename]
+
+        raise Exception("Cannot match filename or determinate framenumber for {} filename".format(filename))