Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dataset examples and update some links in docs #6259

Merged
merged 1 commit into from
Jun 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions site/content/en/docs/manual/advanced/formats/format-camvid.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 10

# [CamVid](http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/camvid_dataset)

## CamVid export

Downloaded file: a zip archive of the following structure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ weight: 16
# [Cityscapes](https://www.cityscapes-dataset.com/login/)

- [Format specification](https://github.com/mcordts/cityscapesScripts#the-cityscapes-dataset)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/cityscapes_dataset)

- Supported annotations

Expand Down
40 changes: 32 additions & 8 deletions site/content/en/docs/manual/advanced/formats/format-coco.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,33 @@ weight: 5
# [MS COCO Object Detection](http://cocodataset.org/#format-data)

- [Format specification](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/coco_dataset)

## COCO export

Downloaded file: a zip archive with the structure described [here](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#import-coco-dataset)
Downloaded file: a zip archive with the structure described [here](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#import-coco-dataset)

```
archive.zip/
├── images/
│ ├── train/
│ │ ├── <image_name1.ext>
│ │ ├── <image_name2.ext>
│ │ └── ...
│ └── val/
│ ├── <image_name1.ext>
│ ├── <image_name2.ext>
│ └── ...
└── annotations/
├── <task>_<subset_name>.json
└── ...
```

If the dataset is exported from a Project, the subsets are named the same way as they are named
in the project. In other cases there will be a single `default` subset, containing all the data.
The `<task>` part corresponds to one of the COCO tasks: `instances`, `person_keypoints`,
`panoptic`, `image_info`, `labels`, `captions`, `stuff`. There can be several annotation
files in the archive.

- supported annotations: Polygons, Rectangles
- supported attributes:
Expand All @@ -21,7 +44,7 @@ Downloaded file: a zip archive with the structure described [here](https://openv
- `score` (number) - the annotation `score` field
- arbitrary attributes - will be stored in the `attributes` annotation section

Support for COCO tasks via Datumaro is described [here](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#export-to-other-formats)
Support for COCO tasks via Datumaro is described [here](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#export-to-other-formats)
For example, [support for COCO keypoints over Datumaro](https://github.com/openvinotoolkit/cvat/issues/2910#issuecomment-726077582):

1. Install [Datumaro](https://github.com/openvinotoolkit/datumaro)
Expand All @@ -35,19 +58,20 @@ keypoint lists (without the `visibility` COCO flag).

## COCO import

Uploaded file: a single unpacked `*.json` or a zip archive with the structure described
[here](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#import-coco-dataset)
Uploaded file: a single unpacked `*.json` or a zip archive with the structure described above or
[here](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#import-coco-dataset)
(without images).

- supported annotations: Polygons, Rectangles (if the `segmentation` field is empty)
- supported tasks: `instances`, `person_keypoints` (only segmentations will be imported), `panoptic`

# [MS COCO Keypoint Detection](https://cocodataset.org/#keypoints-2020)

- [Format specification](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/)
- [Format specification](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html)

## COCO export

Downloaded file: a zip archive with the structure described [here](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#import-coco-dataset)
Downloaded file: a zip archive with the structure described [here](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#import-coco-dataset)

- supported annotations: Skeletons
- supported attributes:
Expand All @@ -62,14 +86,14 @@ Downloaded file: a zip archive with the structure described [here](https://openv
## COCO import

Uploaded file: a single unpacked `*.json` or a zip archive with the structure described
[here](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#import-coco-dataset)
[here](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#import-coco-dataset)
(without images).

- supported annotations: Skeletons

## How to create a task from MS COCO dataset

1. Download the [MS COCO dataset](https://openvinotoolkit.github.io/datumaro/docs/formats/coco/#import-coco-dataset).
1. Download the [MS COCO dataset](https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/coco.html#import-coco-dataset).

For example `val images` and `instances` annotations

Expand Down
5 changes: 3 additions & 2 deletions site/content/en/docs/manual/advanced/formats/format-cvat.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ weight: 1

# CVAT

- [Format specification](/docs/manual/advanced/xml_format/)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/cvat_dataset)

This is the native CVAT annotation format. It supports all CVAT annotations
features, so it can be used to make data backups.

Expand All @@ -16,8 +19,6 @@ features, so it can be used to make data backups.

- attributes are supported

- [Format specification](/docs/manual/advanced/xml_format/)

## CVAT for images export

Downloaded file: a ZIP file of the following structure:
Expand Down
2 changes: 2 additions & 0 deletions site/content/en/docs/manual/advanced/formats/format-icdar.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 14

# [ICDAR13/15](https://rrc.cvc.uab.es/?ch=2)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/icdar_dataset)

## ICDAR13/15 export

Downloaded file: a zip archive of the following structure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 9

# [ImageNet](http://www.image-net.org)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/imagenet_dataset)

## ImageNet export

Downloaded file: a zip archive of the following structure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ weight: 17

- [Format specification for KITTI detection](https://s3.eu-central-1.amazonaws.com/avg-kitti/devkit_object.zip)
- [Format specification for KITTI segmentation](https://s3.eu-central-1.amazonaws.com/avg-kitti/devkit_semantics.zip)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/kitti_dataset)

- supported annotations:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 2

# [LabelMe](http://labelme.csail.mit.edu/Release3.0)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/labelme_dataset)

## LabelMe export

Downloaded file: a zip archive of the following structure:
Expand Down
3 changes: 2 additions & 1 deletion site/content/en/docs/manual/advanced/formats/format-lfw.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ weight: 17

# [LFW](http://vis-www.cs.umass.edu/lfw/)

- Format specification available [here](http://vis-www.cs.umass.edu/lfw/README.txt)
- [Format specification](http://vis-www.cs.umass.edu/lfw/README.txt)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/lfw_dataset)

- Supported annotations: tags, points.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 13

# [Market-1501](https://www.aitribune.com/dataset/2018051063)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/market1501_dataset)

## Market-1501 export

Downloaded file: a zip archive of the following structure:
Expand Down
2 changes: 2 additions & 0 deletions site/content/en/docs/manual/advanced/formats/format-mot.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 3

# [MOT sequence](https://arxiv.org/pdf/1906.04567.pdf)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/mot_dataset)

## MOT export

Downloaded file: a zip archive of the following structure:
Expand Down
2 changes: 2 additions & 0 deletions site/content/en/docs/manual/advanced/formats/format-mots.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 4

# [MOTS PNG](https://www.vision.rwth-aachen.de/page/mots)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/mots_dataset)

## MOTS PNG export

Downloaded file: a zip archive of the following structure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ weight: 15
# [Open Images](https://storage.googleapis.com/openimages/web/index.html)

- [Format specification](https://storage.googleapis.com/openimages/web/download.html)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/open_images_dataset)

- Supported annotations:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 8

# [TFRecord](https://www.tensorflow.org/tutorials/load_data/tfrecord)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/tf_detection_api_dataset)

TFRecord is a very flexible format, but we try to correspond the
format that used in
[TF object detection](https://github.com/tensorflow/models/tree/master/research/object_detection)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 12

# [VGGFace2](https://github.com/ox-vgg/vgg_face2)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/vgg_face2_dataset)

## VGGFace2 export

Downloaded file: a zip archive of the following structure:
Expand Down
7 changes: 4 additions & 3 deletions site/content/en/docs/manual/advanced/formats/format-voc.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,13 @@ weight: 6
# [Pascal VOC](http://host.robots.ox.ac.uk/pascal/VOC/)

- [Format specification](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/devkit_doc.pdf)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/voc_dataset)

- supported annotations:

- Rectangles (detection and layout tasks)
- Tags (action- and classification tasks)
- Polygons (segmentation task)
- Rectangles (`detection` and `layout` tasks)
- Tags (`action-` and `classification` tasks)
- Polygons (`segmentation` task)

- supported attributes:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ weight: 9

# [WIDER Face](http://shuoyang1213.me/WIDERFACE/)

- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/widerface_dataset)

## WIDER Face export

Downloaded file: a zip archive of the following structure:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ weight: 7
# [YOLO](https://pjreddie.com/darknet/yolo/)

- [Format specification](https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects)
- [Dataset examples](https://github.com/cvat-ai/datumaro/tree/v0.3/tests/assets/yolo_dataset)
- supported annotations: Rectangles

## YOLO export
Expand Down