Co-authored-by: Jihyeon Yi <jihyeon.yi@intel.com>

…ocs, update tests (#203)

* better handling multi-attributes for classification_split

* handling multi-attributes better for detection

* bugfix in calculating required number of images for splitting 2 correct side effect of the changes for re-id split

* allow multiple subsets with arbitrary names

* rename _is_number to _is_float and improve it

* Fix export filtering

* update example in readme

* change export filename for LabelMe format

* Allow simple merge for datasets with no labels

* Add a more complex test on relative paths

* Support escaping in attributes

* update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* split unlabeled data into subsets for classification, detection. for re-id, 'not-supported' subsets for this data

* fix image saving in cvat format

* update changelog

* add face label

* update changelog

…g to Pascal VOC if they are not present (#216)

* remove check for 'difficult' attribute

* remove check for 'truncated' and 'occluded' attributes

* update changelog

* Ignore empty lines in yolo annotations

* Add type hints for image class, catch image opening errors in image.size

* update changelog

* add classification

* update changelog

* update documentation

* added segmentation_split

* updated changelog

* rename reidentification to reid

* add CIFAR dataset format

* add CIFAR to documentation

* update Changelog

* add validation item for instance segmentation

* add coco stuff and panoptic formats

* update CHANGELOG

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

…es (#235)

* add documentation for validator

* add user documentation for Pascal VOC format

* add integration tests

* update changelog

* add mnist format

* add mnist csv format

* add mnist to documentation

* Make formats docs folder, move format docs

* Create COCO format documentation

* Add folder creation

* Update changelog

* add user documentation file for yolo

* add integraion tests

* update user manual

* update changelog

* add cityscapes format

* add format docs

* update changelog

* add fixes

* update changelog

* add validator cli

* add configurable validator threshold

* update changelog

* Moving to GitHub Actions

* Rename workflows

* Update PR template

* Update build status badge

* Update rise docs

* Update cli help

* Tests moved to pytest. Updated CI. Updated requirements.

* Updated contribution guide

* Added annotations for tests

* Updated tests

* Added code style guide

* Fix script call

* change script call to binary call

* Fix prog name

* Add mark_bug test annotation

* Fix labelmap parameter in camvid

* Update version

* Update changelog

* Document find_sources

* Add VOC subformat importers

* Add coco subformat importers

* Fix LFW

* Reduce voc detect dataset cases

* Reorganize coco tests, add subformat tests

* Fix default subset handling in Dataset

* Fix getting subset

* Fix coco tests

* Fix voc tests

* Update changelog

* add tests

* add image_zip format

* update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add KITTI detection and segmentation formats

* Remove unused import

* Add KITTI user manual

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Fix image layout and encoding problems

* Update Changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add tests for convert command

…style (#290)

* add info about coordinates in yolo format doc

* Add tests

* Fix xpathfilter transform

* Update changelog

* Updated pytest marking for cityscapes and imagezip.

* Introduce Validator plugin type

* update changelog

* Fixes in validator definitions

* Update validator cli

* Make tf availability check optional

* update changelog

* Add import order check in pylint

* Fix some linter problems

* Remove warning suppression comments

* Refactor env code

* Load builtin plugins lazily

* update changelog

* Update builtin transforms

* Optimize dataset length computation when no source

* Add filter test

* Fix transforms affecting categories

* Optimize categories transforms

* Update filters

* fix imports

* Avoid using default docstrings in plugins

* Fix patch saving in VOC, add keep_empty export parameter

* Fix flush_changes

* Fix removed images and subsets in dataset patch

* Update changelog

* Update voc doc

* Skip item transform base class in plugins

* Do not force ASCII in COCO and Datumaro JSONs for readable CJK

* Add tests

* Use utf-8 encoding for writing

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add extensions for images to annotation file

* Remove image search in extractor

* Update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Move requirements from setup.py to requirements-base.txt

* Fix whitespace errors

As detected with `git diff --check`.

* Add a job to check for whitespace errors

I called it "lint" so that other checks could be added to it later.

* Bump copyright years in changed files

* Support reading or Labels in Open Images (v4, v5, v6)

* Add tests for the Open Images extractor/importer

* Add Open Images documentation

* Update changelog

* Fix remark-lint issues.

* Align continuation lines with the first line.

Apply comments

* Added remark check

…on in 8.3 (#323)

I accidentally swapped the URLs for test and validation sets.

open_images_user_manual.md: fix image description file URLs

* add test

* Fix integer overflow in bgr2index

* Added pylint job for CI

* Rework pip install

* Fixed remaining pylint warnings

Co-authored-by: Andrey Zhavoronkov <andrey.zhavoronkov@intel.com>

* open_images_user_manual.md: fix image description file URLs

* open_images_format: add conversion support

* open_images_format: add support for images in subdirectories

* open_images_format: add tests for writing support

* open_images_format: add documentation for the writing support

* Update the changelog entry for the Open Images support

* Add bandit dependency

* Add bandit checks on CI

* Disable some warnings

Co-authored-by: Andrey Zhavoronkov <andrey.zhavoronkov@intel.com>
Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Remove Pylint unused-import warning suppressions

* Reformat all imports using isort

* Implement a workflow for checking import formatting based on isort

Put each code on its own line and add a comment with its symbolic name.
That makes the list more understandable and easier to edit.

Doing it this way means that on GitHub's Checks page, all jobs are displayed
under one "Linter" category, instead of multiple indistinguishable "Linter"
categories with one job each.

Move the whitespace checking job into the Linter workflow as well, since
that's where it logically belongs.

I also took the opportunity to slightly rename the jobs in order to spell
the linter names correctly.

* CVAT-3D Milestone-6: Added Supervisely Point Cloud and KITTI Raw 3D formats

* Added Cuboid3d annotations

* Added docs for new formats

Co-authored-by: cdp <cdp123>
Co-authored-by: Jayraj <jayrajsolanki96@gmail.com>
Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>

* Clean up the list of messages in .pylintrc

* Remove obsolete Pylint options

* .pylintrc: move the disable setting and its documentation together

* Remove the commented-out setting.

…regression in 8.3 (#323)" (#341)

The regression was fixed in 8.3.1.

This reverts commit 9a85616.

Fix the issues that they found.

* Factor out the images.meta loading code from YoloExtractor

It looks like the same thing will be needed for Open Images, so I'm
moving it to a common module.

* Rework image.meta parsing code to use shell syntax

This allows comments and improves extensibility.

* Add support for CIFAR-100

* Update Changelog

* Update user_manual.md

* Add notes about differences in formats

* correct asset according the original vgg_face2 dataset

* fix importing of the original dataset

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Fix importing arbitrary file names in COCO subformats

* Optimize subset iteration in a simple scenario

* Fix subset iteration in dataset with transforms

* Support cuboid_3d and point cloud in datumaro format

* Add cuboid_3d and point cloud tests in datumaro format

* Add image size type conversions

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add attr name check in kitti raw

* Add sly pcd e2e test

* Rename "object" attribute to "track_id" in sly point cloud

* Add kitti raw e2e test

* Update kitti raw example

* update changelog

* Update changelog

* Add cifar security notice

* Update version