docs: add SemanticMask and InstanceMask example in quick_start/examples

docs/code/VOC2012Segmentation.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+#
+# Copyright 2021 Graviti. Licensed under MIT License.
+#
+
+# pylint: disable=wrong-import-position
+# pylint: disable=wrong-import-order
+# pylint: disable=not-callable
+# pylint: disable=ungrouped-imports
+# pylint: disable=import-error
+# pylint: disable=pointless-string-statement
+# pylint: disable=invalid-name
+# pylint: disable=unused-import
+# flake8: noqa: F401
+
+"""This file includes the python code of VOC2012Segmentation.rst."""
+
+"""Authorize a Client Instance"""
+from tensorbay import GAS
+
+ACCESS_KEY = "Accesskey-*****"
+gas = GAS(ACCESS_KEY)
+""""""
+
+"""Create Dataset"""
+gas.create_dataset("VOC2012Segmentation")
+""""""
+
+"""Organize Dataset / regular import"""
+from tensorbay.dataset import Data, Dataset
+from tensorbay.label import InstanceMask, SemanticMask
+
+""""""
+
+"""Organize dataset / import dataloader"""
+from tensorbay.opendataset import VOC2012Segmentation
+
+dataset = VOC2012Segmentation("path/to/dataset/directory")
+""""""
+
+"""Upload Dataset"""
+dataset_client = gas.upload_dataset(dataset, jobs=8)
+dataset_client.commit("initial commit")
+""""""
+
+"""Read Dataset / get dataset"""
+dataset = Dataset("VOC2012Segmentation", gas)
+""""""
+
+"""Read Dataset / get segment"""
+segment = dataset[0]
+""""""
+
+"""Read Dataset / get data"""
+data = segment[0]
+""""""
+
+"""Read Dataset / get label"""
+label_semantic_mask = data.label.semantic_mask
+semantic_all_attributes = label_semantic_mask.all_attributes
+label_instance_mask = data.label.instance_mask
+instance_all_attributes = label_instance_mask.all_attributes
+""""""
+
+"""Delete Dataset"""
+gas.delete_dataset("VOC2012Segmentation")
+""""""

docs/source/examples/VOC2012Segmentation.rst

@@ -0,0 +1,192 @@
+######################
+ VOC2012 Segmentation
+######################
+
+This topic describes how to manage the `VOC2012 Segmentation dataset`_,
+which is a dataset with :ref:`reference/label_format/SemanticMask:SemanticMask`
+and :ref:`reference/label_format/InstanceMask:InstanceMask` labels
+(:numref:`Fig. %s <example-semantic-mask>` and :numref:`Fig. %s <example-instance-mask>`).
+
+.. _VOC2012 Segmentation dataset: https://gas.graviti.cn/dataset/hello-dataset/VOC2012Segmentation
+
+.. _example-semantic-mask:
+
+.. figure:: ../images/example-semanticmask.png
+   :scale: 150 %
+   :align: center
+
+   The preview of a semantic mask from "VOC2012 Segmentation".
+
+.. _example-instance-mask:
+
+.. figure:: ../images/example-instancemask.png
+   :scale: 150 %
+   :align: center
+
+   The preview of a instance mask from "VOC2012 Segmentation".
+
+
+*****************************
+ Authorize a Client Instance
+*****************************
+
+An :ref:`reference/glossary:accesskey` is needed to authenticate identity when using TensorBay.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Authorize a Client Instance"""
+   :end-before: """"""
+
+****************
+ Create Dataset
+****************
+
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Create Dataset"""
+   :end-before: """"""
+
+******************
+ Organize Dataset
+******************
+
+It takes the following steps to organize "VOC2012 Segmentation" dataset by the :class:`~tensorbay.dataset.dataset.Dataset` instance.
+
+Step 1: Write the Catalog
+=========================
+
+A :ref:`Catalog <reference/dataset_structure:catalog>` contains all label information of one dataset,
+which is typically stored in a json file.
+
+.. literalinclude:: ../../../tensorbay/opendataset/VOC2012Segmentation/catalog.json
+   :language: json
+   :name: voc2012segmentation-catalog
+   :linenos:
+
+The annotation types for "VOC2012 Segmentation" are :ref:`reference/label_format/SemanticMask:SemanticMask`
+and :ref:`reference/label_format/InstanceMask:InstanceMask`, and there are 22
+:ref:`reference/label_format/CommonLabelProperties:Category` types for :ref:`reference/label_format/SemanticMask:SemanticMask`. There are 2
+:ref:`reference/label_format/CommonLabelProperties:Category` types for :ref:`reference/label_format/InstanceMask:InstanceMask`, category 0 represents the
+background, category 255 represents the border of instances, and other different categories represent different instances.
+
+.. important::
+
+   See :ref:`catalog table <reference/dataset_structure:catalog>` for more catalogs with different label types.
+
+Step 2: Write the Dataloader
+============================
+
+A :ref:`reference/glossary:dataloader` is needed to organize the dataset into
+a :class:`~tensorbay.dataset.dataset.Dataset` instance.
+
+.. literalinclude:: ../../../tensorbay/opendataset/VOC2012Segmentation/loader.py
+   :language: python
+   :name: voc2012segmentation-dataloader
+   :linenos:
+
+See :ref:`SemanticMask annotation <reference/label_format/SemanticMask:SemanticMask>`
+and :ref:`InstanceMask annotation <reference/label_format/InstanceMask:InstanceMask>` for more details.
+
+.. note::
+
+   Since the :ref:`VOC2012 Segmentation dataloader <voc2012segmentation-dataloader>` above is already included in
+   TensorBay, so it uses relative import.
+   However, the regular import should be used when writing a new dataloader.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Organize Dataset / regular import"""
+   :end-at: from tensorbay.label import InstanceMask, SemanticMask
+
+There are already a number of dataloaders in TensorBay SDK provided by the community.
+Thus, instead of writing, importing an available dataloader is also feasible.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Organize dataset / import dataloader"""
+   :end-before: """"""
+
+.. note::
+
+   Note that catalogs are automatically loaded in available dataloaders, users do not have to write them again.
+
+.. important::
+
+   See :ref:`dataloader table <reference/glossary:dataloader>` for dataloaders with different label types.
+
+*******************
+ Visualize Dataset
+*******************
+
+Optionally, the organized dataset can be visualized by **Pharos**, which is a TensorBay SDK plug-in.
+This step can help users to check whether the dataset is correctly organized.
+Please see :ref:`features/visualization:Visualization` for more details.
+
+****************
+ Upload Dataset
+****************
+
+The organized "VOC2012 Segmentation" dataset can be uploaded to tensorBay for sharing, reuse, etc.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Upload Dataset"""
+   :end-before: """"""
+
+Similar with Git, the commit step after uploading can record changes to the dataset as a version.
+If needed, do the modifications and commit again.
+Please see :ref:`features/version_control:Version Control` for more details.
+
+**************
+ Read Dataset
+**************
+
+Now "VOC2012 Segmentation" dataset can be read from TensorBay.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Read Dataset / get dataset"""
+   :end-before: """"""
+
+In :ref:`reference/dataset_structure:Dataset` "VOC2012 Segmentation", there are two
+:ref:`segments <reference/dataset_structure:Segment>`: ``train`` and ``val``.
+Get a segment by passing the required segment name or the index.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Read Dataset / get segment"""
+   :end-before: """"""
+
+In the ``train`` :ref:`reference/dataset_structure:Segment`,
+there is a sequence of :ref:`reference/dataset_structure:Data`,
+which can be obtained by index.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Read Dataset / get data"""
+   :end-before: """"""
+
+In each :ref:`reference/dataset_structure:Data`,
+there are one :ref:`reference/label_format/SemanticMask:SemanticMask` annotation and one :ref:`reference/label_format/InstanceMask:InstanceMask` annotation.
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Read Dataset / get label"""
+   :end-before: """"""
+
+There are two label types in "VOC2012 Segmentation" dataset, which are ``semantic_mask`` and ``instance_mask``.
+The information stored in :ref:`reference/label_format/SemanticMask:SemanticMask.all_attributes` is
+attributes for every category in "categories" list of "SEMANTIC_MASK" of :ref:`catalog.json <voc2012segmentation-catalog>`.
+The information stored in :ref:`reference/label_format/InstanceMask:InstanceMask.all_attributes`
+is attributes for every instance.
+See :ref:`reference/label_format/SemanticMask:SemanticMask` and :ref:`reference/label_format/InstanceMask:InstanceMask` label formats for more details.
+
+****************
+ Delete Dataset
+****************
+
+.. literalinclude:: ../../../docs/code/VOC2012Segmentation.py
+   :language: python
+   :start-after: """Delete Dataset"""
+   :end-before: """"""

docs/source/quick_start/examples.rst

@@ -10,9 +10,15 @@ The following table lists a series of examples to help developers to use TensorB
    :align: center
    :widths: auto
 
+<<<<<<< HEAD
    =======================================================  ==========================================================================
    Examples                                                 Description
    =======================================================  ==========================================================================
+=======
+   ======================================================== ===========================================================
+   Examples                                                 Description
+   ======================================================== ===========================================================
+>>>>>>> docs: add SemanticMask and InstanceMask example in quick_start/examples
    :ref:`examples/DogsVsCats:Dogs vs Cats`                  | Topic: Dataset Management
                                                             | Data Type: Image
                                                             | Label Type: :ref:`reference/label_format/Classification:Classification`
@@ -30,11 +36,23 @@ The following table lists a series of examples to help developers to use TensorB
                                                             | Label Type: :ref:`reference/label_format/Keypoints2D:Keypoints2D`
    :ref:`examples/THCHS30:THCHS-30`                         | Topic: Dataset Management
                                                             | Data Type: Audio
+<<<<<<< HEAD
                                                             | Label Type: :ref:`reference/label_format/Sentence:Sentence`
    :ref:`examples/update_dataset:Update Dataset`            | Topic: Update Dataset
    :ref:`examples/move_and_copy:Move And Copy`              | Topic: Move And Copy
    :ref:`examples/merge_datasets:Merge Datasets`             | Topic: Merge Datasets
    =======================================================  ==========================================================================
+=======
+                                                            | Label Type: :ref:`reference/label_format:Sentence`
+   :ref:`examples/VOC2012Segmentation:VOC2012 Segmentation` | Topic: Dataset Management
+                                                            | Data Type: Image
+                                                            | Label Types: :ref:`reference/label_format:SemanticMask`,
+                                                              :ref:`reference/label_format:InstanceMask`
+   :ref:`examples/update_dataset:Update Dataset`            | Topic: Update Dataset
+   :ref:`examples/move_and_copy:Move And Copy`              | Topic: Move And Copy
+   :ref:`examples/merge_datasets:Merge Datasets`             | Topic: Merge Datasets
+   ======================================================== ===========================================================
+>>>>>>> docs: add SemanticMask and InstanceMask example in quick_start/examples
 
 .. toctree::
    :hidden:
@@ -46,6 +64,7 @@ The following table lists a series of examples to help developers to use TensorB
    ../examples/NeolixOD
    ../examples/THCHS30
    ../examples/Newsgroups20
+   ../examples/VOC2012Segmentation
    ../examples/update_dataset
    ../examples/move_and_copy
    ../examples/merge_datasets