Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert most enum definitions from the functional style to the class style #290

Merged
merged 1 commit into from
Jun 11, 2021

Conversation

IRDonch
Copy link

@IRDonch IRDonch commented Jun 10, 2021

Summary

The class style has advantages in terms of IDE support. In particular, with Visual Studio Code, when the functional style is used:

  • The enumeration type is considered to be a variable rather than a class.
  • Autocomplete doesn't work for enum members.
  • "Go to definition" doesn't work for enum members (and indeed, with the functional style they don't really have proper definitions).

Using the class style solves these problems. I would also argue that the class style is slightly more readable due to not as much punctuation (although I can't deny that the repetition of = auto() is quite ugly).

The class style removes the need to repeat the enum name twice (in fact, there were a few cases in the codebase in which the two names didn't match).

I converted most of the enums automatically by using the following script:

#!/usr/bin/env python3

import regex
import sys

from pathlib import Path

RE_ENUM = regex.compile(
   r'''
   ^(?<Indent>[ \t]*) (?<EnumName>\w+) \s* = \s* Enum \s* \( \s* (?: '.+?' | ".+?") \s* ,
       \s* \[ \s* (?: '(?<MemberName>.+?)' | "(?<MemberName>.+?)" )
       ( \s* , \s* (?: '(?<MemberName>.+?)' | "(?<MemberName>.+?)" ) )* (?: \s* , )?
       \s* \] \s* \) [ \t]*\n
   ''',
   regex.VERBOSE | regex.MULTILINE)

def main():
    for path in sys.argv[1:]:
        path = Path(path)

        contents = path.read_text()

        def replace(match):
            indent = match.group('Indent')
            enum_name = match.group('EnumName')
            member_names = match.captures('MemberName')
            replacement = f'{indent}class {enum_name}(Enum):\n' \
                + ''.join(f'{indent}    {name} = auto()\n' for name in member_names)
            print(f'{match.group()!r} -> {replacement!r}')
            return replacement

        contents = RE_ENUM.sub(replace, contents)

        path.write_text(contents)

main()

The rest of them I converted manually.

The only enum that I kept in the functional style is MotLabel, which has members that are not valid Python identifiers (and probably shouldn't be an enum at all).

How to test

N/A (although I should note that I used flake8 --select=F821 to make sure all modules import auto if needed).

Checklist

  • I submit my changes into the develop branch
  • [ ] I have added description of my changes into CHANGELOG N/A - no user-visible changes
  • [ ] I have updated the documentation accordingly
  • [ ] I have added tests to cover my changes
  • [ ] I have linked related issues

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below)
# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

…style

The class style has advantages in terms of IDE support. In particular, with
Visual Studio Code, when the functional style is used:

* The enumeration type is considered to be a variable rather than a class.
* Autocomplete doesn't work for enum members.
* "Go to definition" doesn't work for enum members (and indeed, with the
  functional style they don't really have proper definitions).

Using the class style solves these problems. I would also argue that the
class style is slightly more readable due to not as much punctuation
(although I can't deny that the repetition of "= auto()" is quite ugly).

The class style removes the need to repeat the enum name twice (in fact,
there were a few cases in the codebase in which the two names didn't match).

I converted most of the enums automatically by using the following script:

```python

import regex
import sys

from pathlib import Path

RE_ENUM = regex.compile(
   r'''
   ^(?<Indent>[ \t]*) (?<EnumName>\w+) \s* = \s* Enum \s* \( \s* (?: '.+?' | ".+?") \s* ,
       \s* \[ \s* (?: '(?<MemberName>.+?)' | "(?<MemberName>.+?)" )
       ( \s* , \s* (?: '(?<MemberName>.+?)' | "(?<MemberName>.+?)" ) )* (?: \s* , )?
       \s* \] \s* \) [ \t]*\n
   ''',
   regex.VERBOSE | regex.MULTILINE)

def main():
    for path in sys.argv[1:]:
        path = Path(path)

        contents = path.read_text()

        def replace(match):
            indent = match.group('Indent')
            enum_name = match.group('EnumName')
            member_names = match.captures('MemberName')
            replacement = f'{indent}class {enum_name}(Enum):\n' \
                + ''.join(f'{indent}    {name} = auto()\n' for name in member_names)
            print(f'{match.group()!r} -> {replacement!r}')
            return replacement

        contents = RE_ENUM.sub(replace, contents)

        path.write_text(contents)

main()
```

The rest of them I converted manually.

The only enum that I kept in the functional style is `MotLabel`, which has
members that are not valid Python identifiers (and probably shouldn't be an
enum at all).
other = auto()
jumping = auto()
phoning = auto()
playinginstrument = auto()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, the need to assign values everywhere is the thing I dislike the most about class-style definitions. Also, I feel names next_idx() / auto_idx() for auto() would be more descriptive.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, it's a nuisance. I think the advantages still outweigh it, though.

@zhiltsov-max
Copy link
Contributor

The enumeration type is considered to be a variable rather than a class.

Having C++ background, I, personally, consider enums classes.

@IRDonch
Copy link
Author

IRDonch commented Jun 11, 2021

The enumeration type is considered to be a variable rather than a class.

Having C++ background, I, personally, consider enums classes.

That's great, because when you use class style they are considered classes. :-)

@zhiltsov-max zhiltsov-max merged commit 64b01b7 into openvinotoolkit:develop Jun 11, 2021
@IRDonch IRDonch deleted the enum-class branch July 7, 2021 14:46
zhiltsov-max pushed a commit that referenced this pull request Jul 14, 2021
* Rename 'openvino' plugin to 'openvino_plugin' (#205)

Co-authored-by: Jihyeon Yi <jihyeon.yi@intel.com>

* Make remap labels more accurate, allow explicit label deletion, add docs, update tests (#203)

* Kate/handling multiple attributes and speed up detection split (#207)

* better handling multi-attributes for classification_split

* handling multi-attributes better for detection

* bugfix in calculating required number of images for splitting 2 correct side effect of the changes for re-id split

* allow multiple subsets with arbitrary names

* rename _is_number to _is_float and improve it

* Fix voc to coco example (#209)

* Fix export filtering

* update example in readme

* Fix export filename for LabelMe format (#200)

* change export filename for LabelMe format

* Allow simple merge for datasets with no labels

* Add a more complex test on relative paths

* Support escaping in attributes

* update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* split unlabeled data into subsets for task-specific splitters (#211)

* split unlabeled data into subsets for classification, detection. for re-id, 'not-supported' subsets for this data

* Fix image ext on saving in cvat format (#214)

* fix image saving in cvat format

* update changelog

* Label "face" for bounding boxes in Wider Face (#215)

* add face label

* update changelog

* Adding "difficult", "truncated", "occluded" attributes when converting to Pascal VOC if they are not present (#216)

* remove check for 'difficult' attribute

* remove check for 'truncated' and 'occluded' attributes

* update changelog

* Ignore empty lines in YOLO annotations (#221)

* Ignore empty lines in yolo annotations

* Add type hints for image class, catch image opening errors in image.size

* update changelog

* Classification task in LFW dataset format (#222)

* add classification

* update changelog

* update documentation

* Add splitter for segmentation task  (#223)

* added segmentation_split

* updated changelog

* rename reidentification to reid

* Support for CIFAR-10/100 format (#225)

* add CIFAR dataset format

* add CIFAR to documentation

* update Changelog

* add validation item for instance segmentation (#227)

* add validation item for instance segmentation

* Add panoptic and stuff COCO format (#210)

* add coco stuff and panoptic formats

* update CHANGELOG

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* update detection splitter algorithm from # of samples to # of instances (#235)

* add documentation for validator (#233)

* add documentation for validator

* add validation item description (#237)

* Fix converter for Pascal VOC format (#239)

* User documentation for Pascal VOC format (#228)

* add user documentation for Pascal VOC format

* add integration tests

* update changelog

* Support for MNIST dataset format (#234)

* add mnist format

* add mnist csv format

* add mnist to documentation

* make formats docs folder, create COCO format documentation (#241)

* Make formats docs folder, move format docs

* Create COCO format documentation

* Fixes in CIFAR dataset format (#243)

* Add folder creation

* Update changelog

* Add user documentation file and integration tests for YOLO format (#246)

* add user documentation file for yolo

* add integraion tests

* update user manual

* update changelog

* Add Cityscapes format (#249)

* add cityscapes format

* add format docs

* update changelog

* Fix saving attribute in WiderFace extractor (#251)

* add fixes

* update changelog

* Fix spelling errors (#252)

* Configurable Threshold CLI support (#250)

* add validator cli

* add configurable validator threshold

* update changelog

* CI. Move to GitHub actions. (#263)

* Moving to GitHub Actions

* Sending a coverage report if python3.6 (#264)

* Rename workflows (#265)

* Rename workflows

* Update repo config and badge (#266)

* Update PR template

* Update build status badge

* Fix deprecation warnings (#270)

* Update RISE docs (#255)

* Update rise docs

* Update cli help

* Pytest related changes (#248)

* Tests moved to pytest. Updated CI. Updated requirements.

* Updated contribution guide

* Added annotations for tests

* Updated tests

* Added code style guide

* Fix CI (#272)

* Fix script call

* change script call to binary call

* Fix help program name, add mark_bug (#275)

* Fix prog name

* Add mark_bug test annotation

* Fix labelmap parameter in CamVid (#262)

* Fix labelmap parameter in camvid

* Release 0.1.9 (dev) (#276)

* Update version

* Update changelog

* Fix numpy conflict (#278)

* Add changelog stub (#279)

* tests/requirements.py: remove the test_wrapper functions (#285)

* Subformat importers for VOC and COCO (#281)

* Document find_sources

* Add VOC subformat importers

* Add coco subformat importers

* Fix LFW

* Reduce voc detect dataset cases

* Reorganize coco tests, add subformat tests

* Fix default subset handling in Dataset

* Fix getting subset

* Fix coco tests

* Fix voc tests

* Update changelog

* Add image zip format (#273)

* add tests

* add image_zip format

* update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add KITTI detection and segmentation formats (#282)

* Add KITTI detection and segmentation formats

* Remove unused import

* Add KITTI user manual

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Fix loading file and image processing in CIFAR (#284)

* Fix image layout and encoding problems

* Update Changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* CLI tests for convert command for VOC dataset (#286)

* Add tests for convert command

* Convert most enum definitions from the functional style to the class style (#290)

* yolo format documentation update (#295)

* add info about coordinates in yolo format doc

* Fix merged dataset item filtering (#258)

* Add tests

* Fix xpathfilter transform

* Update changelog

* Sms/pytest marking cityscapes and zip (#298)

* Updated pytest marking for cityscapes and imagezip.

* Introduce Validator plugin type (#299)

* Introduce Validator plugin type

* Fix validator definitions (#303)

* update changelog

* Fixes in validator definitions

* Update validator cli

* Make TF availability check optional (#305)

* Make tf availability check optional

* update changelog

* Update pylint (#304)

* Add import order check in pylint

* Fix some linter problems

* Remove warning suppression comments

* Add lazy loading for builtin plugins (#306)

* Refactor env code

* Load builtin plugins lazily

* update changelog

* Update transforms handling in Dataset (#297)

* Update builtin transforms

* Optimize dataset length computation when no source

* Add filter test

* Fix transforms affecting categories

* Optimize categories transforms

* Update filters

* fix imports

* Avoid using default docstrings in plugins

* Fix patch saving in VOC, add keep_empty export parameter

* Fix flush_changes

* Fix removed images and subsets in dataset patch

* Update changelog

* Update voc doc

* Skip item transform base class in plugins

* Readable COCO and datumaro format for CJK (#307)

* Do not force ASCII in COCO and Datumaro JSONs for readable CJK

* Add tests

* Use utf-8 encoding for writing

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Force utf-8 everywhere (#309)

* Fix in ImageNet_txt (#302)

* Add extensions for images to annotation file

* Remove image search in extractor

* Update changelog

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Reduce duplication of dependency information (#308)

* Move requirements from setup.py to requirements-base.txt

* Add whitespace error checking to GitHub Actions (#311)

* Fix whitespace errors

As detected with `git diff --check`.

* Add a job to check for whitespace errors

I called it "lint" so that other checks could be added to it later.

* Bump copyright years in changed files

* Add initial support for the Open Images dataset (#291)

* Support reading or Labels in Open Images (v4, v5, v6)

* Add tests for the Open Images extractor/importer

* Add Open Images documentation

* Update changelog

* Fix tensorboardX dependency (#318)

* Fixing remark-lint issues. Adding remark-linter check. (#321)

* Fix remark-lint issues.

* Align continuation lines with the first line.

Apply comments

* Added remark check

* Add an upper bound on the Pillow dependency to work around a regression in 8.3 (#323)

* open_images_user_manual.md: fix image description file URLs

I accidentally swapped the URLs for test and validation sets.

* Fix COCO Panoptic (#319)

* add test

* Fix integer overflow in bgr2index

* Fix pylint issues. Added pylint checking. (#322)

* Added pylint job for CI

* Rework pip install

* Fixed remaining pylint warnings

Co-authored-by: Andrey Zhavoronkov <andrey.zhavoronkov@intel.com>

* Open Images: add writing support (#315)

* open_images_user_manual.md: fix image description file URLs

* open_images_format: add conversion support

* open_images_format: add support for images in subdirectories

* open_images_format: add tests for writing support

* open_images_format: add documentation for the writing support

* Update the changelog entry for the Open Images support

* Add python bandit checks. (#316)

* Add bandit dependency

* Add bandit checks on CI

* Disable some warnings

Co-authored-by: Andrey Zhavoronkov <andrey.zhavoronkov@intel.com>
Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Remove Pylint unused-import warning suppressions (#326)

* Remove Pylint unused-import warning suppressions

* Add a job to check import formatting using isort (#333)

* Reformat all imports using isort

* Implement a workflow for checking import formatting based on isort

* Reformat the enabled checker list in .pylintrc (#335)

Put each code on its own line and add a comment with its symbolic name.
That makes the list more understandable and easier to edit.

* Merge all linting jobs into one workflow file (#331)

Doing it this way means that on GitHub's Checks page, all jobs are displayed
under one "Linter" category, instead of multiple indistinguishable "Linter"
categories with one job each.

Move the whitespace checking job into the Linter workflow as well, since
that's where it logically belongs.

I also took the opportunity to slightly rename the jobs in order to spell
the linter names correctly.

* Fix cuboids / 3d / M6 (#320)

* CVAT-3D Milestone-6: Added Supervisely Point Cloud and KITTI Raw 3D formats

* Added Cuboid3d annotations

* Added docs for new formats

Co-authored-by: cdp <cdp123>
Co-authored-by: Jayraj <jayrajsolanki96@gmail.com>
Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>

* Clean up .pylintrc (#340)

* Clean up the list of messages in .pylintrc

* Remove obsolete Pylint options

* .pylintrc: move the disable setting and its documentation together

* Remove the commented-out setting.

* Revert "Add an upper bound on the Pillow dependency to work around a regression in 8.3 (#323)" (#341)

The regression was fixed in 8.3.1.

This reverts commit 9a85616.

* Enable pylint checkers that find invalid escape sequences (#344)

Fix the issues that they found.

* Factor out the images.meta loading code from YoloExtractor (#343)

* Factor out the images.meta loading code from YoloExtractor

It looks like the same thing will be needed for Open Images, so I'm
moving it to a common module.

* Rework image.meta parsing code to use shell syntax

This allows comments and improves extensibility.

* Support for CIFAR-100 (#301)

* Add support for CIFAR-100

* Update Changelog

* Update user_manual.md

* Add notes about differences in formats

* Fix importing for VGG Face 2 (#345)

* correct asset according the original vgg_face2 dataset

* fix importing of the original dataset

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Dataset caching fixes (#351)

* Fix importing arbitrary file names in COCO subformats

* Optimize subset iteration in a simple scenario

* Fix subset iteration in dataset with transforms

* Cuboid 3D for Datumaro format (#349)

* Support cuboid_3d and point cloud in datumaro format

* Add cuboid_3d and point cloud tests in datumaro format

* Add image size type conversions

Co-authored-by: Maxim Zhiltsov <maxim.zhiltsov@intel.com>

* Add e2e tests for cuboids (#353)

* Add attr name check in kitti raw

* Add sly pcd e2e test

* Rename "object" attribute to "track_id" in sly point cloud

* Add kitti raw e2e test

* Update kitti raw example

* update changelog

* Release 0.1.10 (dev) (#354)

* Update changelog

* Add cifar security notice

* Update version

Co-authored-by: Emily Chun <emily.chun@intel.com>
Co-authored-by: Jihyeon Yi <jihyeon.yi@intel.com>
Co-authored-by: Kirill Sizov <kirill.sizov@intel.com>
Co-authored-by: Anastasia Yasakova <anastasia.yasakova@intel.com>
Co-authored-by: Harim Kang <harimx.kang@intel.com>
Co-authored-by: Zoya Maslova <zoya.maslova@intel.com>
Co-authored-by: Roman Donchenko <roman.donchenko@intel.com>
Co-authored-by: Seungyoon Woo <seung.woo@intel.com>
Co-authored-by: Dmitry Kruchinin <33020454+dvkruchinin@users.noreply.github.com>
Co-authored-by: Slawomir Strehlke <slawomir.strehlke@intel.com>
Co-authored-by: Jaesun Park <diligensloth@gmail.com>
Co-authored-by: Andrey Zhavoronkov <andrey.zhavoronkov@intel.com>
Co-authored-by: Jayraj <jayrajsolanki96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants