pyproject.toml #2981

AlexanderDokuchaev · 2024-09-23T08:58:58Z

Changes

Use pyproject.toml instead of setup.py

Reason for changes

PEP 517/518
Tools like Black, Isort, Ruff, and Mypy can automatically use settings defined in a pyproject.toml file, allowing these tools to apply configurations directly from the repository without requiring additional configuration in text editor or extra arguments in terminal.

Tests

nightly/job/install_onnx/543/
nightly/job/install_ov/567/
nightly/job/install_pt_cpu/566/
nightly/job/ubuntu20_install_tf/672/

AlexanderDokuchaev · 2024-09-24T08:59:19Z

whl package

setup.py:

unzip -l dist/*whl | tail -n7
    11357  2024-09-24 10:37   nncf-2.14.0.dev0+unknown.version.dist-info/LICENSE
    10220  2024-09-24 10:37   nncf-2.14.0.dev0+unknown.version.dist-info/METADATA
       91  2024-09-24 10:37   nncf-2.14.0.dev0+unknown.version.dist-info/WHEEL
        5  2024-09-24 10:37   nncf-2.14.0.dev0+unknown.version.dist-info/top_level.txt
    60943  2024-09-24 10:37   nncf-2.14.0.dev0+unknown.version.dist-info/RECORD
---------                     -------
  4583702                     615 files

pyproject.toml:

unzip -l dist/*whl | tail -n 7
    11357  2024-09-24 13:11   nncf-2.14.0.dev0+unknown.version.dist-info/LICENSE
    10279  2024-09-24 13:11   nncf-2.14.0.dev0+unknown.version.dist-info/METADATA
       91  2024-09-24 13:11   nncf-2.14.0.dev0+unknown.version.dist-info/WHEEL
        5  2024-09-24 13:11   nncf-2.14.0.dev0+unknown.version.dist-info/top_level.txt
    60943  2024-09-24 13:11   nncf-2.14.0.dev0+unknown.version.dist-info/RECORD
---------                     -------
  4581438                     615 files

pip show

setup.py

pip show nncf
Name: nncf
Version: 2.14.0.dev0+unknown.version
Summary: Neural Networks Compression Framework
Home-page: https://github.com/openvinotoolkit/nncf
Author: Intel
Author-email: alexander.kozlov@intel.com
License: Apache-2.0
Location: /home/adokucha/work/venv/lib/python3.10/site-packages
Requires: jsonschema, jstyleson, natsort, networkx, ninja, numpy, openvino-telemetry, packaging, pandas, psutil, pydot, pymoo, rich, scikit-learn, scipy, tabulate, tqdm
Required-by:

pyproject.toml

Name: nncf
Version: 2.14.0.dev0+unknown.version
Summary: Neural Networks Compression Framework
Home-page: https://github.com/openvinotoolkit/nncf
Author: Intel
Author-email: alexander.kozlov@intel.com
License: Apache-2.0
Location: /home/adokucha/work/venv/lib/python3.10/site-packages
Requires: jsonschema, jstyleson, natsort, networkx, ninja, numpy, openvino-telemetry, packaging, pandas, psutil, pydot, pymoo, rich, scikit-learn, scipy, tabulate, tqdm
Required-by:

AlexanderDokuchaev · 2024-09-24T09:15:01Z

.github/workflows/python-publish.yml

 run: |
- python -m build -C--global-option=--release


There is no way to pass global option to control format of version, that used env variable instead

It was more convenient to pass this option into CLI.

AlexanderDokuchaev · 2024-09-24T09:16:19Z

_version_helper.py

+ repo_root = os.path.dirname(os.path.realpath(__file__))
+ run = subprocess.run(["git", "diff-index", "--quiet", "HEAD"], cwd=repo_root) # nosec
+ if run.returncode == 1:
+ dev_version_id += "dirty"


Detect editable mode pip install -e is not possible to check here, instead of it add dirty suffix if code was modified.

The previous editable suffix was more correct.

AlexanderDokuchaev · 2024-09-24T11:55:46Z

tests/cross_fw/shared/helpers.py

 elif package_type == "build_w":
- run_cmd_line = f"{python_executable_with_venv} -m build -n -w"


--no-isolation, -n disable building the project in an isolated virtual environment. Build dependencies must be installed separately when this option is used

alexsu52

LGTM

KodiaqQ · 2024-10-01T12:40:54Z

.github/workflows/python-publish.yml

@@ -39,8 +39,9 @@ jobs:
 env:
 TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
 TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
+ NNCF_RELEASE_BUILD: "1"


These changes need to be placed in our release procedure documentation.

KodiaqQ · 2024-10-01T12:41:35Z

.github/workflows/python-publish.yml

 run: |
- python -m build -C--global-option=--release


It was more convenient to pass this option into CLI.

KodiaqQ · 2024-10-01T12:44:08Z

_version_helper.py

From my perspective, it is not correct to place helper-like files named with _ at the root of the project.
@alexsu52, @AlexKoff88, could you please comment on this?

KodiaqQ · 2024-10-01T12:46:26Z

_version_helper.py

+# NOTE 1: This module generates the dynamic version for the package during the build process.
+# It provides the version attribute for setuptools as specified in pyproject.toml:
+# [tool.setuptools.dynamic]
+# version = { attr = "_version_helper.version" }


This is not correct NOTE and behaviour in general, since trying to build without the NNCF_RELEASE_BUILD variable leads to incorrect file naming:

Need to be fixed despite the fact that this issue exists on the develop.

KodiaqQ · 2024-10-01T12:49:22Z

_version_helper.py

+ repo_root = os.path.dirname(os.path.realpath(__file__))
+ run = subprocess.run(["git", "diff-index", "--quiet", "HEAD"], cwd=repo_root) # nosec
+ if run.returncode == 1:
+ dev_version_id += "dirty"


The previous editable suffix was more correct.

KodiaqQ · 2024-10-01T12:58:49Z

pyproject.toml

@@ -0,0 +1,150 @@
+[build-system]
+requires = ["setuptools>=61.0"]


Why the recommended setuptools version was bupmed?

AlexanderDokuchaev added 2 commits September 23, 2024 00:42

pyproject.toml instead of setup.py

96de78c

tools

7488abe

github-actions bot added dependencies Any changes in any dependencies (new dep or its version) should be produced via Change Request on PM NNCF TF Pull requests that updates NNCF TensorFlow NNCF PT Pull requests that updates NNCF PyTorch experimental labels Sep 23, 2024

AlexanderDokuchaev added 2 commits September 23, 2024 13:46

mypy

d0a95d9

namespaces = false

4a94095

docs/PyPiPublishing.md

b8b4c3c

AlexanderDokuchaev marked this pull request as ready for review September 25, 2024 08:29

AlexanderDokuchaev requested a review from a team as a code owner September 25, 2024 08:29

AlexanderDokuchaev requested review from alexsu52 and KodiaqQ September 25, 2024 08:29

AlexanderDokuchaev commented Sep 25, 2024

View reviewed changes

alexsu52 approved these changes Sep 25, 2024

View reviewed changes

KodiaqQ requested changes Oct 1, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pyproject.toml #2981

pyproject.toml #2981

AlexanderDokuchaev commented Sep 23, 2024 •

edited

Loading

AlexanderDokuchaev commented Sep 24, 2024 •

edited

Loading

AlexanderDokuchaev Sep 24, 2024

KodiaqQ Oct 1, 2024

AlexanderDokuchaev Sep 24, 2024

KodiaqQ Oct 1, 2024

AlexanderDokuchaev Sep 24, 2024

alexsu52 left a comment

KodiaqQ Oct 1, 2024

KodiaqQ Oct 1, 2024

KodiaqQ Oct 1, 2024

KodiaqQ Oct 1, 2024

KodiaqQ Oct 1, 2024

KodiaqQ Oct 1, 2024

		elif package_type == "build_w":
		run_cmd_line = f"{python_executable_with_venv} -m build -n -w"

		@@ -0,0 +1,150 @@
		[build-system]
		requires = ["setuptools>=61.0"]

pyproject.toml #2981

Are you sure you want to change the base?

pyproject.toml #2981

Conversation

AlexanderDokuchaev commented Sep 23, 2024 • edited Loading

Changes

Reason for changes

Tests

AlexanderDokuchaev commented Sep 24, 2024 • edited Loading

whl package

pip show

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexsu52 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexanderDokuchaev commented Sep 23, 2024 •

edited

Loading

AlexanderDokuchaev commented Sep 24, 2024 •

edited

Loading