[PDDF] Build and install Python 3 package #6286

jleveque · 2020-12-23T22:47:30Z

- Why I did it

As part of migrating the SONiC codebase from Python 2 to Python 3

- How I did it

Make PDDF code compliant with both Python 2 and Python 3
Align code with PEP8 standards using autopep8
Build and install both Python 2 and Python 3 PDDF packages
Once all vendors build and install Python 3 versions of their sonic-platform package, we can simply stop building and installing the Python 2 version of the package.

- How to verify it

- Which release branch to backport (provide reason below if selected)

201811
201911
202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

jleveque · 2020-12-23T22:48:38Z

@fk410167: Can you please review and test this PR?

FuzailBrcm · 2020-12-24T16:00:00Z

@jleveque
There is a build failure and some more changes are required. I resolved the build breakage locally. Build is in progress. Once done, I will test PDDF and update you.

FuzailBrcm · 2020-12-28T06:11:13Z

@jleveque

I tested the build and found these changes are required for successful build. Please add them.

git diff platform/pddf/platform-api-pddf-base.dep

diff --git a/platform/pddf/platform-api-pddf-base.dep b/platform/pddf/platform-api-pddf-base.dep
index 9184925b1..0adf0ff2d 100644
--- a/platform/pddf/platform-api-pddf-base.dep
+++ b/platform/pddf/platform-api-pddf-base.dep
@@ -7,3 +7,7 @@ DEP_FILES += $(shell git ls-files $(MPATH))
$(PDDF_PLATFORM_API_BASE_PY2)_CACHE_MODE := GIT_CONTENT_SHA
$(PDDF_PLATFORM_API_BASE_PY2)_DEP_FLAGS := $(SONIC_COMMON_FLAGS_LIST)
$(PDDF_PLATFORM_API_BASE_PY2)_DEP_FILES := $(DEP_FILES)
+
+$(PDDF_PLATFORM_API_BASE_PY3)_CACHE_MODE := GIT_CONTENT_SHA
+$(PDDF_PLATFORM_API_BASE_PY3)_DEP_FLAGS := $(SONIC_COMMON_FLAGS_LIST)
+$(PDDF_PLATFORM_API_BASE_PY3)_DEP_FILES := $(DEP_FILES)

git diff platform/pddf/platform-api-pddf-base.mk

diff --git a/platform/pddf/platform-api-pddf-base.mk b/platform/pddf/platform-api-pddf-base.mk
index 80478d4f3..6938199dd 100644
--- a/platform/pddf/platform-api-pddf-base.mk
+++ b/platform/pddf/platform-api-pddf-base.mk
@@ -13,3 +13,14 @@ SONIC_PYTHON_WHEELS += $(PDDF_PLATFORM_API_BASE_PY2)

export pddf_platform_api_base_py2_wheel_path="$(addprefix $(PYTHON_WHEELS_PATH)/,$(PDDF_PLATFORM_API_BASE_PY2))"
export PDDF_PLATFORM_API_BASE_PY2
+
+PDDF_PLATFORM_API_BASE_PY3 = sonic_platform_pddf_common-$(PDDF_PLATFORM_API_BASE_VERSION)-py3-none-any.whl
+$(PDDF_PLATFORM_API_BASE_PY3)_SRC_PATH = $(PLATFORM_PDDF_PATH)/platform-api-pddf-base
+$(PDDF_PLATFORM_API_BASE_PY3)_PYTHON_VERSION = 3
+$(PDDF_PLATFORM_API_BASE_PY3)_DEPENDS = $(SONIC_CONFIG_ENGINE)
+#$(PDDF_PLATFORM_API_BASE_PY3)_DEPENDS += $(PDDF_PLATFORM_API_BASE_PY2)
+$(PDDF_PLATFORM_API_BASE_PY3)_TEST = n
+SONIC_PYTHON_WHEELS += $(PDDF_PLATFORM_API_BASE_PY3)
+
+export pddf_platform_api_base_py3_wheel_path="$(addprefix $(PYTHON_WHEELS_PATH)/,$(PDDF_PLATFORM_API_BASE_PY3))"
+export PDDF_PLATFORM_API_BASE_PY3

git diff build_debian.sh

diff --git a/build_debian.sh b/build_debian.sh
index 6505a8e56..ba08965fd 100755
--- a/build_debian.sh
+++ b/build_debian.sh
@@ -281,6 +281,7 @@ sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y in
python-setuptools
python3-setuptools
python-jsonschema
*+ python3-jsonschema *
python-apt
traceroute
iputils-ping \

jleveque · 2020-12-28T20:00:08Z

@fk410167: Thank you for testing and for your diffs! I have made the changes you suggested with one difference:

Rather than explicitly installing the python3-jsonschema Debian package, I removed the explicit installation of the Python 2 python-jsonschema package from both the host OS and the PMon container and instead added jsonschema=2.6.0 to the install_requires= list in setup.py. This way, the appropriate jsonschema package (Python 2 or Python 3) will get installed along with the PDDF package at install time via pip. Note that I pinned the version to 2.6.0 to match the latest version which is available via Debian. Version 3.2.0 is available via pip, but I wasn't sure if there would be any issues with PDDF and that version. You can feel free to test this out in your own time and update the version in setup.py in the future.

Please test and review again at your next convenience.

FuzailBrcm · 2020-12-29T04:12:40Z

retest this please

FuzailBrcm · 2020-12-29T09:21:57Z

@jleveque

Getting this error during the build.

12:25:29 /sonic/platform/pddf/platform-api-pddf-base /sonic
12:25:29 error in sonic-platform-pddf-common setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'=2.6.0'"
12:25:29 [ FAIL LOG END ] [ target/python-wheels/sonic_platform_pddf_common-1.0-py2-none-any.whl ]
12:25:29 make: *** [slave.mk:596: target/python-wheels/sonic_platform_pddf_common-1.0-py2-none-any.whl] Error 1
12:25:30 DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
12:26:47 make[1]: *** [Makefile.work:263: target/sonic-broadcom.bin] Error 2
12:26:47 make[1]: Leaving directory '/data/johnar/workspace/broadcom/buildimage-brcm-all-pr'
12:26:47 make: *** [Makefile:9: target/sonic-broadcom.bin] Error 2

jleveque · 2020-12-29T22:11:49Z

@fk410167: Thanks for the heads-up. I omitted an =. Should be fixed now.

FuzailBrcm · 2020-12-30T11:32:13Z

retest broadcom please

…pendency in setup.py

FuzailBrcm · 2021-01-06T04:25:11Z

retest mellanox please

jleveque · 2021-01-06T20:05:52Z

@fk410167: All check builds have passed. Can you please confirm the PR is good on your end?

FuzailBrcm · 2021-01-07T03:53:20Z

@jleveque
Yes. Changes are good to go from my end.

lguohan · 2021-01-07T14:29:31Z

@fk410167, please sign off then?

FuzailBrcm

Changes looks good to me

- Make PDDF code compliant with both Python 2 and Python 3 - Align code with PEP8 standards using autopep8 - Build and install both Python 2 and Python 3 PDDF packages

jleveque marked this pull request as draft December 23, 2020 22:47

jleveque added PDDF Python 3 labels Dec 23, 2020

lguohan force-pushed the master branch from 3690c1a to 512eb6b Compare December 25, 2020 18:34

jleveque added 7 commits January 4, 2021 23:03

[PDDF] Build and install Python 3 package

e3e94c1

[PDDF] Make Python 3-compliant

6541212

Align code with PEP8 standards using autopep8

8ebc300

Enhance makefile and deps file

48c0465

Don't explicitly install Python jsonschema Debian package, list as de…

0825157

…pendency in setup.py

Add missing equals sign

5f34206

Install Python 2 setuptools package, needed to install Python 2 thrift

206de39

jleveque added the Request for 202012 Branch label Jan 5, 2021

jleveque marked this pull request as ready for review January 6, 2021 20:05

jleveque requested a review from lguohan January 7, 2021 06:43

lguohan approved these changes Jan 7, 2021

View reviewed changes

FuzailBrcm approved these changes Jan 7, 2021

View reviewed changes

jleveque merged commit e52581e into sonic-net:master Jan 7, 2021

jleveque deleted the pddf_py3 branch January 7, 2021 18:03

yxieca added the Included in 202012 Branch label Feb 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PDDF] Build and install Python 3 package #6286

[PDDF] Build and install Python 3 package #6286

jleveque commented Dec 23, 2020

jleveque commented Dec 23, 2020

FuzailBrcm commented Dec 24, 2020

FuzailBrcm commented Dec 28, 2020

jleveque commented Dec 28, 2020 •

edited

Loading

FuzailBrcm commented Dec 29, 2020

FuzailBrcm commented Dec 29, 2020

jleveque commented Dec 29, 2020

FuzailBrcm commented Dec 30, 2020

FuzailBrcm commented Jan 6, 2021

jleveque commented Jan 6, 2021

FuzailBrcm commented Jan 7, 2021

lguohan commented Jan 7, 2021

FuzailBrcm left a comment

[PDDF] Build and install Python 3 package #6286

[PDDF] Build and install Python 3 package #6286

Conversation

jleveque commented Dec 23, 2020

jleveque commented Dec 23, 2020

FuzailBrcm commented Dec 24, 2020

FuzailBrcm commented Dec 28, 2020

git diff platform/pddf/platform-api-pddf-base.dep

git diff platform/pddf/platform-api-pddf-base.mk

git diff build_debian.sh

jleveque commented Dec 28, 2020 • edited Loading

FuzailBrcm commented Dec 29, 2020

FuzailBrcm commented Dec 29, 2020

jleveque commented Dec 29, 2020

FuzailBrcm commented Dec 30, 2020

FuzailBrcm commented Jan 6, 2021

jleveque commented Jan 6, 2021

FuzailBrcm commented Jan 7, 2021

lguohan commented Jan 7, 2021

FuzailBrcm left a comment

Choose a reason for hiding this comment

jleveque commented Dec 28, 2020 •

edited

Loading