Skip to content

Package everything #248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

DifferentialOrange
Copy link
Member

@DifferentialOrange DifferentialOrange commented Oct 19, 2022

make: remove obsolete build commands

Part of #198

python: use explicit binaries version

Specify explicit Python 3 python3 and pip3 binaries everywhere to make
commands more portable.

Part of #198

make: separate phony for each target

Part of #198

ci: remove source code with script

Remove connector source code for pure install tests with Python script.
Python was chosen since it would work both on Windows and Linux.

Part of #198

make: target for test pure install

Part of #198

setup: update description

Part of #198

cd: pack pip package

Pack source code and wheel file with Github Actions. Result is stored as
artifact.

Part of #198

ci: run tests with pip package

Part of #198

cd: publish pip package

Publish pip package on tag.

To test, comment "run only on tags" condition, change PYPI_REPO
to testpypi and use ${{ secrets.TEST_PYPI_TOKEN }} as PYPI_TOKEN. You
also need to remove 5dfdae5 commit
changes about scm version and set some constant version: using local
version identifiers [1] is not allowed by test.pypi.org.

  1. https://peps.python.org/pep-0440/

Part of #198

cd: pack RPM package

New RPM spec is based on RHEL RPM guide for Python packages [1] merged
with results of python3 setup.py bdist_rpm --spec-only.

Beware that RPM name is changed based on recommendations for all
mainstream distributives (for example, see [2]). Binary RPM is named
python3-tarantool and source RPM is named python-tarantool. Before the
patch they both were called tarantool-python (even though there wasn't
new RPM releases since 0.6.5).

RPM is suitable for distributives with Python 3.7 or newer. See [3]
about pre-Python 3.7 systems support.

  1. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/installing_and_using_dynamic_programming_languages/assembly_packaging-python-3-rpms_installing-and-using-dynamic-programming-languages
  2. https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines#Python_source_package_naming
  3. Pack RPM/DEB for obsolete distros #257

Part of #164, #198

test: make host more portable

Docker container localhost resolve fails to connect test suites
to test Tarantool instances.

Part of #164, #198

ci: run tests with RPM package

See [1] about CentOS support. Fedora 37 is not tested yet since there is
no Tarantool for it.

  1. Pack RPM/DEB for obsolete distros #257

Part of #164, #198

cd: publish RPM package

Publish RPM package on tag.

To test, comment "run only on tags" condition, change RWS_REPO
to https://rws-dev.tarantool.org.

See [1] about other OS support.

  1. Pack RPM/DEB for obsolete distros #257

Closes #164, part of #198

cd: pack deb package

New deb spec files are based on stdeb plugin generated files [1] and
Debian Style Guide for Packaging Python Libraries [2].

Beware that RPM name is changed based on recommendations for all
mainstream distributives [2]. deb files are named with python3-tarantool
prefix. Before the patch they were called tarantool-python (even though
there wasn't new deb releases since 0.6.5).

deb is suitable for distributives with Python 3.7 or newer. See [3]
about pre-Python 3.7 systems support.

Since deb requires changelog consistency to build, we generate
a changelog entry for all nightly CI builds.

  1. https://pypi.org/project/stdeb/
  2. https://wiki.debian.org/Python/LibraryStyleGuide?action=show&redirect=Python%2FPackaging
  3. Pack RPM/DEB for obsolete distros #257

Part of #198

fix: timezone offset with old pytz and pandas

It seems that older pandas and pytz distributions (for example, ones
from Debian 10 deb repositories) are prone to issues when pytz timezone
offset ignores current datetime value [1]. Using explicit localize is
both valid in modern version and fixes the bug for older ones.

  1. https://stackoverflow.com/questions/11473721/weird-timezone-issue-with-pytz

Part of #198

ci: run tests with deb package

See [1] about older versions support. Only LTS Ubuntu versions are
included here.

  1. Pack RPM/DEB for obsolete distros #257

Part of #198

cd: publish deb package

Publish deb artifacts on tag.

To test, comment "run only on tags" condition, change RWS_REPO
to https://rws-dev.tarantool.org.

See [1] about other OS support.

  1. Pack RPM/DEB for obsolete distros #257

Closes #198

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 5 times, most recently from 91311b6 to d09ccd8 Compare October 20, 2022 06:21
@DifferentialOrange DifferentialOrange changed the base branch from master to DifferentialOrange/gh-238-autoversion October 20, 2022 06:38
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 9 times, most recently from 8966797 to 64d366b Compare October 20, 2022 11:57
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-238-autoversion branch from f759b8d to 4b6d298 Compare October 21, 2022 12:00
Base automatically changed from DifferentialOrange/gh-238-autoversion to master October 21, 2022 12:03
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 9 times, most recently from a057abe to f9fa4ab Compare November 1, 2022 09:02
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 4 times, most recently from 3f97cc1 to 903db5f Compare November 1, 2022 10:16
Copy link

@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi. Thank you for the patchset.
Great work.

  • "Specify explicit Python 3 python3 and pip3 binaries everywhere to make commands more portable." - why does it make commands more portable?

See several comments bellow.

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 2 times, most recently from 58ee2f4 to 91ed0bc Compare November 9, 2022 08:48
@DifferentialOrange
Copy link
Member Author

DifferentialOrange commented Nov 9, 2022

"Specify explicit Python 3 python3 and pip3 binaries everywhere to make commands more portable." - why does it make commands more portable?

python3 is Python 3 binary for all systems which have Python 3 installed. For some configurations, python is python3 by default, sometimes you can make python to be python3 by installing python-is-python3 package or by messing with aliases (but I don't think we should enforce users to do so), but sometimes system has python2 for python. Same goes for pip (and I think I've met the case when python was python3 but pip was pip for Python 2).

Since our code supports only Python 3, we must ensure that everything runs with Python 3 only.

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch 2 times, most recently from f69ffc0 to a20bd14 Compare November 9, 2022 09:18
The only reason of this dependency is various vulnerability fixes. We
decided not to enforce the user on this.

Follows #223, part of #198
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch from a20bd14 to e7f8202 Compare November 9, 2022 09:37
Copy link

@LeonidVas LeonidVas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Pack source code and wheel file with Github Actions. Result is stored as
artifact.

Part of #198
Publish pip package on tag.

To test, comment "run only on tags" condition, change PYPI_REPO
to testpypi and use ${{ secrets.TEST_PYPI_TOKEN }} as PYPI_TOKEN. You
also need to remove 5dfdae5 commit
changes about scm version and set some constant version: using local
version identifiers [1] is not allowed by test.pypi.org.

1. https://peps.python.org/pep-0440/

Part of #198
New RPM spec is based on RHEL RPM guide for Python packages [1] merged
with results of `python3 setup.py bdist_rpm --spec-only`.

Beware that RPM name is changed based on recommendations for all
mainstream distributives (for example, see [2]). Binary RPM is named
python3-tarantool and source RPM is named python-tarantool. Before the
patch they both were called tarantool-python (even though there wasn't
new RPM releases since 0.6.5).

RPM is suitable for distributives with Python 3.7 or newer. See [3]
about pre-Python 3.7 systems support.

1. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/installing_and_using_dynamic_programming_languages/assembly_packaging-python-3-rpms_installing-and-using-dynamic-programming-languages
2. https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines#Python_source_package_naming
3. #257

Part of #164, #198
Docker container localhost resolve fails to connect test suites
to test Tarantool instances.

Part of #164, #198
See [1] about CentOS support. Fedora 37 is not tested yet since there is
no Tarantool for it.

1. #257

Part of #164, #198
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch from e7f8202 to e499014 Compare November 9, 2022 09:59
Publish RPM package on tag.

To test, comment "run only on tags" condition, change RWS_REPO
to https://rws-dev.tarantool.org.

See [1] about other OS support.

1. #257

Closes #164, part of #198
New deb spec files are based on stdeb plugin generated files [1] and
Debian Style Guide for Packaging Python Libraries [2].

Beware that RPM name is changed based on recommendations for all
mainstream distributives [2]. deb files are named with python3-tarantool
prefix. Before the patch they were called tarantool-python (even though
there wasn't new deb releases since 0.6.5).

deb is suitable for distributives with Python 3.7 or newer. See [3]
about pre-Python 3.7 systems support.

Since deb requires changelog consistency to build, we generate
a changelog entry for all nightly CI builds.

1. https://pypi.org/project/stdeb/
2. https://wiki.debian.org/Python/LibraryStyleGuide?action=show&redirect=Python%2FPackaging
3. #257

Part of #198
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch from e499014 to 64931d4 Compare November 9, 2022 10:04
It seems that older pandas and pytz distributions (for example, ones
from Debian 10 deb repositories) are prone to issues when pytz timezone
offset ignores current datetime value [1]. Using explicit localize is
both valid in modern version and fixes the bug for older ones.

1. https://stackoverflow.com/questions/11473721/weird-timezone-issue-with-pytz

Part of #198
See [1] about older versions support. Only LTS Ubuntu versions are
included here.

1. #257

Part of #198
Publish deb artifacts on tag.

To test, comment "run only on tags" condition, change RWS_REPO
to https://rws-dev.tarantool.org.

See [1] about other OS support.

1. #257

Closes #198
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/gh-198-package-everything branch from 64931d4 to b23193a Compare November 9, 2022 10:13
@DifferentialOrange DifferentialOrange merged commit 53b3b51 into master Nov 9, 2022
@DifferentialOrange DifferentialOrange deleted the DifferentialOrange/gh-198-package-everything branch November 9, 2022 10:28
DifferentialOrange added a commit that referenced this pull request Dec 22, 2022
Use the same rules in packing as in testing to run on dev branches.

Follows #248
DifferentialOrange added a commit that referenced this pull request Dec 22, 2022
Use the same rules in packing as in testing to run on dev branches.

Follows #248
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Autodeploy RPM/Deb packages Deploy packages for CentOS 8 and Fedora 31
2 participants