Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sickchill 6 7 rebuild with Py3.8.12-6 #4920

Merged
merged 26 commits into from
Nov 5, 2021
Merged

Sickchill 6 7 rebuild with Py3.8.12-6 #4920

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
f5bf3b6
Rebuild with Py3.8.12
BKSteve Oct 17, 2021
cfdda49
Sickchill inc 6&7 with new py3.8.12
BKSteve Oct 17, 2021
95d537d
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 18, 2021
fd19365
Sickchill cross and requirements updates
BKSteve Oct 18, 2021
ec6e327
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 20, 2021
1fc3295
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 23, 2021
ca98d0b
Sickchill with 3.8.12-6
BKSteve Oct 23, 2021
6641331
Sickchill for 3.8.12-6 missed a file
BKSteve Oct 23, 2021
e43c748
Update Makefile
BKSteve Oct 23, 2021
cef5f82
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 25, 2021
3b0727f
SickChill 6 7 with Py3.8.12-6 tested working!
BKSteve Oct 26, 2021
cf3d0b4
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 26, 2021
267bcde
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 30, 2021
f69ad99
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Oct 31, 2021
b1c559a
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Nov 2, 2021
8ddde6b
Update wizard
publicarray Nov 3, 2021
47cb257
Merge branch 'SynoCommunity:master' into Sickchill-6-7
BKSteve Nov 3, 2021
5a4fdd5
Update spk/sickchill/Makefile v3
BKSteve Nov 3, 2021
8432579
Update spk/sickchill/src/requirements.txt v3
BKSteve Nov 3, 2021
504081f
fix build
publicarray Nov 3, 2021
48b7d0b
Merge branch 'master' into Sickchill-6-7
publicarray Nov 4, 2021
0696846
revert some 47cb2579fb5a7c543d4c94b3161c1f696510c48c changes
publicarray Nov 4, 2021
24b205c
Revert "revert some 47cb2579fb5a7c543d4c94b3161c1f696510c48c changes"
publicarray Nov 4, 2021
7c5511d
python: Fix build issue of -j $(CPU) flag not always recognized
th0ma7 Nov 4, 2021
81dfac8
Revert "Revert "revert some 47cb2579fb5a7c543d4c94b3161c1f696510c48c …
publicarray Nov 5, 2021
51e2673
SickChill 6 7 update requirements* and makefile
BKSteve Nov 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions mk/spksrc.common.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ RUN = cd $(WORK_DIR)/$(PKG_DIR) && env $(ENV)
PIP ?= pip
# Why ask for the same thing twice? Always cache downloads
PIP_CACHE_OPT ?= --cache-dir $(PIP_DIR)
PIP_ARGS = --no-binary :all: $(PIP_CACHE_OPT) --no-deps
PIP_WHEEL = $(PIP) wheel $(PIP_ARGS) --global-option="build_ext" --global-option="-j$(NCPUS)" --wheel-dir $(WHEELHOUSE)
PIP_WHEEL_ARGS = wheel --no-binary :all: $(PIP_CACHE_OPT) --no-deps --wheel-dir $(WHEELHOUSE)
PIP_WHEEL = $(PIP) $(PIP_WHEEL_ARGS)
PIP_DOWNLOAD = $(PIP) download $(PIP_ARGS) --dest $(BASE_DISTRIB_DIR)

# Available languages
Expand Down
2 changes: 1 addition & 1 deletion mk/spksrc.wheel.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ build_wheel_target: $(PRE_WHEEL_TARGET)
if [ -z "$(CROSSENV)" ]; then \
$(RUN) _PYTHON_HOST_PLATFORM="$(TC_TARGET)" CFLAGS="$(CFLAGS) -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) $(WHEELS_CFLAGS)" LDFLAGS="$(LDFLAGS) $(WHEELS_LDFLAGS)" $(PIP_WHEEL) --requirement $(WHEELHOUSE)/$(WHEELS_CROSS_COMPILE) ; \
else \
. $(CROSSENV) && $(RUN) _PYTHON_HOST_PLATFORM="$(TC_TARGET)" CFLAGS="$(CFLAGS) -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) $(WHEELS_CFLAGS)" LDFLAGS="$(LDFLAGS) $(WHEELS_LDFLAGS)" $(PIP_WHEEL) --no-build-isolation --requirement $(WHEELHOUSE)/$(WHEELS_CROSS_COMPILE) ; \
. $(CROSSENV) && $(RUN) _PYTHON_HOST_PLATFORM="$(TC_TARGET)" CFLAGS="$(CFLAGS) -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) $(WHEELS_CFLAGS)" LDFLAGS="$(LDFLAGS) $(WHEELS_LDFLAGS)" pip $(PIP_WHEEL_ARGS) --no-build-isolation --requirement $(WHEELHOUSE)/$(WHEELS_CROSS_COMPILE) ; \
Copy link
Member

@publicarray publicarray Nov 5, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@th0ma7 Ah found it! pip and $(PIP) are not the same. Or at least pip works and $(PIP) doesn't

# dosn't work
. $(CROSSENV) && $(RUN) _PYTHON_HOST_PLATFORM="$(TC_TARGET)" CFLAGS="$(CFLAGS) -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) $(WHEELS_CFLAGS)" LDFLAGS="$(LDFLAGS) $(WHEELS_LDFLAGS)" $(PIP) wheel $(PIP_ARGS) --wheel-dir $(WHEELHOUSE) --no-build-isolation --requirement $(WHEELHOUSE)/$(WHEELS_CROSS_COMPILE) ; \

# works
. $(CROSSENV) && $(RUN) _PYTHON_HOST_PLATFORM="$(TC_TARGET)" CFLAGS="$(CFLAGS) -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) $(WHEELS_CFLAGS)" LDFLAGS="$(LDFLAGS) $(WHEELS_LDFLAGS)" pip wheel $(PIP_ARGS) --wheel-dir $(WHEELHOUSE) --no-build-isolation --requirement $(WHEELHOUSE)/$(WHEELS_CROSS_COMPILE) ; \

Copy link
Member

@publicarray publicarray Nov 5, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we in a virtual environment? aka CROSSENV

Copy link
Member

@publicarray publicarray Nov 5, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @th0ma7 I'm going to revert the 2 files (mk/spksrc.common.mk, mk/spksrc.wheel.mk) from 98dc6b4#diff-b2c976fb01fa035ca37be782346b9773732c78d931102770d1a305d9cfdc6a06

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to go over the various changes you did (any left or not?) so my other PR doesn't get too funky neither.
I'm glad you have sorted it out, good job!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reading it I don't understand:

  1. why would it work using pip instead of $(PIP) when really it's the same thing? If not we need to find out what's the difference and why (and probably add a comment on top of PIP ?= pip to explain it)
  2. Otherwise my "guess" is that the re-ordering in PIP_WHEEL_ARGS is what fixed the issue. On the other hand, if it was only pip then why changing that?
  3. Currently PIP_DONWLOAD calls PIP_ARGS which is now existent (probably easy to fix)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before I rebase against master my PR, can you double-check this and confirm a few things? Otherwise I fear I may break things back for sickchill while fixing PIP_DOWNLOAD?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, just wow!
@publicarray I've been double-checking and I just don't get it... it does solves the issue no matter why?

I've checked the following:

                which $(PIP) ;\
                which pip ;\
                $(MSG) $(PIP_WHEEL) ;\
                $(MSG) $(PIP) $(PIP_WHEEL_ARGS) ;\
                $(MSG) pip $(PIP_WHEEL_ARGS) ;\

Which gives this:

/usr/local/bin/pip
/usr/local/bin/pip
===>  pip wheel --no-binary :all: --cache-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/../../distrib/pip --no-deps --wheel-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/wheelhouse
===>  pip wheel --no-binary :all: --cache-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/../../distrib/pip --no-deps --wheel-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/wheelhouse
===>  pip wheel --no-binary :all: --cache-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/../../distrib/pip --no-deps --wheel-dir /home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/wheelhouse

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found it!!!!! 😄

  • The defined $(PIP) in spksrc.common.mk points to the native binary.
  • When direct pip is called with . $(CROSSENV) && $(RUN) ... it points to cross-environment as the path gets changed such as:
$ cat work-x64-7.0/crossenv/bin/activate
. /home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/crossenv/cross/bin/activate
export PATH=/home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/crossenv/bin:$PATH

Resulting in:

::::::::::::::
work-x64-7.0/pip.1
::::::::::::::
pip 21.3 from /home/spksrc/python310-fixes/spksrc/native/python38/work-native/install/usr/local/lib/python3.8/site-packages/pip (python 3.8)
::::::::::::::
work-x64-7.0/pip.2
::::::::::::::
pip 21.3 from /home/spksrc/python310-fixes/spksrc/spk/sickchill/work-x64-7.0/crossenv/build/lib/python3.8/site-packages/pip (python 3.8)

fi ; \
fi ; \
if [ -f "$(WHEELHOUSE)/$(WHEELS_PURE_PYTHON)" ]; then \
Expand Down
19 changes: 8 additions & 11 deletions spk/sickchill/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
SPK_NAME = sickchill
SPK_VERS = 20210729
SPK_REV = 2
SPK_REV = 3
SPK_ICON = src/sickchill.png

DEPENDS = cross/$(SPK_NAME)
SPK_DEPENDS = "python38:git"
PIP=pip3
BUILD_DEPENDS = cross/python38 cross/poetry cross/lxml
# cross/cffi cross/setuptools cross/cryptography
DEPENDS = cross/PyNaCl cross/$(SPK_NAME)
SPK_DEPENDS = "python38"

WHEELS = src/requirements.txt
WHEELS = src/requirements-cross.txt src/requirements.txt

MAINTAINER = miigotu
DESCRIPTION = Automatic Video Library Manager for TV Shows. It watches for new episodes of your favorite shows, and when they are posted it does its magic.
DESCRIPTION_SPN = Gestor automático para bibliotecas de series. Busca nuevos episodios de tus series favoritas, y cuando son publicados hace su magia.
BKSteve marked this conversation as resolved.
Show resolved Hide resolved
STARTABLE = yes
DISPLAY_NAME = SickChill
CHANGELOG = "Initial SickChill env fix and move to Python 3.8"
CHANGELOG = "1. SickChill environment fix<br/>2. Move to Python 3.8 without git<br/>3. Added DSM7 Support"

HOMEPAGE = https://sickchill.github.io/
LICENSE = GPL
LICENSE = GPLv3+

SERVICE_USER = auto
SERVICE_SETUP = src/service-setup.sh
Expand All @@ -31,9 +31,6 @@ WIZARDS_DIR = src/wizard/

POST_STRIP_TARGET = sickchill_extra_install

# Pure Python package, make sure ARCH is not defined
override ARCH=

include ../../mk/spksrc.spk.mk

.PHONY: sickchill_extra_install
Expand Down
15 changes: 15 additions & 0 deletions spk/sickchill/src/requirements-cross.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
cffi==1.14.6
chardet==4.0.0
click==8.0.1
###cryptography==3.4.7
future==0.18.2
greenlet==1.1.0
###importlib-metadata==1.7.0
kodipydent-alt==2021.7.16
###py38 lxml==4.6.3
MarkupSafe==2.0.1
msgpack==1.0.2
###PyNaCl==1.4.0
sgmllib3k==1.0.0
SQLAlchemy==1.4.25 #py38
wrapt==1.12.1
108 changes: 49 additions & 59 deletions spk/sickchill/src/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,88 +1,78 @@
appdirs==1.4.4
attrdict==2.0.1
babelfish==0.5.5
beautifulsoup4==4.9.3
###beekeeper==0.9.2
###py38 appdirs==1.4.4
babelfish==0.6.0
beautifulsoup4==4.10.0
beekeeper-alt==2021.7.16
bencode.py==4.0.0
CacheControl==0.12.6
certifi==2020.11.8
###cffi==1.14.3
chardet==3.0.4
click==7.1.2
cloudscraper==1.2.48
cachecontrol==0.12.6
certifi==2021.10.8
charset-normalizer==2.0.3
cloudscraper==1.2.58
colorama==0.4.4
configobj==5.0.6
###cryptography==3.2.1
decorator==4.4.2
decorator==5.0.9
deluge-client==1.9.0
Deprecated==1.2.10
dogpile.cache==1.0.2
deprecated==1.2.12
dogpile.cache==1.1.3
enzyme==0.4.1
feedparser==6.0.2
future==0.18.2
feedparser==6.0.8
gntp==1.0.3
guessit==3.1.1
guessit==3.3.1
html5lib==1.1
httplib2==0.18.1
idna==2.10
httplib2==0.19.1
###py38 idna==3.3
ifaddr==0.1.7
imagesize==1.2.0
IMDbPY==2020.9.25
ipaddress==1.0.23
Js2Py==0.70
IMDbPY==2021.4.18
###py38 ipaddress==1.0.23
Js2Py==0.71
jsonrpclib-pelix==0.4.2
###kodipydent==0.3.1
###lxml==4.6.1
Mako==1.1.3
markdown2==2.3.10
MarkupSafe==1.1.1
msgpack==1.0.0
Mako==1.1.4
markdown2==2.4.0
new-rtorrent-python==1.0.1a0
oauthlib==3.1.0
packaging==20.4
pbr==5.5.1
oauthlib==3.1.1
packaging==20.9
pbr==5.6.0
profilehooks==1.12.0
putio.py==8.7.0
pyaes==1.6.1
pycparser==2.20
PyGithub==1.53
###py38 pycparser==2.20
pygithub==1.55
###py38 pyopenssl==21.0.0
pyjsparser==2.7.1
PyJWT==1.7.1
pymediainfo==4.3
PyNMA==1.0
pyOpenSSL==19.1.0
pyjwt==2.1.0
pymediainfo==5.1.0
pynma==1.0
###py38 pyopenssl==20.0.1
pyparsing==2.4.7
PySocks==1.7.1
pysocks==1.7.1
pysrt==1.1.2
python-dateutil==2.8.1
python-slugify==4.0.1
###python-dateutil==2.8.2
python-slugify==5.0.2
python-twitter==3.5
python3-fanart==2.0.0
pytz==2020.4
qbittorrent-api==2020.10.11
pytz==2021.1
qbittorrent-api==2021.5.22
rarfile==4.0
rebulk==2.0.1
requests==2.24.0
rebulk==3.0.1
requests==2.26.0
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
Send2Trash==1.5.0
sgmllib3k==1.0.0
six==1.15.0
soupsieve==2.0.1
SQLAlchemy==1.3.20
stevedore==3.2.2
send2trash==1.7.1
###py38 six==1.16.0
soupsieve==2.2.1
stevedore==3.3.0
subliminal==2.1.0
text-unidecode==1.3
timeago==1.0.14
tmdbsimple==2.6.6
timeago==1.0.15
tmdbsimple==2.8.0
tornado==6.1
tus.py==1.3.4
tvdbsimple==1.0.6
twilio==6.47.0
tzlocal==2.1
Unidecode==1.1.1
urllib3==1.26.0
validators==0.18.1
unidecode==1.2.0
urllib3==1.26.6
validators==0.18.2
webencodings==0.5.1
win-inet-pton==1.1.0
wrapt==1.12.1
xmltodict==0.10.1
xmltodict==0.12.0
###0.0.0 issue zipp==3.6.0
24 changes: 16 additions & 8 deletions spk/sickchill/src/service-setup.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
PYTHON_DIR="/var/packages/python38/target"
PIP=${SYNOPKG_PKGDEST}/env/bin/pip3
PATH="${SYNOPKG_PKGDEST}/bin:${SYNOPKG_PKGDEST}/env/bin:${PYTHON_DIR}/bin:${PATH}"
HOME="${SYNOPKG_PKGDEST}/var"
VIRTUALENV="${PYTHON_DIR}/bin/virtualenv"
HOME="${SYNOPKG_PKGVAR}"
VIRTUALENV="${PYTHON_DIR}/bin/python3 -m venv"
PYTHON="${SYNOPKG_PKGDEST}/env/bin/python3"
SC_INSTALL_DIR="${SYNOPKG_PKGDEST}/share/SickChill"
SC_BINARY="${SC_INSTALL_DIR}/SickChill.py"
SC_DATA_DIR="${SYNOPKG_PKGDEST}/var/data"
SC_DATA_DIR="${SYNOPKG_PKGVAR}/data"
SC_CFG_FILE="${SC_DATA_DIR}/config.ini"


Expand Down Expand Up @@ -42,20 +42,28 @@ EOF
service_postinst() {
# Create a Python virtualenv
${VIRTUALENV} --system-site-packages ${SYNOPKG_PKGDEST}/env


# attempt to get current pip updated during install procedure
${PYTHON} -m pip install --upgrade pip

# Install the wheels
${PIP} install --no-deps --no-index -U --force-reinstall -f ${SYNOPKG_PKGDEST}/share/wheelhouse ${SYNOPKG_PKGDEST}/share/wheelhouse/*.whl

wheelhouse=${SYNOPKG_PKGDEST}/share/wheelhouse
${PIP} install --extra-index-url https://wheel-index.linuxserver.io/ubuntu/ --upgrade --force-reinstall --find-links ${wheelhouse} ${wheelhouse}/*.whl
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious what is the --extra-index-url for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an initial suggestion from the SickChill team as there were lots of build issues with cryptography and this site has builds cp38 of crypography not on Pypi.org. So just giving those poor old DSM a chance to look here. Not sufficiently knowledge on this, so probably unnecessary. But my assumption was that if we install and have this here then the DSM can see if there are valid newer versions of some packages for their particular hardware not provided on pypi e.g. SickChill looks for cryptography 3.4.7.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation, yea the less the DSM has to compile the better 👍


if [ "${SYNOPKG_PKG_STATUS}" == "INSTALL" ]; then
set_config
fi

set_unix_permissions "${SYNOPKG_PKGDEST}"
if [ $SYNOPKG_DSM_VERSION_MAJOR -lt 7 ]; then
set_unix_permissions "${SYNOPKG_PKGDEST}"
fi
}

service_postupgrade() {
set_config
set_unix_permissions "${SYNOPKG_PKGDEST}"
if [ $SYNOPKG_DSM_VERSION_MAJOR -lt 7 ]; then
set_unix_permissions "${SYNOPKG_PKGDEST}"
fi
}

service_preupgrade ()
Expand Down
2 changes: 1 addition & 1 deletion spk/sickchill/src/wizard/install_uifile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@
}]
},
{
"desc": "Permissions for all download-related packages are managed with the group <b>'sc-download'</b> in DSM.<br>Please read <a target=\"_blank\" href=\"https://github.com/SynoCommunity/spksrc/wiki/Permission-Management\">Permission Management</a> for details."
"desc": "<p>The first time SickChill is started it might take a few moments for the interface to become available!</p><p>Permissions for all download-related packages are managed with the group <b>'sc-download'</b> in DSM.<br>Please read <a target=\"_blank\" href=\"https://github.com/SynoCommunity/spksrc/wiki/Permission-Management\">Permission Management</a> for details.</p>"
}]
}]
4 changes: 2 additions & 2 deletions spk/sickchill/src/wizard/upgrade_uifile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[{
"step_title": "Attention! DSM Permissions",
"step_title": "Attention!",
"items": [{
"desc": "Permissions for all download-related packages are managed with the group <b>'sc-download'</b> in DSM.<br>Please read <a target=\"_blank\" href=\"https://github.com/SynoCommunity/spksrc/wiki/Permission-Management\">Permission Management</a> for details."
"desc": "<p>The first time SickChill is started it might take a few moments for the interface to become available!</p><p>Permissions for all download-related packages are managed with the group <b>'sc-download'</b> in DSM.<br>Please read <a target=\"_blank\" href=\"https://github.com/SynoCommunity/spksrc/wiki/Permission-Management\">Permission Management</a> for details.</p>"
}]
}]