Add DSM 7 Support #4395

publicarray · 2021-01-17T21:34:56Z

I think we are getting close enough that packages can be build and run, things left to do are documenting the new permissions/wizards, database creation from Synology resource worker, maybe more testing that there are no regressions for DSM 6 packages (I think I've found them all but one can't be too careful). Also, there is a Docker resource worker, but that will need individual packages to migrate to it.

Note: I frequently rebase with master to get the latest fixes and to avoid a large merge conflict.
This means there will be force pushes!

hgy59 · 2021-01-17T22:04:54Z

@publicarray sorry, had to fixup the gh build action for noarch-7.0

publicarray · 2021-01-17T22:14:01Z

@hgy59 no need to apologize, If you follow the git log I've made many silly mistakes. Also debugging GitHub actions is hard since you can't run them locally (There is https://github.com/nektos/act/ but It's a little harder to setup with GH events)

hgy59 · 2021-01-17T22:54:08Z

Now both the noarch-7.0 (dsm7) and noarch (all) are built...

th0ma7

This is looking quite good.
I noticed quite a few missing SHA256 in the toolchains.
Beyond that I'm much excited to give this a road test...

toolchain/syno-aarch64-7.0/Makefile

.github/workflows/build.yml

cross/libaom/Makefile

mk/spksrc.service.start-stop-status

spk/nzbget-testing/src/service-setup.sh

toolchain/syno-apollolake-7.0/digests

toolchain/syno-armada370-7.0/digests

toolchain/syno-armada375-7.0/digests

toolchain/syno-armadaxp-7.0/digests

toolchain/syno-avoton-7.0/digests

SynoCommunity#4395 (comment)

mk/spksrc.spk.mk

mk/spksrc.tc-vers.mk

toolkit/syno-evansport-7.0/digests

toolchain/syno-geminilake-7.0/digests

hgy59 · 2021-01-20T08:17:38Z

@publicarray finally the adminer package for DSM7 is installable through the package center.

publicarray · 2021-01-25T09:16:29Z

@hgy59 Awesome, Thank you!

mk/spksrc.spk.mk

mk/spksrc.tc-vers.mk

hgy59 · 2021-01-27T20:26:55Z

@publicarray I have added the python script to create the resource file (as discussed)

The integration in spksrc.service.mk would look like:

$(DSM_CONF_DIR)/resource:
	$(create_target_dir)
	@echo '{}' > $@
ifneq ($(strip $(SPK_COMMANDS) $(SERVICE_WIZARD_SHARE)),)
	@../../mk/get_resources.py SPK_COMMANDS "$(SPK_COMMANDS)" SERVICE_WIZARD_SHARE "$(SERVICE_WIZARD_SHARE)"  >$@
endif
SERVICE_FILES += $(DSM_CONF_DIR)/resource
# STARTABLE needs to be yes, the resource linking and unlinking works on start and stop
# see spsrc.spk.mk

SynoCommunity#4395 (comment)

th0ma7 · 2021-02-06T13:20:15Z

@publicarray how far you think we are to getting this merge?
I haven't been involved much myself on this but let me know how I can help.

publicarray · 2021-02-06T13:54:39Z

I have been waiting for a reply from Synology regarding the creation of databases. It's been a few days... but after that I think we can merge this. There is always a need to build more packages and find more broken ones. Maybe you can find out why chromaprint fails to build?

th0ma7 · 2021-02-06T18:21:59Z

@publicarray sure, I can definitively have a look at it.
Can you rebase as I just merged to master the latest ffmpeg changes which will have an impact on chromaprint (which built all-right on dsm6).

SynoCommunity#4395 (comment)

@th0ma7

Fixes wrong comparisons e.g. where 3.10 was erroneously less than 3.2 Thanks @th0ma7

depreciate SPK_LINKS and add new make variable: SPK_USR_LOCAL_LINKS

publicarray · 2021-02-21T07:26:41Z

Thanks @hgy59

It's done:

dsm7-framework (this PR)
dsm7-packages (to be clear: needs dsm7 otherwise builds fail)
dsm7-combined (probably won't be kept in sync, but good for quick building)

I am just wondering, how you separate the package specific stuff to a separate branch without much effort...

I used a GUI tool Sublime Merge. It makes it easy to look at individual commits and the files that changed, then cherry-pick and rebase (drop commit in the GUI)... Another tip: the CTRL key allows you to select multiple commits.

🤞 Fingers crossed I didn't mess up. So far the packages that I've tested still work on the combined branch.

hgy59

just two minor suggestions...

mk/spksrc.service.start-stop-status

mk/spksrc.spk.mk

Co-authored-by: hgy59 <hpgy59@gmail.com>

publicarray · 2021-03-11T01:35:35Z

@hgy59 do you think this is ready to be merged?

hgy59

LGTM

publicarray · 2021-03-11T05:40:28Z

Ok I'm hitting the big green button...

th0ma7 · 2021-03-11T11:20:05Z

woot! Awesome work, big big kudos for this!

Casuallynoted · 2021-03-27T05:35:11Z

Not super familiar with a lot of this stuff, does this mean that DSM 7 packages will begin going live on the repo?

hgy59 · 2021-03-27T19:31:11Z

Only the base (i.e. framework) part is merged yet. All the work done for packages to support dsm7 is detached on the dsm7-packages branch.

AFAIC see most of the CLI packages are close to be ready for dsm7 (synocli*, ruby, python*, git, gnupg, ...)
Packages that cannot run without root permissions are almost impossible to support dsm7 (ntopng, ...) other might be redesigned to run without root.
Some packages depend on mysql db and must be redesigned to use mariadb-10, as mysql/mariadb-5 is not available on dsm7 (fengoffice, cops, horde, mantisbt, owncloud, roundcube, selfoss, ttrss)

One breaking feature is the logging of package installation.
All packages that use >> ${INST_LOG} for logging fail as the log-file variable is not defined (and cannot be solved by defining the variable for the DSM7 log file).

@th0ma7

* prepare for dsm7 * Add 7.0 toolchain for testing * Add x64-7.0 to CI * start on dsm7 install/service script * fix DSM_APP_NAME in dsm7 * remove tools that require root permissions: (synouser, synogroup, synoshare) * remove tool that fail to modify permissions (synoacltool) * remove tool that fail to apply configuration (servicetool) servicetool, synouser, synogroup and synoshare have alternative ways to do the same thing using resource configuration * fix username, keep DSM6 name convention sc-packagename * Issue user facing warning on unsupported functions * change package icon size - change icon size from 72x72 to 64x64 for DSM 7 * cleanup * Store log files, pid and other files in permanent storage fixes missing log entries on reinstall * Migrate some JSON processing to jq TODO add moreutils package, use sponge to prevent overriding file with 0 content - 1<> $@ + | sponge $@ * fix adding GROUPs DSM7 * join-pkg-groupnames Creates group but is different from the groups the user can create, they are invisible in the UI an are only useful to access another packages permissions (ffmpeg comes to mind) For DSM7 I recommend setting permissions for individual packages (System Internal User) or use the shared folder resource worker to add permissions. We can ask the user from the wizard the folder name "join-groupname": "http" is for system groups, mainly for the web service ("root" don't work but "system" does) * DSM6->DSM7 move target/var to var/ * Added dsm7 preview toolchain for syno-armada38x-7.0 * Add DSM7.0 toolchains * Add v1000 to generic x64-7.0 * Add evansport DSM 7.0 Toolkit * fix builds for dsm versions < 7 * DSM7 Log install script to /var/log/packages/{package}.log Remove some more statements that modify files in places where we don't have permissions or is just unnecessary in DSM7 * add back firewall install/removal * Acquire Shared folder resource/permissions set using SERVICE_WIZARD_SHARE only disadvantage is that this does not support sub-folders currently * Only migrate /var directory to permanent storage when destination is empty * set SYNOPKG_PKGVAR for DSM versions less than 7.0 * Add uninstaller that can optionally remove all package files Includes files in the new permanent storage (/var) and home folder * uninstaller: also remove /etc * fix noarch in dsm7 simple hack to set os_min_ver to 7.0-4000 build using: make noarch-7.0 or make noarch-6.1 build must be clean between builds SynoCommunity#4215 (comment) * fix missing files on clean install Copy config files on first install from "target/var" to "/var" * Error on dropped support for busybox start-stop-daemon aka SERVICE_EXE * fix don't override old configurations * Add ability for packages to join a SYSTEM_GROUP and cleanup Available options are: http and system * fix DSM6 regression and code review * fix regression: backwards compatibility with DSM6 * for DSM6 don't set "groupname". fixes permission issues with sc-{groups} Add SPK_GROUP variable to override "groupname", default is sc-{packagename} * fix noarch for dsm7 - use noarch-dsm7 to distinguish from noach-all - use different work dirs for all and dsm7 - incorporate SynoCommunity#4392 - followup of 05c2383 to avoid to clean between builds * update github action to build noarch-7.0 packages * fix github build action for noarch-7.0 * another fix for noarch build action * fix DSM7 running with su command: SynoCommunity#4395 (comment) * Toolchains: Update DSM7 hashes and fix typos * Add supported DSM 7 archs to CI/GH Actions * fix os_min_vers for DSM 7 noarch - update min. build from 4000 to 40000 * regenerated digests of dsm7 toolchains * Add sha256 digest to evansport toolkit * fix TC_BUILD for DSM7.0 * Apply new version comparison function Fixes wrong comparisons e.g. where 3.10 was erroneously less than 3.2 Thanks @th0ma7 * framework, resource: simplify jq command * framework: code review * Add linking support for /usr/local/lib and /usr/local/etc depreciate SPK_LINKS and add new make variable: SPK_USR_LOCAL_LINKS * fix SPK_USR_LOCAL_LINKS * Fix comments being output to command line * Framework: DSM7 Make sure folder exists before migrating files * Framework: Fix unable to override uninstall wizard * Update mk/spksrc.service.start-stop-status Co-authored-by: hgy59 <hpgy59@gmail.com> * Fraework: avoid creating wizard folder when it's not needed Co-authored-by: hgy59 <hpgy59@gmail.com> Co-authored-by: hgy59 <hpgy59@gmail.com> Co-authored-by: root <zyrill@github.com>

publicarray requested a review from th0ma7 January 17, 2021 21:34

publicarray added the dsm 7 label Jan 17, 2021

publicarray mentioned this pull request Jan 17, 2021

ffmpeg Q1-2021 update #4390

Merged

8 tasks

th0ma7 requested changes Jan 18, 2021

View reviewed changes

publicarray added a commit to publicarray/spksrc that referenced this pull request Jan 19, 2021

fix DSM7 running with su command:

91a680d

SynoCommunity#4395 (comment)

th0ma7 requested changes Jan 19, 2021

View reviewed changes

mk/spksrc.spk.mk Outdated Show resolved Hide resolved

mk/spksrc.tc-vers.mk Outdated Show resolved Hide resolved

toolkit/syno-evansport-7.0/digests Outdated Show resolved Hide resolved

toolchain/syno-geminilake-7.0/digests Outdated Show resolved Hide resolved

publicarray mentioned this pull request Jan 25, 2021

Unable to install Mono package on DSM 7.0 #4322

Closed

th0ma7 requested changes Jan 27, 2021

View reviewed changes

mk/spksrc.spk.mk Outdated Show resolved Hide resolved

mk/spksrc.tc-vers.mk Outdated Show resolved Hide resolved

publicarray added a commit to publicarray/spksrc that referenced this pull request Jan 30, 2021

fix DSM7 running with su command:

965f355

SynoCommunity#4395 (comment)

publicarray force-pushed the dsm7 branch 3 times, most recently from f1462e2 to 5737457 Compare January 31, 2021 01:01

publicarray mentioned this pull request Jan 31, 2021

Framework: add version compare functions #4417

Merged

3 tasks

publicarray self-assigned this Jan 31, 2021

publicarray linked an issue Jan 31, 2021 that may be closed by this pull request

DSM 7 support framework design #4215

Closed

publicarray added a commit to publicarray/spksrc that referenced this pull request Feb 1, 2021

fix DSM7 running with su command:

e2b2785

SynoCommunity#4395 (comment)

publicarray force-pushed the dsm7 branch from 6a3906e to ca87acb Compare February 1, 2021 09:10

hgy59 mentioned this pull request Feb 2, 2021

Minor error in readme (virtual machine instructions) #4425

Closed

publicarray added a commit to publicarray/spksrc that referenced this pull request Feb 6, 2021

fix DSM7 running with su command:

46a8173

SynoCommunity#4395 (comment)

hgy59 and others added 11 commits February 21, 2021 17:22

regenerated digests of dsm7 toolchains

5f3fab5

Add sha256 digest to evansport toolkit

e1ca6fa

fix TC_BUILD for DSM7.0

c4de2da

Apply new version comparison function

e4b2fb3

Fixes wrong comparisons e.g. where 3.10 was erroneously less than 3.2 Thanks @th0ma7

framework, resource: simplify jq command

dbfde24

framework: code review

94d76fa

Add linking support for /usr/local/lib and /usr/local/etc

d558293

depreciate SPK_LINKS and add new make variable: SPK_USR_LOCAL_LINKS

fix SPK_USR_LOCAL_LINKS

d714eb1

Fix comments being output to command line

2bb13c6

Framework: DSM7 Make sure folder exists before migrating files

0f59242

Framework: Fix unable to override uninstall wizard

9900138

publicarray force-pushed the dsm7 branch from b274362 to 9900138 Compare February 21, 2021 06:33

hgy59 reviewed Feb 21, 2021

View reviewed changes

mk/spksrc.service.start-stop-status Outdated Show resolved Hide resolved

mk/spksrc.spk.mk Outdated Show resolved Hide resolved

publicarray and others added 2 commits February 24, 2021 23:18

Update mk/spksrc.service.start-stop-status

186f1dd

Co-authored-by: hgy59 <hpgy59@gmail.com>

Fraework: avoid creating wizard folder when it's not needed

178bd04

Co-authored-by: hgy59 <hpgy59@gmail.com>

hgy59 approved these changes Mar 11, 2021

View reviewed changes

publicarray merged commit 196483f into master Mar 11, 2021

hgy59 mentioned this pull request Mar 11, 2021

fix builds for DSM<7 without wizard files #4489

Merged

publicarray deleted the dsm7 branch March 14, 2021 00:39

hgy59 mentioned this pull request Mar 14, 2021

fix wizard creation #4496

Merged

This was referenced Apr 4, 2021

Change in behaviour - latest framework snapshot with DSM7 support #4533

Closed

Fishshell: Fix DSM7 build #4494

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DSM 7 Support #4395

Add DSM 7 Support #4395

publicarray commented Jan 17, 2021 •

edited

Loading

hgy59 commented Jan 17, 2021

publicarray commented Jan 17, 2021

hgy59 commented Jan 17, 2021

th0ma7 left a comment

hgy59 commented Jan 20, 2021

publicarray commented Jan 25, 2021

hgy59 commented Jan 27, 2021

th0ma7 commented Feb 6, 2021

publicarray commented Feb 6, 2021

th0ma7 commented Feb 6, 2021

publicarray commented Feb 21, 2021 •

edited

Loading

hgy59 left a comment

publicarray commented Mar 11, 2021

hgy59 left a comment

publicarray commented Mar 11, 2021

th0ma7 commented Mar 11, 2021

Casuallynoted commented Mar 27, 2021

hgy59 commented Mar 27, 2021

Add DSM 7 Support #4395

Add DSM 7 Support #4395

Conversation

publicarray commented Jan 17, 2021 • edited Loading

hgy59 commented Jan 17, 2021

publicarray commented Jan 17, 2021

hgy59 commented Jan 17, 2021

th0ma7 left a comment

Choose a reason for hiding this comment

hgy59 commented Jan 20, 2021

publicarray commented Jan 25, 2021

hgy59 commented Jan 27, 2021

th0ma7 commented Feb 6, 2021

publicarray commented Feb 6, 2021

th0ma7 commented Feb 6, 2021

publicarray commented Feb 21, 2021 • edited Loading

hgy59 left a comment

Choose a reason for hiding this comment

publicarray commented Mar 11, 2021

hgy59 left a comment

Choose a reason for hiding this comment

publicarray commented Mar 11, 2021

th0ma7 commented Mar 11, 2021

Casuallynoted commented Mar 27, 2021

hgy59 commented Mar 27, 2021

publicarray commented Jan 17, 2021 •

edited

Loading

publicarray commented Feb 21, 2021 •

edited

Loading