Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mastodon: 4.2.13 -> 4.3.0 #337545

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

Izorkin
Copy link
Contributor

@Izorkin Izorkin commented Aug 26, 2024

Description of changes

Updata package to version 4.3.0.
Changelog: https://github.com/mastodon/mastodon/releases/tag/v4.3.0-beta.1

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Copy link
Member

@erictapen erictapen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far. The first commit is missing a link to the changelog (please put that also in the PR description) and the two commits would need to be squashed, as it wouldn't make sense to apply/revert them individually, afait.

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved
pkgs/servers/mastodon/update.sh Outdated Show resolved Hide resolved
@erictapen
Copy link
Member

Will close #277697

Comment on lines 716 to 726
bin/rails db:encryption:init | grep --only-matching "ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=[^ ]\+" | sed 's/^ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=//' > /var/lib/mastodon/secrets/active-record-encryption-deterministic-key
fi

if ! test -f /var/lib/mastodon/secrets/active-record-encryption-key-derivation-salt; then
mkdir -p /var/lib/mastodon/secrets
bin/rails db:encryption:init | grep --only-matching "ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=[^ ]\+" | sed 's/^ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=//' > /var/lib/mastodon/secrets/active-record-encryption-key-derivation-salt
fi

if ! test -f /var/lib/mastodon/secrets/active-record-encryption-primary-key; then
mkdir -p /var/lib/mastodon/secrets
bin/rails db:encryption:init | grep --only-matching "ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=[^ ]\+" | sed 's/^ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=//' > /var/lib/mastodon/secrets/active-record-encryption-deterministic-key
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safe to run this as three separate invocations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the keys are independent of each other.
I haven't finished this piece of code yet.

@Izorkin
Copy link
Contributor Author

Izorkin commented Aug 27, 2024

The precompiled bootsnap code requires write access, me need to do it differently:

strace[2341]: openat(AT_FDCWD, "/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap/compile-cache-iseq/2c/e8c63056794755.tmp.4RYrhX", O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EROFS (Read-only file system)
strace[2341]: mkdir("/nix", 0775)                     = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store", 0775)               = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1", 0775) = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile", 0775) = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap", 0775) = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap/compile-cache-iseq", 0775) = -1 EEXIST (File exists)
strace[2341]: mkdir("/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap/compile-cache-iseq/2c", 0775) = -1 EEXIST (File exists)
strace[2341]: clock_gettime(CLOCK_MONOTONIC, {tv_sec=478, tv_nsec=979962067}) = 0
strace[2341]: openat(AT_FDCWD, "/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap/compile-cache-iseq/2c/e8c63056794755.tmp.0rUwZL", O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EROFS (Read-only file system)
strace[2341]: clock_gettime(CLOCK_MONOTONIC, {tv_sec=478, tv_nsec=980354575}) = 0
strace[2341]: openat(AT_FDCWD, "/nix/store/36hx44ri59l0cz2p9mwz76fnjm24cnxg-mastodon-4.3.0-beta.1/.precompile/bootsnap/compile-cache-iseq/2c/e8c63056794755.tmp.ouEjmu", O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EROFS (Read-only file system)
strace[2341]: close(5)                                = 0
strace[2341]: openat(AT_FDCWD, "/nix/store/frq2xac60pwdcyy7av1qlvllpcn271j6-ruby3.2-propshaft-0.9.1/lib/ruby/gems/3.2.0/gems/propshaft-0.9.1/lib/propshaft/compilers.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = 5
strace[2341]: fstat(5, {st_mode=S_IFREG|0444, st_size=1010, ...}) = 0
strace[2341]: close(5)      

@erictapen
Copy link
Member

How about we set BOOTSNAP_READONLY then? Or just disable bootsnap alltogether using BOOTSNAP_DISABLE.

@Izorkin
Copy link
Contributor Author

Izorkin commented Aug 27, 2024

How about we set BOOTSNAP_READONLY then? Or just disable bootsnap alltogether using BOOTSNAP_DISABLE.

Moved to /var/cache/mastodon/precompile

@Izorkin Izorkin force-pushed the update-mastodon branch 2 times, most recently from ccf4b4e to 240d739 Compare August 30, 2024 17:20
@Izorkin
Copy link
Contributor Author

Izorkin commented Aug 30, 2024

Everything seems to be working :)

@Izorkin
Copy link
Contributor Author

Izorkin commented Sep 30, 2024

Update to 4.3.0-rc.1.

@Izorkin Izorkin changed the title mastodon: 4.2.12 -> 4.3.0 mastodon: 4.2.13 -> 4.3.0 Sep 30, 2024
Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a bigger topic for another PR but can't we put all the secrets into one file read by EnviromentFile?

hash = yarnHash;
};

nativeBuildInputs = [ fixup-yarn-lock nodejs-slim yarn mastodonGems mastodonGems.wrappedRuby brotli ];
nativeBuildInputs = [ nodejs-slim yarn-berry mastodonGems mastodonGems.wrappedRuby brotli python3 ];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need such an old python version? Maybe distutils? Can we add it in an env instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If use the latest versions of Pythons a build error occurs:

➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info it worked if it ends with ok
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info using node-gyp@10.0.1
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info using node@20.17.0 | linux | x64
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info find Python using Python version 3.12.5 found at "/nix/store/h3i0acpmr8mrjx07519xxmidv8mpax4y-python3-3.12.5/bin/python3"
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn /nix/store/h3i0acpmr8mrjx07519xxmidv8mpax4y-python3-3.12.5/bin/python3
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args [
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '/build/source/node_modules/node-gyp/gyp/gyp_main.py',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args 'binding.gyp',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-f',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args 'make',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-I',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '/build/source/node_modules/utf-8-validate/build/config.gypi',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-I',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '/build/source/node_modules/node-gyp/addon.gypi',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-I',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '/nix/store/a3rj28n8mblqs26nrn7wzl4m8hda8rkk-nodejs-slim-22.8.0/include/node/common.gypi',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dlibrary=shared_library',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dvisibility=default',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dnode_root_dir=/nix/store/a3rj28n8mblqs26nrn7wzl4m8hda8rkk-nodejs-slim-22.8.0',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dnode_gyp_dir=/build/source/node_modules/node-gyp',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dnode_lib_file=/nix/store/a3rj28n8mblqs26nrn7wzl4m8hda8rkk-nodejs-slim-22.8.0/$(Configuration)/node.lib',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dmodule_root_dir=/build/source/node_modules/utf-8-validate',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Dnode_engine=v8',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '--depth=.',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '--no-parallel',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '--generator-output',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args 'build',
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args '-Goutput_dir=.'
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args ]
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR <string>:17: SyntaxWarning: invalid escape sequence '\.'
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn make
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT make: Entering directory '/build/source/node_modules/utf-8-validate/build'
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT   CXX(target) Release/obj.target/validation/src/validation.o
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT   CXX(target) Release/obj.target/validation/deps/is_utf8/src/is_utf8.o
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT   SOLINK_MODULE(target) Release/obj.target/validation.node
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT   COPY Release/validation.node
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDOUT make: Leaving directory '/build/source/node_modules/utf-8-validate/build'
➤ YN0000: │ utf-8-validate@npm:6.0.4 STDERR gyp info ok

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see an error in the logs and locally it build also fine. Am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be seen with the additional parameter --inline-builds.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try adding setuptools. That often fixed issues with pre-gyp and the like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setuptools package does not resolve the error.

@Izorkin
Copy link
Contributor Author

Izorkin commented Oct 8, 2024

Probably a bigger topic for another PR but can't we put all the secrets into one file read by EnviromentFile?

I think the current option is easier to handle with scripts.

@Izorkin Izorkin force-pushed the update-mastodon branch 2 times, most recently from 269b1a5 to 05f12f0 Compare October 8, 2024 21:01
@Izorkin Izorkin marked this pull request as ready for review October 8, 2024 21:18
@infinisil

This comment was marked as resolved.

hash,
}:
stdenvNoCC.mkDerivation {
name = "yarn-deps";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name = "yarn-deps";
pname = "yarn-deps";
inherit version;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error:

       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:34:12:
           33|
           34|   strict = derivationStrict drvAttrs;
             |            ^
           35|

       … while evaluating derivation 'mastodon-4.3.0'
         whose name attribute is located at /home/lafiel/works/src-nix/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:336:7

       … while evaluating attribute 'mastodonModules' of derivation 'mastodon-4.3.0'
         at /home/user/works/src-nix/nixpkgs/pkgs/servers/mastodon/default.nix:26:3:
           25|
           26|   mastodonModules = stdenv.mkDerivation {
             |   ^
           27|     pname = "${pname}-modules";

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: undefined variable 'version'
       at /home/user/works/src-nix/nixpkgs/pkgs/servers/mastodon/yarn.nix:10:11:
            9|   pname = "yarn-deps";
           10|   inherit version;
             |           ^
           11|   nativeBuildInputs = [

@SuperSandro2000

This comment was marked as off-topic.

Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise I deployed this to my instance without any problems other than the yarn-berry issue mentioned above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants