Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linux_*_hardened: use linux-hardened patch set #84522

Merged
merged 30 commits into from
Apr 19, 2020

Conversation

emilazy
Copy link
Member

@emilazy emilazy commented Apr 6, 2020

Motivation for this change

anthraxx/linux-hardened is an updated version of the dormant AndroidHardeningArchive/linux-hardened repository, which was previously offered in NixOS (as linux_copperhead_*; not sure if it was ever used for linux_hardened).

It adds a relatively conservative grab-bag of hardening patches and defaults on top of the stock Kernel Self Protection Project, and is actively used by the Arch linux-hardened package (as documented on the ArchWiki; see also some ASLR comparison notes from the original author @thestinger, circa 2017). The patch set incorporates many of our hardened Kconfig/sysctl defaults, and releases are timely (Linux 5.5.15 and 5.6.2 were released on 2020-04-02; linux-hardened patches for them came out on 2020-04-03 and 2020-04-04 respectively). It also includes the Debian/Arch patch to allow restricting CLONE_USERNS to privileged users, allowing the Nix sandbox to be re-enabled (see #73763 for discussion).

You can see a list of commits on top of the vanilla 5.6.2 kernel here: anthraxx/linux-hardened@9fbe5c8...5.6.2.a

linux{,Packages}_testing_hardened is removed as the patch set only tracks releases; if anyone actually uses it then it could probably be re-added on a best-effort basis with the 5.6 patches.

I'm running this on my laptop and server, including ZFS (with #84303), Firefox and Wine. (I had to disable the hardened allocator for Firefox, per #73763 (comment), but that's unrelated to these kernel changes.)

paxtest blackhat results for 5.6.2-hardened from this branch:

Using the libc allocator (measuring the kernel ASLR entropy directly)
Executable anonymous mapping             : Killed
Executable bss                           : Killed
Executable data                          : Killed
Executable heap                          : Killed
Executable stack                         : Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed
Executable anonymous mapping (mprotect)  : Vulnerable
Executable bss (mprotect)                : Vulnerable
Executable data (mprotect)               : Vulnerable
Executable heap (mprotect)               : Vulnerable
Executable stack (mprotect)              : Vulnerable
Executable shared library bss (mprotect) : Vulnerable
Executable shared library data (mprotect): Vulnerable
Writable text segments                   : Vulnerable
Anonymous mapping randomization test     : 32 quality bits (guessed)
Heap randomization test (ET_EXEC)        : 26 quality bits (guessed)
Heap randomization test (PIE)            : 40 quality bits (guessed)
Main executable randomization (ET_EXEC)  : No randomization
Main executable randomization (PIE)      : 32 quality bits (guessed)
Shared library randomization test        : 32 quality bits (guessed)
VDSO randomization test                  : 32 quality bits (guessed)
Stack randomization test (SEGMEXEC)      : 40 quality bits (guessed)
Stack randomization test (PAGEEXEC)      : 40 quality bits (guessed)
Arg/env randomization test (SEGMEXEC)    : 44 quality bits (guessed)
Arg/env randomization test (PAGEEXEC)    : 44 quality bits (guessed)
Offset to library randomisation (ET_EXEC): 32 quality bits (guessed)
Offset to library randomisation (ET_DYN) : 34 quality bits (guessed)
Randomization under memory exhaustion @~0: 32 bits (guessed)
Randomization under memory exhaustion @0 : 32 bits (guessed)
Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Killed
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Killed

Improvements over current NixOS:

--- paxtest-linux-5.6.2-hardened-libc-allocator-before.log
+++ paxtest-linux-5.6.2-hardened-libc-allocator-after.log
@@ -17,2 +17,2 @@
-Heap randomization test (ET_EXEC)        : 13 quality bits (guessed)
-Heap randomization test (PIE)            : 32 quality bits (guessed)
+Heap randomization test (ET_EXEC)        : 26 quality bits (guessed)
+Heap randomization test (PIE)            : 40 quality bits (guessed)
@@ -22,5 +22,5 @@
-VDSO randomization test                  : 20 quality bits (guessed)
-Stack randomization test (SEGMEXEC)      : 30 quality bits (guessed)
-Stack randomization test (PAGEEXEC)      : 30 quality bits (guessed)
-Arg/env randomization test (SEGMEXEC)    : 22 quality bits (guessed)
-Arg/env randomization test (PAGEEXEC)    : 22 quality bits (guessed)
+VDSO randomization test                  : 32 quality bits (guessed)
+Stack randomization test (SEGMEXEC)      : 40 quality bits (guessed)
+Stack randomization test (PAGEEXEC)      : 40 quality bits (guessed)
+Arg/env randomization test (SEGMEXEC)    : 44 quality bits (guessed)
+Arg/env randomization test (PAGEEXEC)    : 44 quality bits (guessed)
@@ -28 +28 @@
-Offset to library randomisation (ET_DYN) : 32 quality bits (guessed)
+Offset to library randomisation (ET_DYN) : 34 quality bits (guessed)
Using the hardened profile default scudo allocator
Executable anonymous mapping             : Killed
Executable bss                           : Killed
Executable data                          : Killed
Executable heap                          : Killed
Executable stack                         : Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed
Executable anonymous mapping (mprotect)  : Vulnerable
Executable bss (mprotect)                : Vulnerable
Executable data (mprotect)               : Vulnerable
Executable heap (mprotect)               : Vulnerable
Executable stack (mprotect)              : Vulnerable
Executable shared library bss (mprotect) : Vulnerable
Executable shared library data (mprotect): Vulnerable
Writable text segments                   : Vulnerable
Anonymous mapping randomization test     : 32 quality bits (guessed)
Heap randomization test (ET_EXEC)        : 37 quality bits (guessed)
Heap randomization test (PIE)            : 37 quality bits (guessed)
Main executable randomization (ET_EXEC)  : No randomization
Main executable randomization (PIE)      : 32 quality bits (guessed)
Shared library randomization test        : 32 quality bits (guessed)
VDSO randomization test                  : 32 quality bits (guessed)
Stack randomization test (SEGMEXEC)      : 40 quality bits (guessed)
Stack randomization test (PAGEEXEC)      : 40 quality bits (guessed)
Arg/env randomization test (SEGMEXEC)    : 44 quality bits (guessed)
Arg/env randomization test (PAGEEXEC)    : 44 quality bits (guessed)
Offset to library randomisation (ET_EXEC): 32 quality bits (guessed)
Offset to library randomisation (ET_DYN) : 34 quality bits (guessed)
Randomization under memory exhaustion @~0: 34 bits (guessed)
Randomization under memory exhaustion @0 : 34 bits (guessed)
Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Killed
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Killed

Improvements over current NixOS:

--- paxtest-linux-5.6.2-hardened-scudo-allocator-before.log
+++ paxtest-linux-5.6.2-hardened-scudo-allocator-after.log
@@ -22,5 +22,5 @@
-VDSO randomization test                  : 20 quality bits (guessed)
-Stack randomization test (SEGMEXEC)      : 30 quality bits (guessed)
-Stack randomization test (PAGEEXEC)      : 30 quality bits (guessed)
-Arg/env randomization test (SEGMEXEC)    : 22 quality bits (guessed)
-Arg/env randomization test (PAGEEXEC)    : 22 quality bits (guessed)
+VDSO randomization test                  : 32 quality bits (guessed)
+Stack randomization test (SEGMEXEC)      : 40 quality bits (guessed)
+Stack randomization test (PAGEEXEC)      : 40 quality bits (guessed)
+Arg/env randomization test (SEGMEXEC)    : 44 quality bits (guessed)
+Arg/env randomization test (PAGEEXEC)    : 44 quality bits (guessed)
@@ -28,3 +28,3 @@
-Offset to library randomisation (ET_DYN) : 32 quality bits (guessed)
-Randomization under memory exhaustion @~0: 33 bits (guessed)
-Randomization under memory exhaustion @0 : 33 bits (guessed)
+Offset to library randomisation (ET_DYN) : 34 quality bits (guessed)
+Randomization under memory exhaustion @~0: 34 bits (guessed)
+Randomization under memory exhaustion @0 : 34 bits (guessed)

These mostly match the expected results from ArchWiki. The mprotect "Vulnerable" results are because there's no port of the PaX MPROTECT/NOEXECSTACK features, as these can be implemented with a Mandatory Access Control LSM like SELinux. I'm not sure why ET_EXEC is doing worse; the main executable randomization shouldn't be too relevant as position-independent executables generally use the ET_DYN/PIE mechanism and Arch bug FS#52089 suggests that grsecurity dropped support for this anyway, but the heap randomization results could be a little better according to ArchWiki; it might be that results there were measured with GrapheneOS hardened_malloc, which is recommended elsewhere on the page. In any case, it's a distinct improvement on the status quo.

Note the commit message of b309ef56e8d8; disabling unprivileged CLONE_NEWUSER has security drawbacks for desktop machines due to the impact on the Chromium/Firefox sandboxes. They both have minimal setuid helpers as fallbacks for unprivileged user namespaces, and it'd be nice to arrange for them to get the necessary privileges when CLONE_NEWUSER is restricted, but I haven't addressed that in this PR as the status quo for the hardened profile has already been that user namespaces are silently disabled as a whole.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@emilazy emilazy requested a review from joachifm as a code owner April 6, 2020 23:20
@emilazy emilazy force-pushed the add-linux-hardened-patches branch from cf6273b to 24e0f45 Compare April 6, 2020 23:23
@emilazy
Copy link
Member Author

emilazy commented Apr 6, 2020

@GrahamcOfBorg test hardened latestKernel.hardened

@ofborg ofborg bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: clean-up 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 1001-2500 labels Apr 6, 2020
@lukateras
Copy link
Member

lukateras commented Apr 6, 2020

Tests pass on aarch64-linux after applying the following diff:

diff --git a/pkgs/development/libraries/graphene-hardened-malloc/default.nix b/pkgs/development/libraries/graphene-hardened-malloc/default.nix
index be32a74ea45..5a788a0b7f0 100644
--- a/pkgs/development/libraries/graphene-hardened-malloc/default.nix
+++ b/pkgs/development/libraries/graphene-hardened-malloc/default.nix
@@ -52,6 +52,6 @@ stdenv.mkDerivation rec {
     '';
     license = licenses.mit;
     maintainers = with maintainers; [ ris ];
-    platforms = [ "x86_64-linux" ];
+    platforms = [ "x86_64-linux" "aarch64-linux" ];
   };
 }

kmcopper referenced this pull request Apr 7, 2020
As discussed in #73763, prevailing
consensus is to revert that commit. People use the hardened profile on
machines and run nix builds, and there's no good reason to use
unsandboxed builds at all unless you're in a platform that doesn't
support them.

This reverts commit 00ac71a.
Copy link
Member

@Mic92 Mic92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Looks solid. I also want to get the opinion of @joachifm

In future it would be great to also integrate this kernel into nixpkgs/pkgs/os-specific/linux/kernel/update.sh Otherwise we the hardened kernel might get out-of-date.

@Mic92
Copy link
Member

Mic92 commented Apr 8, 2020

@a13xp0p0v I might send you an updated kernel config to https://github.com/a13xp0p0v/kconfig-hardened-check as soon as this is merged.

@Izorkin
Copy link
Contributor

Izorkin commented Apr 8, 2020

If kernel = 5.6.3 and patch = 5.6.2 error build

nix build -f ./default.nix linux_latest_hardened --show-trace
error: while evaluating the attribute 'drvPath' at /home/user/works/src-nix/nixpkgs/lib/customisation.nix:163:7:
while evaluating the attribute 'configurePhase' of the derivation 'linux-5.6.3' at /home/user/works/src-nix/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:191:11:
while evaluating the attribute 'kernelConfig' of the derivation 'linux-config-5.6.3' at /home/user/works/src-nix/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:191:11:
while evaluating 'kernelConfigFun' at /home/user/works/src-nix/nixpkgs/pkgs/os-specific/linux/kernel/generic.nix:88:21, called from /home/user/works/src-nix/nixpkgs/pkgs/os-specific/linux/kernel/generic.nix:101:20:
while evaluating the attribute 'moduleStructuredConfig.intermediateNixConfig' at /home/user/works/src-nix/nixpkgs/pkgs/os-specific/linux/kernel/generic.nix:151:7:
while evaluating the attribute 'config' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:98:9:
while evaluating 'yieldConfig' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:83:29, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:82:16:
while evaluating 'mergeModules' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:237:26, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:73:17:
while evaluating 'mergeModules'' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:241:36, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:238:5:
while evaluating 'flip' at /home/user/works/src-nix/nixpkgs/lib/trivial.nix:138:16, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:284:6:
while evaluating 'byName' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:264:25, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:272:21:
while evaluating 'reverseList' at /home/user/works/src-nix/nixpkgs/lib/lists.nix:393:17, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:73:38:
while evaluating anonymous function at /home/user/works/src-nix/nixpkgs/lib/modules.nix:171:37, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:68:19:
while evaluating 'filterModules' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:161:36, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:172:7:
while evaluating anonymous function at /home/user/works/src-nix/nixpkgs/lib/modules.nix:165:31, called from undefined position:
while evaluating 'loadModule' at /home/user/works/src-nix/nixpkgs/lib/modules.nix:110:53, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:148:22:
while evaluating 'isFunction' at /home/user/works/src-nix/nixpkgs/lib/trivial.nix:333:16, called from /home/user/works/src-nix/nixpkgs/lib/modules.nix:111:12:
while evaluating the attribute 'hardened."5.6.3"' at /home/user/works/src-nix/nixpkgs/pkgs/os-specific/linux/kernel/patches.nix:41:3:
attribute '5.6.3' missing, at /home/user/works/src-nix/nixpkgs/pkgs/top-level/all-packages.nix:16946:7

@emilazy emilazy force-pushed the add-linux-hardened-patches branch from 24e0f45 to 11f3bff Compare April 8, 2020 20:07
@emilazy
Copy link
Member Author

emilazy commented Apr 8, 2020

Pushed to fix conflicts, unbreak the test on AArch64, and update for newer kernels.

I'd definitely like to integrate automatic updates into this (and my other packages); is there an existing update script that interfaces with the GitHub releases API that I could reference?

FWIW, I have plans to improve the Kconfig hardening further insofar as that can be done without jeopardizing general server/desktop use, but wanted to keep this PR relatively standalone.

@GrahamcOfBorg test hardened latestKernel.hardened

@emilazy emilazy force-pushed the add-linux-hardened-patches branch 2 times, most recently from dbb8f31 to 8bd37be Compare April 9, 2020 02:48
@emilazy
Copy link
Member Author

emilazy commented Apr 9, 2020

Now with fancy over-engineered Python update script~

@emilazy emilazy force-pushed the add-linux-hardened-patches branch 3 times, most recently from caf765f to daffeaf Compare April 9, 2020 02:52
@emilazy
Copy link
Member Author

emilazy commented Apr 9, 2020

@GrahamcOfBorg test hardened latestKernel.hardened

@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Apr 9, 2020
@ofborg ofborg bot requested a review from risicle April 9, 2020 03:00
@emilazy
Copy link
Member Author

emilazy commented Apr 9, 2020

FWIW, since keyservers are a trash fire and the PGP key links on the Arch site are consequently broken: anthraxx.asc is from https://leventepolyak.net/anthraxx.asc via https://leventepolyak.net/#gnupg.

You can reproduce hardened-patches.json by doing:

$ echo -n {} > hardened-patches.json
$ ./update-hardened.py

linux-hardened sets kernel.unprivileged_userns_clone=0 by default; see
anthraxx/linux-hardened@104f440.

This allows the Nix sandbox to function while reducing the attack
surface posed by user namespaces, which allow unprivileged code to
exercise lots of root-only code paths and have lead to privilege
escalation vulnerabilities in the past.

We can safely leave user namespaces on for privileged users, as root
already has root privileges, but if you're not running builds on your
machine and really want to minimize the kernel attack surface then you
can set security.allowUserNamespaces to false.

Note that Chrome's sandbox requires either unprivileged CLONE_NEWUSER or
setuid, and Firefox's silently reduces the security level if it isn't
allowed (see about:support), so desktop users may want to set:

    boot.kernel.sysctl."kernel.unprivileged_userns_clone" = true;
Seems like a good idea to ensure that you can always use the latest
stable upstream kernel.
These now depend on an external patch set; add them to the release tests
to ensure that the build doesn't break silently as new kernel updates
are merged.
@lukateras
Copy link
Member

@GrahamcOfBorg test hardened latestKernel.hardened

@lukateras
Copy link
Member

I would like to merge this tomorrow at 6 pm UTC unless anyone has objections.

@lukateras lukateras merged commit 6f1165a into NixOS:master Apr 19, 2020
@dasJ
Copy link
Member

dasJ commented Apr 19, 2020

Well this fails our Hydra with:

hydra-eval-jobs returned exit code 1:
error: aggregate job 'tested' references non-existent job 'nixos.tests.latestKernel.login.x86_64-linux'

@eadwu
Copy link
Member

eadwu commented Apr 19, 2020

Seems like the test was broken to begin with as it was already there before this PR.

@emilazy
Copy link
Member Author

emilazy commented Apr 19, 2020

Oof. Doesn't seem directly related to the patched kernel, at least, but this PR did add it to the release channel tests.

Edit: It seems like the test passes fine on nixpkgs master, and the error is about it not being present on your local Hydra. I don't use Hydra so unfortunately I'm not sure what the problem might be; maybe jobs for the tests have to be manually set up?

@dasJ
Copy link
Member

dasJ commented Apr 19, 2020

Wha… surely seems to work. Sorry for bothering you

@dasJ
Copy link
Member

dasJ commented Apr 19, 2020

Nvm, upstream Hydra is affected as well: https://hydra.nixos.org/jobset/nixos/unstable-small#tabs-errors

@emilazy emilazy deleted the add-linux-hardened-patches branch April 19, 2020 22:56
@Izorkin
Copy link
Contributor

Izorkin commented Apr 20, 2020

With this kernel, nginx started with this warning:

nginx[675]: 2020/04/20 07:51:14 [emerg] 718#718: io_setup() failed (38: Function not implemented)

@Izorkin
Copy link
Contributor

Izorkin commented Apr 20, 2020

zcat /proc/config.gz | grep AIO

# CONFIG_AIO is not set
CONFIG_DELL_WMI_AIO=m

It is possible to activate CONFIG_AIO?

@eadwu
Copy link
Member

eadwu commented Apr 20, 2020

You'll have to manually patch it (i.e. with a kernelPatch), CONFIG_AIO is disabled on hardened.

@Mic92
Copy link
Member

Mic92 commented Apr 20, 2020

It is possible to disable aio in nginx as far as I know.

@jonringer
Copy link
Contributor

This seems to be breaking ofborg now, these patches need to be updated along with the kernel bumps

example failure: https://gist.github.com/GrahamcOfBorg/dc0c1bbb7502ad569c8bc49213949f73

@NeQuissimus

@8573
Copy link
Contributor

8573 commented Jun 17, 2020

... disabling unprivileged CLONE_NEWUSER has security drawbacks for desktop machines due to the impact on the Chromium/Firefox sandboxes. They both have minimal setuid helpers as fallbacks for unprivileged user namespaces, ...

I don't know about Firefox, but I find that, using the SUID sandbox, Chromium's internal sandbox status page reports all sandboxing features as operational, which is one more than it reports as operational when using the unprivileged-user-namespace sandbox. (The feature whose status differs is "Ptrace Protection with Yama LSM (Non-broker)".)

(Edit: Also, thanks! I hadn't realized we were running that linux-hardened now! Back when there was linux_copperhead_*, I would consider switching to it but would worry that too much would break; now the switch happened "in the background" and I didn't notice anything go wrong.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: clean-up 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 1001-2500
Projects
None yet
Development

Successfully merging this pull request may close these issues.