Ob deploy push fails with exit code 1

[SebastianCallh]

After having installed Obelisk on Ubuntu 16.04 by following the instructions in the readme (installing Obelisk in the nix-env, and using hub) and trying to deploy the example app(also following the readme) the command ob deploy push fails with the following error message.

✔ Running nix-build on ./.obelisk/impl [command]
RawCommand "nix-build" ["./src","-A","server","--argstr","hostName","ec2-xx-xxxx-xx-xxxx.eu-central-1.compute.amazonaws.com","--no-out-link"] failed with exit code: 1
✖ Running nix-build on ./src [server]

The content of the ./src folder in the myapp-deploy have not been edited by hand and consists of the default.nix file

# DO NOT HAND-EDIT THIS FILE
import ((import <nixpkgs> {}).fetchFromGitHub (
  let json = builtins.fromJSON (builtins.readFile ./github.json);
  in { inherit (json) owner repo rev sha256;
       private = json.private or false;
     }
))

the github.json file

{
  "owner": "...",
  "repo": "...",
  "branch": "...",
  "private": ...,
  "rev": "...",
  "sha256": "..."
}

and the .attr-cache folder which is empty.

[srid]

@SebastianCallh Could you show the output of ob deploy push -v?

[SebastianCallh]

Hi @srid! Great idea, apparently it tries to fetch from an url that gives 404.

Sorry about the formatting, the output is below

Starting Obelisk </nix/store/s200msjpipi4jpyv2xkas35dsb101s9m-obelisk-command-0.1/bin/ob> args=Args {_args_noHandOffPassed = False, _args_verbose = True, _args_command = ObCommand_Deploy DeployCommand_Push} noSpinner=False logging-level=Debug
Creating process: CreateProcess {cmdspec = RawCommand "nix-build" ["./.obelisk/impl","-A","command","--no-out-link"], cwd = Nothing, env = Nothing, std_in = Inherit, std_out = CreatePipe, std_err = CreatePipe, close_fds = False, create_group = False, delegate_ctlc = False, detach_console = False, create_new_console = False, new_session = False, child_group = Nothing, child_user = Nothing}
✔ Running nix-build on ./.obelisk/impl [command]
Handing off to /nix/store/s200msjpipi4jpyv2xkas35dsb101s9m-obelisk-command-0.1/bin/ob
Starting Obelisk </nix/store/s200msjpipi4jpyv2xkas35dsb101s9m-obelisk-command-0.1/bin/ob> args=Args {_args_noHandOffPassed = True, _args_verbose = True, _args_command = ObCommand_Deploy DeployCommand_Push} noSpinner=False logging-level=Debug
Creating process: CreateProcess {cmdspec = RawCommand "nix-build" ["./src","-A","server","--argstr","hostName","ec2-18-184-54-251.eu-central-1.compute.amazonaws.com","--no-out-link"], cwd = Nothing, env = Nothing, std_in = Inherit, std_out = CreatePipe, std_err = CreatePipe, close_fds = False, create_group = False, delegate_ctlc = False, detach_console = False, create_new_console = False, new_session = False, child_group = Nothing, child_user = Nothing}
building '/nix/store/1qnvkmviagcd6sg9rl3kzsjyhx2l2ffj-source.drv'...

trying https://github.com/obsidiansystems/obelisk-https/archive/5bc5cd81b669a0ea1b4415a1be8fd8c5a0680f0b.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download source from any mirror
builder for '/nix/store/1qnvkmviagcd6sg9rl3kzsjyhx2l2ffj-source.drv' failed with exit code 1
error: build of '/nix/store/1qnvkmviagcd6sg9rl3kzsjyhx2l2ffj-source.drv' failed
(use '--show-trace' to show detailed location information)
RawCommand "nix-build" ["./src","-A","server","--argstr","hostName","ec2-18-184-54-251.eu-central-1.compute.amazonaws.com","--no-out-link"] failed with exit code: 1
✖ Running nix-build on ./src [server]

[luigy]

@SebastianCallh just made that package public which should resolve the 404

[SebastianCallh]

@luigy Yeah it ran past that step now but after a while it got stuck on a permission denied. This shouldn't need sudo, right? I don't really understand what it's doing in the /boot folder.

depmod: FATAL: could not load /boot/System.map-4.4.0-127-generic: Permission denied
make: *** [Makefile:108: install] Error 1
builder for '/nix/store/aik0k9s8a8dgkppfj561d4y6bsrha96a-ixgbevf-4.2.1-4.9.50.drv' failed with exit code 2
cannot build derivation '/nix/store/2r30zinrmpm64xrdawxzb5r61dr38zdg-kernel-modules.drv': 1 dependencies couldn't be built
building '/nix/store/cf2sdd99xdiima8xn5rafg17hazc5b8g-linux-4.9.50-shrunk.drv'... 
cannot build derivation '/nix/store/5w6f1gsz9c2sypnavmfrzgmw1gkzclah-nixos-system-ec2-xx-xxx-xx-xxx.eu-central-1.compute.amazonaws.com-18.03pre-git.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/72w96dcgdpnjqn1i29adyq6z8nscq2bh-nixos-system-    ec2-xx-xxx-xx-xxx.eu-central-1.compute.amazonaws.com-18.03pre-git.drv': 1 dependencies couldn't   be built
error: build of '/nix/store/72w96dcgdpnjqn1i29adyq6z8nscq2bh-nixos-system-ec2-xx-xxx-xx-xxx.eu-   central-1.compute.amazonaws.com-18.03pre-git.drv', '/nix/store/aj2xkjxj1fkrl8fbas590c6bsnh4ra1w-nixos-vm.drv', '/nix/store/wz9csp4xcyzkrisg0kzc1r385hhz06x1-nixos-vm.drv' failed
RawCommand "nix-build" ["./src","-A","server","--argstr","hostName","ec2-xx-xxx-xx-xxx.eu-central-  1.compute.amazonaws.com","--no-out-link"] failed with exit code: 100
✖ Running nix-build on ./src [server]

[3noch]

@SebastianCallh Thanks for pressing through. I sadly have never seen this before. What system are you running on? Ubuntu?

[SebastianCallh]

@3noch No problem! That's right, Ubuntu 16.04. I have put the entire output from ob deploy push -v in this paste bin.

[SebastianCallh]

@3noch I see now that the instance is in fact running NixOS 18.03 and not NixOS 17.09. Whoups. That could be it? I will try with an instance with the proper version later today and get back to you.

[SebastianCallh]

@3noch That was not the issue. The error remains.

[alexfmpe]

Looks like NixOS/nixpkgs#29448

[SebastianCallh]

@alexfmpe You're totally right. How do we proceed with this?

[3noch]

@SebastianCallh Looks like that issue has a workaround in the comments (but it's ugly). This is a nasty bug with nixpkgs on Ubuntu. I have another possible workaround coming that uses docker to run the actual deployment builds. We need this for macOS but I think it could probably be a workaround for this issue as well.

[SebastianCallh]

@3noch Yeah I saw it. Unfortunately I don't trust myself enough to restore the boot folder after every build. This is quite the bummer

[3noch]

@SebastianCallh It may only need to be done once (populate the cache and be done) but I don't know for certain. I wouldn't want to do that myself either... Let's hope that my virtualized remote builder helps here until nixpkgs gets things together.

[SebastianCallh]

@3noch That's a possibility but I'd rather not. Looking forward to it! Shoot me a message if I can assist with testing or anything.

[ryantrinkle]

@3noch Yeah, that workaround is quite a bit more dangerous than people can reasonably do. Can we try to sort this out upstream?

…

On Sat, May 26, 2018, 07:11 Sebastian Callh ***@***.***> wrote: @3noch <https://github.com/3noch> That's a possibility but I'd rather not. Looking forward to it! Shoot me a message if I can assist with testing or anything. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#6 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABGlYHMyiegglavRrEQlgRStU9W6tg1gks5t2HOagaJpZM4UMz5l> .

[ryantrinkle]

@SebastianCallh Can you check whether /etc/nix/nix.conf says sandbox = true anywhere inside it? It looks like the builder for one of the libraries is checking things it shouldn't; I'm hoping that if we turn on sandboxing, it won't be able to do that anymore.

[SebastianCallh]

@ryantrinkle It did not have sandbox = true in it. After adding it and rerunning ob deploy push the entire process completed and the example app was deployed successfully!

Are there any consequences to be aware of when running in sandbox mode?

[ryantrinkle]

@SebastianCallh Awesome! I'm glad that fixed it; nix stuff really should not be reading outside of its own area.

The consequences of using sandboxing are mostly that it is slightly slower (on the order of 30ms per derivation, if I recall correctly), and that it is more robust. On NixOS, sandboxing is on by default. On some OSes, creating/entering a sandbox may require elevated permissions, but that doesn't seem to have been the case for you.

I've created an issue to help with this: #28

[3noch]

@ryantrinkle Awesome! I haven't used sandbox = true before. I'm very glad there is a clean solution to this.