Let’s start out with a simple project with only registry dependencies and no
install scripts in the dependency graph.
Our project will be a simple Typescript CLI tool.
This means we will have a build stage, a global
installation target, and a dist target ( a tarball suitable for publishing ).
I want to highlight the fact that our dependency graph only contains trivial packages that do not require installations - we will cover handling these dependencies in another guide. For now we’re keeping things simple.
Please note that the boilerplate default.nix, foverrides.nix, and a related
file floco-cfg.nix are provided in the
nix flake init -t github:aakropotkin/floco; template.
The template is functionally the same as this examples’ files except that
portions of default.nix are moved to a general purpose file floco-cfg.nix
( our trivial example doesn’t have any use for this separation ).
In the spirit of completeness, this guide uses real sources so that we can automatically detect if it’s out of date and so that you can copy these files to try them on your box.
This executable will be trivial, we’ll just print a version string for
lodash, but we’ll gate the output behind a few typed expressions to prove
that they work.
// index.ts
import {
VERSION, flatten, isEqual, last, omit, pick, pickBy, uniq
} from 'lodash';
const f : Array<number> = flatten( [[1, 2], [3, 4]] );
console.log( f );
const l : number = last( [1, 2, 3, 4] )
console.log( l );
const u : Array<number> = uniq( [1, 2, 3, 1, 2, 3, 2] );
console.log( u );
console.log( VERSION );A JavaScript CLI wrapper that requires the built form of our TypeScript shit. Notably we won’t set executable permissions on this just to prove that the installer handles that for us.
#! /usr/bin/env node
// bin.js
require( './index.js' );The package.json, nothing special here.
The only note I’ll make is that the script names build and prepublish are
treated as synonyms, and that prebuild and postbuild are also recognized
but preprepublish and postprepublish are not because their names are
completely idiotic and they aren’t commonly used so I ain’t supporting them.
{
"name": "@floco/test",
"version": "4.2.0",
"bin": {
"test": "./bin.js"
},
"scripts": {
"clean": "rm -rf ./node_modules ./index.js",
"build": "tsc ./index.ts"
},
"devDependencies": {
"@types/lodash": "^4.14.191",
"typescript": "^4.9.4"
},
"dependencies": {
"lodash": "^4.17.21"
}
}Just for kicks we can ensure this project works using npm by running:
$ nix shell nixpkgs#nodejs nixpkgs#nodejs.pkgs.npm;
$ npm install --lockfile-version=3;
$ npm run build;
$ node ./bin.js
[ 1, 2, 3, 4 ]
4
[ 1, 2, 3 ]
4.17.21
$ npm run clean;
$ rm ./package-lock.json; # OptionalTo Nixify our project we’ll generate a file called pdefs.nix from our
existing package-lock.json.
Once generated the package-lock.json file can be deleted allowing floco
and nix to fly solo.
After that we’ll add a default.nix file to expose a few derivations to the
nix CLI.
# default.nix
# ============================================================================ #
#
# Package shim exposing installable targets from `floco' modules.
#
# ---------------------------------------------------------------------------- #
{ floco ? builtins.getFlake "github:aakropotkin/floco"
, lib ? floco.lib
, system ? builtins.currentSystem
}: let
# ---------------------------------------------------------------------------- #
fmod = lib.evalModules {
modules = [
floco.nixosModules.floco
{ config.floco.settings = { inherit system; basedir = ./.; }; }
# Loads our generated `pdefs.nix' as a "module config".
./pdefs.nix
];
};
# ---------------------------------------------------------------------------- #
# This attrset holds a few derivations related to our package.
# We'll expose these below to the CLI.
pkg = fmod.config.floco.packages."@floco/test"."4.2.0";
# ---------------------------------------------------------------------------- #
in {
inherit (pkg)
dist # A tarball form of our built package suitable for publishing
prepared # The "prepared" form of our project for use by other Nix builds
global # A globally installed form to run our executable
;
built = pkg.built.packages; # Our project in it's "built" state
}
# ---------------------------------------------------------------------------- #Lets generate pdefs.nix and take this bad boy for a spin:
$ nix run github:aakropotkin/floco#fromPlock;
$ rm *~||:; # Delete any backup files that might've been created
# Run our executable from the `global' target.
# We add the flag `-L' to show build logs.
# If this is your first time building with `floco' this may take a minute to
# initialize your box's cache, but successive builds will fly.
$ nix run -f ./. -L global;
...
test-built> unpacking sources
test-built> unpacking source archive /nix/store/4xna8iwywa57wrv8j64p4cimhy819sq3-basic
test-built> source root is basic
test-built> patching sources
test-built> configuring
test-built> building
test-built> installing
test-built> post-installation fixup
test-built> shrinking RPATHs of ELF executables and libraries in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0
test-built> patching script interpreter paths in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0
test-built> /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0/bin.js: interpreter directive changed from "#! /usr/bin/env node" to "/nix/store/mwd1dxh5rcy0wi9vgv2brlxpr5gmngr7-nodejs-14.20.1/bin/node"
test-built> checking for references to /build/ in /nix/store/51dibgxp3na6q21p50slmfw02ql3cqn0-test-built-4.2.0...
# If we run again you'll see we skip the build:
$ nix run -f ./. -L global;
# Lets build our tarball:
$ nix build -f ./. dist;
$ tar tzf ./result;
package/bin.js
package/index.js
package/package.json
package/default.nix
package/pdefs.nix
package/index.tsPretty slick.
Right off the bat you might be asking: how is this any different from npm,
aside from the fact that I had to write extra files and read a guide?
It’s a fair question, and in the next few sections we’ll try to win you over.
Upfront let’s just say that there isn’t a practical reason to optimize this trivial package; but as an exercise let’s just treat it as a playground to show techniques that can be used out in the field where it really matters.
The largest opportunity to speed up most builds is by treating CLI tools as
“global” dependencies.
The reason this speeds up builds is that rather than copying the contents
of these dependencies into the build areas we can instead add them
to PATH.
Doing so allows us to avoid copying the entire dependency closure - not
just the target package.
As an added bonus this tends to simplify ideal tree formation.
In a tool like npm this is like doing npm i -g foo except that in the
case of floco we actually have the ability to declare these in a
standardized way.
With our example project typescript can be handled this way.
To mark typescript as a globally installed dependency we will delete it
from a fragment of our config metadata named
treeInfo,
and then move it to the buildInputs field of our built target.
We could accomplish this same goal with other types of config settings,
which we might prefer for projects that we regenerate frequently; but this
is the simplest approach.
Pop open the pdefs.nix file and we’ll drop typescript.
# `pdefs.nix'
{
floco = {
pdefs = {
"@floco/test" = {
"4.2.0" = {
ident = "@floco/test";
version = "4.2.0";
# ...
treeInfo = {
"@types/lodash" = {
key = "@types/lodash/4.14.191";
dev = true;
};
"lodash" = {
key = "lodash/4.17.21";
};
# We're removing `typescript':
## typescript = {
## key = "typescript/4.9.4";
## dev = true;
## };
};
};
};
# ...
};
};
}Next we’ll make a new file called foverrides.nix to get the global form
of the package added to the sandbox.
The seperation between these files is somewhat arbitrary but we’ll revist
that later in a discussion about project organization.
# `foverrides.nix'
{ config, ... }: {
config.floco.packages."@floco/test"."4.2.0".built.extraBuildInputs = [
config.floco.packages."typescript"."4.9.4".global
];
}This file allows you to explicitly fill config values by hand.
Separation from the default.nix file allows it to be used
“anywhere” especially external projects.
To use an override file in our project we just need to add it to
our list of modules in default.nix ( order doesn’t matter among
list members ).
# default.nix
# ---------------------------------------------------------------------------- #
{
# ...
fmod = lib.evalModules {
modules = [
floco.nixosModules.floco
{ config.floco.settings = { inherit system; basedir = ./.; }; }
./pdefs.nix
# Add an override file
./foverrides.nix
];
};
}
# ---------------------------------------------------------------------------- #Now if you rebuild you won’t have to copy typescript, instead you will
just add its executables to PATH.
This same approach can be used for any type of config setting.
In later guides we’ll cover a few more common customizations like cleaning
local source trees, activating symlinked node_modules/ dirs, adding
tests, custom build hooks, and more.