feat: zinniad for Filecoin Station

[bajtos]

Implement a new binary zinniad that can run multiple modules at the same time and acts as another Filecoin Station Module.

See #75

Out of Scope

• zinniad: validate the wallet address #142
• Graceful shutdown #143
• zinniad: run multiple modules #144
• zinniad: report metrics for observability #145
• zinniad: handle module exit #146
• zinniad: module name & version #147
• zinniad: persist job counter across restarts #148

Usage

❯ zinniad --help
Zinnia daemon runs Zinnia modules inside Filecoin Station.

Usage: zinniad [OPTIONS] --wallet-address <WALLET_ADDRESS> [FILES]...

Arguments:
  [FILES]...  List of modules to run, where each module is a single JS file. We don't make any assumptions about the directory layout of modules. Paths are resolved relatively to the current working directory

Options:
  -w, --wallet-address <WALLET_ADDRESS>
          Address of Station's built-in wallet (required) [env: WALLET_ADDRESS=]
  -r, --root-dir <ROOT_DIR>
          Directory where to keep state files (optional). Defaults to a platform-specific location, e.g. $XDG_STATE_HOME/zinniad on Linux [env: ROOT_DIR=] [default: "/Users/bajtos/Library/Application Support/app.filstation.zinniad"]
  -h, --help
          Print help
  -V, --version
          Print version

In Action

Raw stdout (ND-JSON):

❯ cargo run -p zinniad -- -w f1asd ping-probe.js 2>/dev/null
{"type":"activity:info","module":null,"message":"Module Runtime started."}
{"type":"jobs-completed","total":1,"modules":{"ping-probe":1}}
{"type":"jobs-completed","total":2,"modules":{"ping-probe":2}}

Raw stderr - notice that Console logs are printed via Rust logging API, annotated with module name and timestamp:

❯ NO_COLOR=1 cargo run -p zinniad -- -w f1asd ping-probe.js >/dev/null
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `target/debug/zinniad -w f1asd ping-probe.js`
[2023-03-27T15:50:18Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "f1asd", root_dir: "/Users/bajtos/Library/Application Support/app.filstation.zinniad", files: ["ping-probe.js"] }
[2023-03-27T15:50:18Z INFO  zinniad] Starting module file:///Users/bajtos/src/zinnia/zinnia/ping-probe.js
[2023-03-27T15:50:18Z INFO  module:ping-probe] Pinging /dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb
[2023-03-27T15:50:19Z INFO  module:ping-probe] RTT: 518ms
[2023-03-27T15:50:19Z INFO  module:ping-probe] Submitted stats to InfluxDB.
[2023-03-27T15:50:20Z INFO  module:ping-probe] Pinging /dns/saturn-link-poc.fly.dev/tcp/3030/p2p/12D3KooWRH71QRJe5vrMp6zZXoH4K7z5MDSWwTXXPriG9dK8HQXk
[2023-03-27T15:50:20Z INFO  module:ping-probe] RTT: 538ms
[2023-03-27T15:50:21Z INFO  module:ping-probe] Submitted stats to InfluxDB.

[juliangruber]

Suggested change

	self.tracker
	.borrow_mut()
	.job_completed(|n| self.print_jobs_completed(n));
	self.print_jobs_completed(self.tracker
	.borrow_mut()
	.job_completed());

API question: Is a closure the way to go, or could we simply return the value?

[bajtos]

So I want to log the message only if it's at least 500ms since we printed the message for the last time.

I suppose I could change job_completed() and flush() function to return a tuple (job_count: u64, should_log: bool) and let the caller handle should_log.

let (job_count, should_log) = self.tracker.borrow_mut().job_completed();
if should_log { self.print_jobs_completed(job_count); }

Do you find this version easier to understand and work with?

[bajtos]

So I want to log the message only if it's at least 500ms since we printed the message for the last time.

Meaning that if we call reporter.job_completed twice within 500ms, the first call will trigger calling the printer closure but the second call will not.

[bajtos]

Well, returning (job_count, should_log) would not work well. The tracker will not know whether it should reset the timer or not. It can reset the timer every time job_completed returns should_log = true, but that relies on the caller logging the message when told so. I find that brittle.

[juliangruber]

Ah gotcha, I didn't know the closure was being called optionally. That works 👍