Abscissify light node (#125)

* Boilerplate: Add lite-node crate

- ran `abscissa new lite-node`
- added deps (Cargo.toml) and minimal changes to README.md
- add to root workspace

* Added config options & copied code into new app crate
- copied from tendermint-lite/src/main.rs to lite-node/src/command/start.rs

* Delete tendermint-lite: replaced by lite-node

* lite -> light

* minor improvements to comments / docs

minor improvements to comments / docs

* Fix a few merge hicks (catch up with latest changes from master)

rename some vars and more logical bisection in cmd

* fix rebasing hicks

* Bucky/abscissify adr (#148)

* adr-002 lite client (#54)

* adr-002 lite client

* finish adr

* address Dev's comments

* lite -> light

* Apply suggestions from code review

Co-Authored-By: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* updates from review

* Apply suggestions from code review

Co-Authored-By: Ismail Khoffi <Ismail.Khoffi@gmail.com>

* update for better abstraction and latest changes

* note about detection

* update image. manager -> syncer

* update image

* update image

* More detailed diagram of lite client data flow (#106)

* refactor

* refactor into more adrs

* minor fixes from review

* sync adr-003 with latest and call it

* rust code blocks

Co-authored-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Co-authored-by: jibrown <jackie.ilana.brown@gmail.com>

* working on adr-004

Co-authored-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
Co-authored-by: jibrown <jackie.ilana.brown@gmail.com>

* Dealing with the merging in master aftermath

* merged in master

* Fix merge master fallout (related to #169)

* Use `abscissa_tokio` and resolve merge conflicts

* New stable rust -> new clippy errs -> fixed

Co-authored-by: Ethan Buchman <ethan@coinculture.info>
Co-authored-by: jibrown <jackie.ilana.brown@gmail.com>

Cargo.toml

@@ -2,5 +2,5 @@
 
 members = [
     "tendermint",
-    "tendermint-lite",
+    "light-node",
 ]

docs/architecture/adr-003-light-client-core-verification.md

@@ -241,6 +241,8 @@ where
 }
 ```
 
+In practice, this can be implemented as a Tendermint RPC client making requests 
+to the `/commit` and `/validators` endpoints of full nodes. 
 For testing, the Requester can be implemented by JSON files.
 
 ### Verification

docs/architecture/adr-004-light-client-cli.md

@@ -2,40 +2,67 @@
 
 ## Changelog
 
-2020-01-22: Some content copied from old ADR-002
+- 2020-02-09: Update about Abscissa
+- 2020-01-22: Some content copied from old ADR-002
 
 ## Status
 
-WIP. Just copied over from old ADR. Needs rework
+WIP. 
 
 ## Context
 
+The high level context for the light client is described in
+[ADR-002](adr-002-light-client-adr-index.md).
 
-### State
+For reference, a schematic of the light node is below:
+
+![Light Node Diagram](assets/light-node.png).
+
+Here we focus on how the Light Node process itself is composed.
+The light node process must consider the following features:
+
+- command line UX and flags
+- config file
+- logging
+- error handling
+- state management
+- exposing RPC servers
+
+Ideally, it can support all of this with a minimum of dependencies.
+
+We'd like to be able to start a light node process and have it sync to the
+latest height and stay synced while it runs.
+
+## Decision
+
+### Abscissa
+
+[Abscissa](https://github.com/iqlusioninc/abscissa) is a framework for building CLI 
+tools in Rust by Tony Arcieri of Iqlusion. 
+It's focus is on security and minimizing dependencies. 
+The full list of dependencies can be found [here](https://github.com/iqlusioninc/abscissa#depencencies).
+
+For instance, while it includes functionality for command-line option parsing like that 
+provided by `structopt` + `clap`, it does so with far less dependencies.
+
+[Users](https://github.com/iqlusioninc/abscissa#projects-using-abscissa)
+of note include the [Tendermint KMS](https://github.com/tendermint/kms)
+for validators and the new 
+[Zebra ZCash full node](https://github.com/ZcashFoundation/zebra).
 
-The light node state contains the following:
+See the [introductory blog
+post](https://iqlusion.blog/introducing-abscissa-rust-application-framework)
+for more details.
 
-- current height (H) - height for the next header we want to verify
-- last header (H-1) - the last header we verified
-- current validators (H) - validators for the height we want to verify (including all validator pubkeys and voting powers)
+### Config
 
-It also includes some configuration, which contains:
+Config includes:
 
 - trusting period
 - initial list of full nodes
 - method (sequential or skipping)
 - trust level (if method==skipping)
 
-The node is initialized with a trusted header for some height H-1
-(call this header[H-1]), and a validator set for height H (call this vals[H]).
-
-The node may be initialized by the user with only a height and header hash, and
-proceed to request the full header and validator set from a full node. This
-reduces the initialization burden on the user, and simplifies passing this
-information into the process, but for the state to be properly initialized it
-will need to get the correct header and validator set before starting the light
-client syncing protocol.
-
 The configuration contains an initial list of full nodes (peers).
 For the sake of simplicity, one of the peers is selected as the "primary", while the
 rest are considered "backups". Most of the data is downloaded from the primary,
@@ -46,75 +73,19 @@ the time from the trusted header is greater than a configurable "trusting
 period". If at any point the state is expired, the node should log an error and
 exit - it's needs to be manually reset.
 
-### Syncer
-
-The Syncing co-ordinates the syncing and is the highest level component. 
-We consider two approaches to syncing the light node: sequential and skipping.
-
-#### Sequential Sync
 
-Inital state: 
+### Initialization
 
-    - time T
-    - height H 
-    - header[H-1]
-    - vals[H]
+The node is initialized with a trusted header for some height and a validator set for the next height.
 
-Here we describe the happy path:
-
-1) Request header[H], commit[H], and vals[H+1] from the primary, and check that they are well formed and from the correct height
-2) Pass header[H], commit[H], vals[H], and vals[H+1] to the verification library, which will:
-
-    - check that vals[H] and vals[H+1] are correctly reflected in header[H]
-    - check that commit[H] is for header[H]
-    - check that +2/3 of the validators correctly signed the hash of header[H]
-
-3) Request header[H] from each of the backups and check that they match header[H] received from the primary
-4) Update the state with header[H] and vals[H+1], and increment H
-5) return to (1)
-
-If (1) or (2) fails, mark the primary as bad and select a new peer to be the
-primary.
-
-If (3) returns a conflicting header, verify the header by requesting the
-corresponding commit and running the verification of (2). If the verification
-passes, there is a fork, and evidence should be published so the validators get
-slashed. We leave the mechanics of evidence to a future document. For now, the
-light client will just log an error and exit. If the verification fails, it
-means the backup that provided the conflict is bad and should be removed.
-
-#### Skipping Sync
-
-Skipping sync is essentially the same as sequential, except for a few points:
-
-- instead of verifying sequential headers, we attempt to "skip" ahead to the
-  full node's most recent height
-- skipping is only permitted if the validator set has not changed too much - ie.
-  if +1/3 of the last trusted validator set has signed the commit for the height we're attempting to skip to
-- if the validator set changes too much, we "bisect" the height space,
-  attempting to skip to a lower height, recursively. 
-- in the worst case, the bisection takes us to a sequential height
-
-### Requester
-
-The requester is simply a Tendermint RPC client. It makes requests to full
-nodes. It uses the `/commit` and `/validators` endpoints to get signed headers
-and validator sets for relevant heights. It may also use the `/status` endpoint
-to get the latest height of the full node (for skipping verification). It 
-uses the following trait (see below for definitions of the referenced types):
-
-```rust
-pub trait Requester {
-    type SignedHeader: SignedHeader;
-    type ValidatorSet: ValidatorSet;
-
-    fn signed_header<H>(&self, h: H) -> Result<Self::SignedHeader, Error>
-        where H: Into<Height>;
-
-    fn validator_set<H>(&self, h: H) -> Result<Self::ValidatorSet, Error>
-        where H: Into<Height>;
-}
-```
+The node may be initialized by the user with only a height and header hash, and
+proceed to request the full header and validator set from a full node. This
+reduces the initialization burden on the user, and simplifies passing this
+information into the process, but for the state to be properly initialized it
+will need to get the correct header and validator set before starting the light
+client syncing protocol.
 
-Note that trait uses `Into<Height>` which is a common idiom for the codebase.
+### State
 
+The light node will need to maintain state including the current height, the
+last verified and trusted header, and the current set of trusted validators.

docs/architecture/adr-005-light-client-fork-detection.md

@@ -45,3 +45,47 @@ one correct full node in order to detect conflicts in a timely fashion. We keep
 this mechanism simple for now, but in the future a more advanced peer discovery
 mechanism may be utilized.
 
+
+#### Sequential Sync
+
+Inital state: 
+
+    - time T
+    - height H 
+    - header[H-1]
+    - vals[H]
+
+Here we describe the happy path:
+
+1) Request header[H], commit[H], and vals[H+1] from the primary, and check that they are well formed and from the correct height
+2) Pass header[H], commit[H], vals[H], and vals[H+1] to the verification library, which will:
+
+    - check that vals[H] and vals[H+1] are correctly reflected in header[H]
+    - check that commit[H] is for header[H]
+    - check that +2/3 of the validators correctly signed the hash of header[H]
+
+3) Request header[H] from each of the backups and check that they match header[H] received from the primary
+4) Update the state with header[H] and vals[H+1], and increment H
+5) return to (1)
+
+If (1) or (2) fails, mark the primary as bad and select a new peer to be the
+primary.
+
+If (3) returns a conflicting header, verify the header by requesting the
+corresponding commit and running the verification of (2). If the verification
+passes, there is a fork, and evidence should be published so the validators get
+slashed. We leave the mechanics of evidence to a future document. For now, the
+light client will just log an error and exit. If the verification fails, it
+means the backup that provided the conflict is bad and should be removed.
+
+#### Skipping Sync
+
+Skipping sync is essentially the same as sequential, except for a few points:
+
+- instead of verifying sequential headers, we attempt to "skip" ahead to the
+  full node's most recent height
+- skipping is only permitted if the validator set has not changed too much - ie.
+  if +1/3 of the last trusted validator set has signed the commit for the height we're attempting to skip to
+- if the validator set changes too much, we "bisect" the height space,
+  attempting to skip to a lower height, recursively. 
+- in the worst case, the bisection takes us to a sequential height

light-node/.gitignore

@@ -0,0 +1,2 @@
+/target
+**/*.rs.bk

light-node/Cargo.toml

@@ -0,0 +1,24 @@
+[package]
+name = "light_node"
+authors = ["Ethan Buchman <ethan@coinculture.info>", "Ismail Khoffi <Ismail.Khoffi@gmail.com>"]
+version = "0.1.0"
+edition = "2018"
+
+[dependencies]
+gumdrop = "0.7"
+serde = { version = "1", features = ["serde_derive"] }
+tendermint = { version = "0.12.0-rc0", path = "../tendermint" }
+async-trait = "0.1"
+tokio = { version = "0.2", features = ["full"] }
+abscissa_tokio = "0.5"
+
+[dependencies.abscissa_core]
+version = "0.5.0"
+# optional: use `gimli` to capture backtraces
+# see https://github.com/rust-lang/backtrace-rs/issues/189
+# features = ["gimli-backtrace"]
+
+[dev-dependencies]
+abscissa_core = { version = "0.5.0", features = ["testing"] }
+once_cell = "1.2"
+

light-node/README.md

@@ -0,0 +1,14 @@
+# LightNode
+
+Tendermint light client node.
+
+## Getting Started
+
+This application is authored using [Abscissa], a Rust application framework.
+
+For more information, see:
+
+[Documentation]
+
+[Abscissa]: https://github.com/iqlusioninc/abscissa
+[Documentation]: https://docs.rs/abscissa_core/

light-node/src/application.rs

@@ -0,0 +1,111 @@
+//! LightNode Abscissa Application
+
+use crate::{commands::LightNodeCmd, config::LightNodeConfig};
+use abscissa_core::{
+    application::{self, AppCell},
+    config, trace, Application, EntryPoint, FrameworkError, StandardPaths,
+};
+use abscissa_tokio::TokioComponent;
+
+/// Application state
+pub static APPLICATION: AppCell<LightNodeApp> = AppCell::new();
+
+/// Obtain a read-only (multi-reader) lock on the application state.
+///
+/// Panics if the application state has not been initialized.
+pub fn app_reader() -> application::lock::Reader<LightNodeApp> {
+    APPLICATION.read()
+}
+
+/// Obtain an exclusive mutable lock on the application state.
+pub fn app_writer() -> application::lock::Writer<LightNodeApp> {
+    APPLICATION.write()
+}
+
+/// Obtain a read-only (multi-reader) lock on the application configuration.
+///
+/// Panics if the application configuration has not been loaded.
+pub fn app_config() -> config::Reader<LightNodeApp> {
+    config::Reader::new(&APPLICATION)
+}
+
+/// LightNode Application
+#[derive(Debug)]
+pub struct LightNodeApp {
+    /// Application configuration.
+    config: Option<LightNodeConfig>,
+
+    /// Application state.
+    state: application::State<Self>,
+}
+
+/// Initialize a new application instance.
+///
+/// By default no configuration is loaded, and the framework state is
+/// initialized to a default, empty state (no components, threads, etc).
+impl Default for LightNodeApp {
+    fn default() -> Self {
+        Self {
+            config: None,
+            state: application::State::default(),
+        }
+    }
+}
+
+impl Application for LightNodeApp {
+    /// Entrypoint command for this application.
+    type Cmd = EntryPoint<LightNodeCmd>;
+
+    /// Application configuration.
+    type Cfg = LightNodeConfig;
+
+    /// Paths to resources within the application.
+    type Paths = StandardPaths;
+
+    /// Accessor for application configuration.
+    fn config(&self) -> &LightNodeConfig {
+        self.config.as_ref().expect("config not loaded")
+    }
+
+    /// Borrow the application state immutably.
+    fn state(&self) -> &application::State<Self> {
+        &self.state
+    }
+
+    /// Borrow the application state mutably.
+    fn state_mut(&mut self) -> &mut application::State<Self> {
+        &mut self.state
+    }
+
+    /// Register all components used by this application.
+    ///
+    /// If you would like to add additional components to your application
+    /// beyond the default ones provided by the framework, this is the place
+    /// to do so.
+    fn register_components(&mut self, command: &Self::Cmd) -> Result<(), FrameworkError> {
+        let mut components = self.framework_components(command)?;
+        components.push(Box::new(TokioComponent::new()?));
+        self.state.components.register(components)
+    }
+
+    /// Post-configuration lifecycle callback.
+    ///
+    /// Called regardless of whether config is loaded to indicate this is the
+    /// time in app lifecycle when configuration would be loaded if
+    /// possible.
+    fn after_config(&mut self, config: Self::Cfg) -> Result<(), FrameworkError> {
+        // Configure components
+        self.state.components.after_config(&config)?;
+        self.config = Some(config);
+        Ok(())
+    }
+
+    /// Get tracing configuration from command-line options
+    fn tracing_config(&self, command: &EntryPoint<LightNodeCmd>) -> trace::Config {
+        if command.verbose {
+            trace::Config::verbose()
+        } else {
+            trace::Config::default()
+        }
+    }
+}