pokt-network · okdas · Feb 6, 2023 · Nov 12, 2022 · Nov 12, 2022 · Nov 15, 2022
diff --git a/Makefile b/Makefile
@@ -102,6 +102,7 @@ install_cli_deps: ## Installs `protoc-gen-go`, `mockgen`, 'protoc-go-inject-tag'
 	go install "github.com/favadi/protoc-go-inject-tag@latest"
 	go install "github.com/deepmap/oapi-codegen/cmd/oapi-codegen@v1.11.0"
 	curl -fsSL https://raw.githubusercontent.com/tilt-dev/tilt/master/scripts/install.sh | bash
+	curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
 
 .PHONY: develop_start
 develop_start: ## Run all of the make commands necessary to develop on the project
@@ -445,7 +446,7 @@ clear_genesis_and_config: ## Clear the genesis and config files for LocalNet
 
 .PHONY: localnet_up
 localnet_up: ## Starts up a k8s LocalNet with all necessary dependencies (tl;dr `tilt up`)
-	tilt up
+	tilt up --file=build/localnet/Tiltfile
 
 .PHONY: localnet_client_debug
 localnet_client_debug: ## Opens a `client debug` cli to interact with blockchain (e.g. change pacemaker mode, reset to genesis, etc). Though the node binary updates automatiacally on every code change (i.e. hot reloads), if client is already open you need to re-run this command to execute freshly compiled binary.
@@ -465,8 +466,8 @@ localnet_logs_validators_follow: ## Outputs logs from all validators and follows
 
 .PHONY: localnet_down
 localnet_down: ## Stops LocalNet and cleans up dependencies (tl;dr `tilt down` + postgres database)
-	tilt down
-	kubectl delete pvc data-dependencies-postgresql-0
+	tilt down --file=build/localnet/Tiltfile
+	kubectl delete pvc --ignore-not-found=true data-dependencies-postgresql-0
 
 .PHONY: check_cross_module_imports
 check_cross_module_imports: ## Lists cross-module imports

diff --git a/app/docs/CHANGELOG.md → app/pocket/docs/CHANGELOG.md b/app/docs/CHANGELOG.md → app/pocket/docs/CHANGELOG.md
diff --git a/build/localnet/README.md b/build/localnet/README.md
@@ -3,7 +3,8 @@
 This guide shows how to deploy a LocalNet using [pocket-operator](https://github.com/pokt-network/pocket-operator).
 
 - [Dependencies](#dependencies)
-  - [Enabling Kubernetes](#enabling-kubernetes)
+  - [Choosing Kubernetes Distribution](#choosing-kubernetes-distribution)
+  - [Enabling Kubernetes For Docker Desktop](#enabling-kubernetes-for-docker-desktop)
 - [LocalNet](#localnet)
   - [Starting LocalNet](#starting-localnet)
   - [Viewing Logs](#viewing-logs)
@@ -13,30 +14,40 @@ This guide shows how to deploy a LocalNet using [pocket-operator](https://github
   - [Stopping \& Clean Resources](#stopping--clean-resources)
   - [Interacting w/ LocalNet](#interacting-w-localnet)
     - [Make Targets](#make-targets)
+  - [Addresses and keys on LocalNet](#addresses-and-keys-on-localnet)
+- [How to change configuration files](#how-to-change-configuration-files)
 - [How does it work?](#how-does-it-work)
 - [Troubleshooting](#troubleshooting)
   - [Why?](#why)
   - [Force Trigger an Update](#force-trigger-an-update)
   - [Force Restart](#force-restart)
   - [Full Cleanup](#full-cleanup)
-  - [Docker Desktop](#docker-desktop)
-- [How to change configuration files](#how-to-change-configuration-files)
 - [Code Structure](#code-structure)
 
 ## Dependencies
 
+All necessary dependencies, except Kubernetes cluster, are installed automatically when running `make install_cli_deps`. The following dependencies are required:
+
 1. [tilt](https://docs.tilt.dev/install.html)
-   - Note: automatically installed when running `make install_cli_deps`
-2. `Kubernetes cluster`: [installation options](https://docs.tilt.dev/choosing_clusters.html)
+2. `Kubernetes cluster`: refer to [Choosing Kubernetes Distribution](#choosing-kubernetes-distribution) section for more details.
 3. `kubectl`: CLI is required and should be configured to access the cluster. This should happen automatically if using Docker Desktop, Rancher Desktop, k3s, k3d, minikube, etc.
-4. `helm`: required to template the yaml manifests for the dependencies (e.g. postgres, grafana). Installation instructions available [here](https://helm.sh/docs/intro/install).
+4. [helm](https://helm.sh/docs/intro/install): required to template the yaml manifests for the dependencies (e.g. postgres, grafana). Installation instructions available .
+
+### Choosing Kubernetes Distribution
+
+While any Kubernetes distribution should work, we verified that LocalNet works on:
+- [Rancher Desktop](https://rancherdesktop.io/), which is GUI powered by a popular distribution `k3s`.
+- [kind](https://kind.sigs.k8s.io/) - official Kubernetes distribution that runs inside docker containers.
+
+Here is a list of alternative set ups that should work: https://docs.tilt.dev/choosing_clusters.html
 
-### Enabling Kubernetes
+### Enabling Kubernetes For Docker Desktop
 
-You may need to manually enable Kubernetes if using Docker desktop:
+You may need to manually enable Kubernetes if using Docker Desktop:
 
 ![Docker desktop kubernetes](https://user-images.githubusercontent.com/1892194/216165581-1372e2b8-c630-4211-8ced-5ec59b129330.png)
 
+
 ## LocalNet
 
 ### Starting LocalNet
@@ -45,6 +56,8 @@ You may need to manually enable Kubernetes if using Docker desktop:
 make localnet_up
 ```
 
+This action will create a file called `localnet_config.yaml` in the root of the repo if it doesn't exist. The default configuration can be found in [Tiltfile](Tiltfile#L11).
+
 ### Viewing Logs
 
 The developer can then view logs either from a browser or terminal.
@@ -67,6 +80,11 @@ When starting a k8s LocalNet, `localnet_config.yaml` is generated (with default
 
 The config file can be modified to scale the number of actors up/down. As long as `localnet_up` is running, the changes should be automatically applied within seconds.
 
+[Watch demo](https://user-images.githubusercontent.com/4950477/216490690-8ac4c16a-25e1-4202-b2e5-03215030c82c.mp4)
+
+![Watch demo](https://user-images.githubusercontent.com/4950477/216490690-8ac4c16a-25e1-4202-b2e5-03215030c82c.mp4)
+
+
 ### Stopping & Clean Resources
 
 ```bash
@@ -93,17 +111,24 @@ Open a `client debug` cli. It allows to interact with blockchain, e.g. change pa
 make localnet_client_debug
 ```
 
-## How does it work?
+### Addresses and keys on LocalNet
 
-[tilt](https://tilt.dev/) reads the [`Tiltfile`](../../Tiltfile), where LocalNet configs are specified. `Tiltfile` is written in [Starlark](https://github.com/bazelbuild/starlark), a dialect of Python.
+You can find private keys and addresses for all actors in the [private-keys.yaml](./manifests/private-keys.yaml) file. They have been pre-generated and follow a specific pattern - they start with pre-determined numbers for easier troubleshooting and debugging.
 
-The k8s manifests that `tilt` submits to the cluster can be found in [this directory](./):
+Addresses begin with `YYYXX` number, where `YYY` is a number of an actor and `XX` is [a type of actor](../../shared/core/types/proto/actor.proto#L7). For example, `420043b854e78f2d5f03895bba9ef16972913320` is a validator #420.
 
-- **[dependencies](./dependencies/)**: a helm chart that installs all necessary dependencies to run and observe LocalNet (postgresql, prometheus, grafana, etc).
-- **[4 Validators](./v1-validator-template.sh)**: The validator binary that runs inside of the container gets updated automatically and process restarted on each code change (i.e. hot reloads).
-- **[V1 CLI client](./cli-client.yaml)**: This binary that can be used to perform debug operations. Run `make localnet_client_debug` to execute commands such as `ResetToGenesis` or `TogglePacemakerMode`. This binary is also automatically updated when you make changes to the codebase.
 
-Tilt continuously monitors files on local filesystem, and it rebuilds the binary and distributes it to the pods on every code change. This allows developers to iterate on the code and see the changes immediately.
+## How to change configuration files
+
+Currently, we provide [a config file](./manifests/configs.yaml) that is shared between all validators and a pocket client. We make use of `pocket` client feature that allows us to override configuration via environment variables. You can check a [validator template](./templates/v1-validator-template.yaml.tpl) as a reference.
+
+## How does it work?
+
+[tilt](https://tilt.dev/) reads the [`Tiltfile`](./Tiltfile), where LocalNet configs are specified. `Tiltfile` is written in [Starlark](https://github.com/bazelbuild/starlark), a dialect of Python.
+
+The k8s manifests that `tilt` submits to the cluster can be found in [this directory](./). Please refer to [code structure](#code-structure) for more details where different parts are located.
+
+Tilt continuously monitors files on local filesystem in [specific directories](Tiltfile#L27), and it rebuilds the binary and distributes it to the pods on every code change. This allows developers to iterate on the code and see the changes immediately.
 
 ## Troubleshooting
 
@@ -132,34 +157,29 @@ If force triggering an update didn't work, try the following:
 
 If a force restart didn't help, try rebuilding local kubernetes cluster using the tool you're managing your cluster with (e.g. Docker Desktop, Rancher Desktop, k3s, k3d, minikube, etc).
 
-### Docker Desktop
-
-# TODO_IN_THIS_COMMIT
-
-## How to change configuration files
-
-Currently, we provide [a config file](./configs.yaml) that is shared between all validators and a pocket client. We make use of `pocket` client feature that allows us to override configuration via environment variables. You can look at [one of the validators](../../build/localnet/v1-validator1.yaml) as a reference.
-
 ## Code Structure
 
 ```bash
-build/localnet # TODO_IN_THIS_COMMIT
-├── README.md # TODO_IN_THIS_COMMIT
-├── cli-client.yaml # TODO_IN_THIS_COMMIT
-├── configs.yaml # TODO_IN_THIS_COMMIT
-├── dependencies # TODO_IN_THIS_COMMIT
-│   ├── Chart.yaml # TODO_IN_THIS_COMMIT
-│   ├── dashboards # TODO_IN_THIS_COMMIT
-│   │   ├── README.md # TODO_IN_THIS_COMMIT
-│   │   └── raintree-telemetry-graphs.json # TODO_IN_THIS_COMMIT
-│   ├── requirements.yaml # TODO_IN_THIS_COMMIT
-│   ├── templates # TODO_IN_THIS_COMMIT
-│   │   ├── _helpers.tpl # TODO_IN_THIS_COMMIT
-│   │   ├── dashboards.yml # TODO_IN_THIS_COMMIT
-│   │   └── datasources.yml # TODO_IN_THIS_COMMIT
-│   └── values.yaml # TODO_IN_THIS_COMMIT
-├── network.yaml # TODO_IN_THIS_COMMIT
-├── private-keys.yaml # TODO_IN_THIS_COMMIT
-├── v1-validator-template.sh # TODO_IN_THIS_COMMIT
-└── v1-validator-template.yaml # TODO_IN_THIS_COMMIT
+build/localnet
+├── README.md # This file
+├── Tiltfile # This file
+├── dependencies # Helm chart that installs all the dependencies needed to run and observe LocalNet
+│   ├── Chart.yaml # Main file of the helm chart, contains metadata
+│   ├── dashboards # Directory with all the dashboards that are automatically imported to Grafana
+│   │   ├── README.md # README file with instructions on how to add a new dashboard
+│   │   └── raintree-telemetry-graphs.json # Raintree Telemetry dashboard
+│   ├── requirements.yaml # Specifies dependencies of the chart, this allows us to install all the dependencies with a single command
+│   ├── templates # Additional Kubernetes manifests that we need to connect different dependencies together
+│   │   ├── _helpers.tpl
+│   │   ├── dashboards.yml
+│   │   └── datasources.yml
+│   └── values.yaml # Configuration values that override the default values of the dependencies, this allows us to connect dependencies together and make them available to our LocalNet services
+├── manifests # Static YAML Kubernetes manifests that are consumed by `tilt`
+│   ├── cli-client.yaml # Pod that has the latest binary of the pocket client. Makefile targets run CLI in this pod.
+│   ├── configs.yaml # Location of the config files (default configs for all validators and a genesis file) that are shared between all actors
+│   ├── network.yaml # Networking configuration that is shared between different actors, currently a Service that points to all validators
+│   └── private-keys.yaml # Pre-generated private keys for everything
+└── templates # Templates for Kubernetes manifests that are consumed by `tilt`
+    ├── v1-validator-template.sh # Shell script that generates Kubenetes manifests for validators, consumed by `tilt`
+    └── v1-validator-template.yaml.tpl # Template for a single validator, consumed by `v1-validator-template.sh`
 ```
diff --git a/Tiltfile → build/localnet/Tiltfile b/Tiltfile → build/localnet/Tiltfile
@@ -3,25 +3,29 @@ load("ext://helm_resource", "helm_resource", "helm_repo")
 load("ext://namespace", "namespace_create")
 load("ext://restart_process", "docker_build_with_restart")
 
+tiltfile_dir = os.path.dirname(config.main_dir)
+root_dir = os.path.dirname(tiltfile_dir + '/../..')
+localnet_config_path = root_dir + "/localnet_config.yaml"
+
 # Create localnet config file from defaults, and if some default configuration doesn't exist in it - populate with default values
 localnet_config_defaults = {"validators": {"count": 4}}
 
 localnet_config_file = read_yaml(
-    "localnet_config.yaml", default=localnet_config_defaults
+    localnet_config_path, default=localnet_config_defaults
 )
+
 localnet_config = {}
 localnet_config.update(localnet_config_defaults)
 localnet_config.update(localnet_config_file)
 
 # Create a default config file if it does not exist
-if localnet_config_file != localnet_config:
-    print("Updating localnet_config.yaml")
-    local("cat - > localnet_config.yaml", stdin=encode_yaml(localnet_config))
+if (localnet_config_file != localnet_config) or (not os.path.exists(localnet_config_path)):
+    print("Updating " + localnet_config_path + " with defaults")
+    local("cat - > " + localnet_config_path, stdin=encode_yaml(localnet_config))
 
-# List of directories Tilt watches to trigger a hot-reload on changes
+# List of directories Tilt watches to trigger a hot-reload on changes. Potentially can be replaced with a list of excluded directories.
 deps = [
     "app",
-    # 'build',
     "consensus",
     "p2p",
     "persistance",
@@ -34,6 +38,8 @@ deps = [
     "logger",
 ]
 
+deps_full_path = [root_dir + "/" + depdir for depdir in deps]
+
 # Deploy dependencies (grafana, postgres, prometheus) and wire it up with localnet
 helm_repo(
     "grafana",
@@ -51,30 +57,34 @@ helm_repo(
 
 # Avoid downloading dependencies if no missing/outdated charts
 check_helm_dependencies = local(
-    "helm dependency list build/localnet/dependencies | awk '{print $4}' | grep -Ev 'ok|STATUS'"
+    "helm dependency list dependencies | awk '{print $4}' | grep -Ev 'ok|STATUS'"
 )
 helm_dependencies_not_ok_count = len(str(check_helm_dependencies).splitlines())
 if helm_dependencies_not_ok_count > 1:
-    local("helm dependency update build/localnet/dependencies")
+    local("helm dependency update dependencies")
 
-k8s_yaml(helm("build/localnet/dependencies", name="dependencies", namespace="default"))
+k8s_yaml(helm("dependencies", name="dependencies", namespace="default"))
 
 # Builds the pocket binary. Note target OS is linux, because no matter what your OS is, container runs linux natively or in VM.
 local_resource(
     "pocket: Watch & Compile",
-    "GOOS=linux go build -o bin/pocket-linux app/pocket/main.go",
-    deps=deps,
+    "GOOS=linux go build -o {root_dir}/bin/pocket-linux {root_dir}/app/pocket/main.go".format(
+        root_dir=root_dir
+    ),
+    deps=deps_full_path,
 )
 local_resource(
     "debug client: Watch & Compile",
-    "GOOS=linux go build -tags=debug -o bin/client-linux app/client/*.go",
-    deps=deps,
+    "GOOS=linux go build -tags=debug -o {root_dir}/bin/client-linux {root_dir}/app/client/*.go".format(
+        root_dir=root_dir
+    ),
+    deps=deps_full_path,
 )
 
 # Builds and maintains the validator container image after the binary is built on local machine, restarts a process on code change
 docker_build_with_restart(
     "validator-image",
-    ".",
+    root_dir,
     dockerfile_contents="""FROM debian:bullseye
 COPY bin/pocket-linux /usr/local/bin/pocket
 WORKDIR /
@@ -91,7 +101,7 @@ WORKDIR /
 # Builds and maintains the client container image after the binary is built on local machine
 docker_build_with_restart(
     "client-image",
-    ".",
+    root_dir,
     dockerfile_contents="""FROM debian:bullseye
 WORKDIR /
 COPY bin/client-linux /usr/local/bin/client
@@ -105,19 +115,19 @@ COPY bin/client-linux /usr/local/bin/client
 # Pushes localnet manifests to the cluster.
 k8s_yaml(
     [
-        "build/localnet/private-keys.yaml",
-        "build/localnet/configs.yaml",
-        "build/localnet/cli-client.yaml",
-        "build/localnet/network.yaml",
+        "manifests/private-keys.yaml",
+        "manifests/configs.yaml",
+        "manifests/cli-client.yaml",
+        "manifests/network.yaml",
         local(
-            "build/localnet/v1-validator-template.sh %s"
+            "templates/v1-validator-template.sh %s"
             % localnet_config["validators"]["count"],
             quiet=True,
         ),
     ]
 )
 
-# # Exposes postgres port to 5432 on the host machine.
+# An example of how to expose postgres port to 5432 on the host machine in case developer wants to connect to postgres directly
 # k8s_resource(new_name='postgres',
 #              objects=['pocket-database:postgresql'],
 #              extra_pod_selectors=[{'cluster-name': 'pocket-database'}],

diff --git a/build/localnet/cli-client.yaml → build/localnet/manifests/cli-client.yaml b/build/localnet/cli-client.yaml → build/localnet/manifests/cli-client.yaml
@@ -21,6 +21,8 @@ spec:
           imagePullPolicy: IfNotPresent
           command: ["sleep", "infinity"]
           env:
+            - name: POCKET_P2P_IS_CLIENT_ONLY
+              value: "true"
             - name: CONFIG_PATH
               value: "/var/pocket/config/config.json"
             - name: GENESIS_PATH
@@ -62,7 +64,6 @@ spec:
       volumes:
         - name: config-volume
           configMap:
-            # should client have it's own config?
             name: v1-validator-default-config
             defaultMode: 420
         - name: genesis-volume

diff --git a/build/localnet/configs.yaml → build/localnet/manifests/configs.yaml b/build/localnet/configs.yaml → build/localnet/manifests/configs.yaml
diff --git a/build/localnet/network.yaml → build/localnet/manifests/network.yaml b/build/localnet/network.yaml → build/localnet/manifests/network.yaml