docs: extract canister configuration from dfx.json into separate files per canister

docs/design/separate_canister_json.adoc

@@ -0,0 +1,252 @@
+= Separate Canister JSON Design Doc
+// Author field:
+Eric Swanson <ericswanson@dfinity.org>
+v0.1, 2020-07-16
+:draft:
+:toc:
+
+== Overview
+
+Keep from concentrating too much configuration in dfx.json.
+
+Reduce repetition in paths in configuration.
+
+Make it so canister definitions are self-contained.
+
+== Background
+
+`dfx.json` currently directly holds the configuration for all canisters.
+
+The `dfx config` command displays or changes this configuration.
+
+Glossary:
+
+* The *project root* is the directory that contains `dfx.json` and
+the rest of the files in the project.
+
+
+Here is a `dfx.json` file from a newly-created sample project.
+
+[source,json]
+----
+include::separate_canister_json/new-project-dfx.json[]
+----
+
+Here is the `dfx.json` file for CanCan:
+
+[source,json]
+----
+include::separate_canister_json/candid-dfx.json[]
+----
+
+=== Problem Statement
+
+Addition of canisters will concentrate a lot of configuration in one file.
+
+Also, there is some duplication of project names in paths:
+
+* Notice redundancies in the sample project's `dfx.json`:
+** `sample` occurs in two places
+** `sample_assets` in four places
+** `src/sample_assets` occurs in two places
+* Notice redundancies in CanCan's `dfx.json`:
+** `dht` occurs in two places.
+** `vendor/bigmap-rs` occurs in 12 places.
+** `cancan_ui` occurs in two places.
+** `bigmap_data_0`, `bigmap_data_1', and `bigmap_data_2` are identical (though this may be rare special case?)
+
+=== Requirements
+
+Representable / enforceable by JSON Schema
+
+== Expected User/Developer Experience
+
+The user can define their canisters independently from each other.
+
+== Prior Art
+
+* link:https://lerna.js.org/[Lerna]
+
+== Detailed Design
+
+* Change `canisters` in `dfx.json` to be an array of directory globs
+for locating canisters.
+* Search those directories for files named `canister.json`
+* For those found, `parent directory` is the canister name, `canister.json` defines the rest.
+* Remove the `dfx config` command.
+* Also accept the old form for `canisters`: a map of canister name -> canister configuration.
+** This form will be deprecated, but allow for a transition.
+
+Rules for paths within a canister.json file are as follows:
+
+* relative paths are relative to the file's parent directory
+* absolute paths are relative to the parent directory of `dfx.json`
+* this applies to custom build steps as well
+
+The canister.json file contains fields that are current stored in
+the value portion of the canisters object in `dfx.json`.
+
+All relative paths must resolve to a canonical path contained within the *project root*, for security.
+
+An option: the `canister.json` file could also contain an `aliases` field that is an array of alternate names.
+
+Example `dfx.json` for a new project:
+
+[source,json]
+----
+{
+  "canisters": [ "canisters/*" ],
+  ...
+}
+----
+
+Example `canisters/sample/canister.json`:
+
+[source,json]
+----
+{
+  "main": "main.mo",
+  "type": "motoko"
+}
+----
+
+Example `canisters/sample_assets/canister.json`:
+
+[source,json]
+----
+{
+  "dependencies": [
+    "sample"
+  ],
+  "frontend": {
+    "entrypoint": "public/index.js"
+  },
+  "source": [
+    "assets",
+    "/dist/sample_assets/"
+  ],
+  "type": "assets"
+}
+----
+
+Example for CanCan:
+
+This doesn't work for CanCan, because it requires three copies of the same canister.
+
+Since this is a workaround, it is not duplicated here.
+
+* Contents of `dfx.json`:
+
+[source,json]
+----
+{
+  "canisters": [
+    "vendor/dht",
+    "vendor/bigmap-rs/index",
+    "vendor/bigmap-rs/data",
+    "cancan_ui"
+  ],
+  ...
+}
+----
+
+* Contents of `vendor/dht/canister.json`:
+
+[source,json]
+----
+{
+  "type": "motoko",
+  "main": "app/Main.mo"
+}
+----
+
+Note that this would live in the linked repository.
+
+
+* Contents of `vendor/bigmap-rs/index/canister.json`:
+
+[source,json]
+----
+{
+  "type": "custom",
+  "candid": "src/bigmap_index.did",
+  "wasm": "target/wasm32-unknown-unknown/release/bigmap_index.wasm",
+  "build": "./build"
+}
+
+----
+
+* Contents of `vendor/bigmap-rs/data/canister.json`:
+
+[source,json]
+----
+{
+  "type": "custom",
+  "candid": "src/bigmap_data.did",
+  "wasm": "target/wasm32-unknown-unknown/release/bigmap_data.wasm",
+  "build": "./build"
+}
+----
+
+* Contents of `cancan_ui/canister.json`:
+
+[source,json]
+----
+{
+  "type": "assets",
+  "frontend": {
+    "entrypoint": "/www/index.tsx",
+    "output": "assets"
+  },
+  "dependencies": ["bigmap"]
+}
+----
+
+
+=== Considered Solutions
+
+* Leave it the same as now (single dfx.json)
+** pro: No additional work
+** con: Not future-friendly for projects with lots of canisters
+* `canisters` as a JSON object (map) with name=canister name, value=(path | canister config)
+** no pros
+** con: Part of the  canister config (the name) in `dfx.json`, and part of it in a file somewhere else
+
+=== Recommended Solution
+
+Adoption of this proposal, and for simplicity's sake WITHOUT canister name aliases.
+
+* canister name aliases introduce complexity: more than one name for the same thing, for example with dependencies
+
+=== Security Considerations
+
+* relative paths must resolve within the *project root*, to avoid things like
+publishing the contents of /etc in an assets canister, or launching an
+arbitrary executable as part of a custom build step.
+* Custom build type canisters define a build step (shell command).  Moving that definition
+out of `dfx.json` into a `canister.json` that may come from another repo (see
+git submodule links used by CanCan, for example) marginally decreases the visibility of those
+custom build steps.
+
+== Breaking Changes
+
+Since we will continue to support the old format during a transition period, this is non-breaking.
+
+=== Deprecation
+
+This deprecates the `canisters` field in map form.
+
+== Documentation
+
+It will be necessary to update all of the docs and tutorials
+that reference canister configuration in `dfx.json`.
+
+== Lifecycle
+
+== Work Breakdown
+
+* Remove `dfx config`
+* Update "new project" files
+* Alter config loader
+* Deal with  canister name aliases, if allowed
+* Update e2e tests

docs/design/separate_canister_json/candid-dfx.json

@@ -0,0 +1,52 @@
+{
+  "canisters": {
+    "dht": {
+      "type": "motoko",
+      "main": "vendor/dht/app/Main.mo"
+    },
+    "bigmap": {
+      "type": "custom",
+      "candid": "vendor/bigmap-rs/src/bigmap_index.did",
+      "wasm": "vendor/bigmap-rs/target/wasm32-unknown-unknown/release/bigmap_index.wasm",
+      "build": "./vendor/bigmap-rs/build"
+    },
+    "bigmap_data_0": {
+      "type": "custom",
+      "candid": "vendor/bigmap-rs/src/bigmap_data.did",
+      "wasm": "vendor/bigmap-rs/target/wasm32-unknown-unknown/release/bigmap_data.wasm",
+      "build": "./vendor/bigmap-rs/build"
+    },
+    "bigmap_data_1": {
+      "type": "custom",
+      "candid": "vendor/bigmap-rs/src/bigmap_data.did",
+      "wasm": "vendor/bigmap-rs/target/wasm32-unknown-unknown/release/bigmap_data.wasm",
+      "build": "./vendor/bigmap-rs/build"
+    },
+    "bigmap_data_2": {
+      "type": "custom",
+      "candid": "vendor/bigmap-rs/src/bigmap_data.did",
+      "wasm": "vendor/bigmap-rs/target/wasm32-unknown-unknown/release/bigmap_data.wasm",
+      "build": "./vendor/bigmap-rs/build"
+    },
+    "cancan_ui": {
+      "type": "assets",
+      "frontend": {
+        "entrypoint": "www/index.tsx",
+        "output": "canisters/cancan_ui/assets"
+      },
+      "dependencies": ["bigmap"]
+    }
+  },
+  "defaults": {
+    "build": {
+      "output": "canisters",
+      "packtool": ""
+    },
+    "start": {
+      "address": "0.0.0.0",
+      "port": 8080,
+      "serve_root": "canisters/cancan_ui/assets"
+    }
+  },
+  "dfx": "0.5.8"
+}

docs/design/separate_canister_json/new-project-dfx.json

@@ -0,0 +1,41 @@
+{
+  "canisters": {
+    "sample": {
+      "main": "src/sample/main.mo",
+      "type": "motoko"
+    },
+    "sample_assets": {
+      "dependencies": [
+        "sample"
+      ],
+      "frontend": {
+        "entrypoint": "src/sample_assets/public/index.js"
+      },
+      "source": [
+        "src/sample_assets/assets",
+        "dist/sample_assets/"
+      ],
+      "type": "assets"
+    }
+  },
+  "defaults": {
+    "build": {
+      "output": "canisters/",
+      "packtool": ""
+    }
+  },
+  "dfx": "0.5.12-3-g60ea1c9",
+  "networks": {
+    "local": {
+      "bind": "127.0.0.1:8000",
+      "type": "ephemeral"
+    },
+    "tungsten": {
+      "providers": [
+        "https://gw.dfinity.network"
+      ],
+      "type": "persistent"
+    }
+  },
+  "version": 1
+}