chore: WIP

pilot/pilot/design/concepts.md

@@ -0,0 +1,27 @@
+# Concepts
+---
+## Global
+
+### Region
+Inter Region Latencies 
+10 - 100
+Maybe 300ms
+
+
+### Availability Zone
+< 10 ms
+~3 ms
+
+Some components are only available in the same Zone
+- AWS EBS volume
+- Subnet
+
+### Node
+
+
+---
+
+## Orchestration
+
+### Allocation
+An Allocation is a mapping between a task group in a job and a client node. A single job may have hundreds or thousands of task groups, meaning an equivalent number of allocations must exist to map the work to client machines. Allocations are created by the Nomad servers as part of scheduling decisions made during an evaluation.

pilot/pilot/design/general.md

@@ -0,0 +1,24 @@
+Single Global Control Plane
+
+e.g. 
+- AWS Dashboard
+- DO Dashboard
+
+For handling Auth / Teams / Billing Centrally
+
+### Regions
+#### Authority Region
+There's still single Primary
+Maybe Control Plane resides here
+
+#### Isolated
+No shared fault domain
+Failures don't impact each other
+
+#### Autonomous ?!?
+
+
+#### Minimal Communication
+Data usually doesn't leave the Region.
+Public Internet is slow and Expensive
+

pilot/pilot/design/influences.md

@@ -0,0 +1,19 @@
+# Nomad
+Single Binary
+Node
+- Drivers
+- VMs
+    - KVM
+    - FireCracker
+- Containers
+    - Podman
+    - Docker
+    - Rkt?
+- Metadata
+
+Self Registration
+
+Federation
+
+#
+

pilot/pilot/design/objects.md

@@ -0,0 +1,39 @@
+Region
+
+Cluster?!?! VPC ?!!?! Smaller Blast Redius
+
+Zone
+- Name
+- Subnet
+
+Node
+- Name / UUID
+- Version
+- Status
+- Eligible?
+- Drained?
+- Address
+
+
+- Pool? 
+- Zone
+- Drivers (KVM, Firecracker, Doker, Podman)
+    - Status
+    - Network (Bridge)
+    - Version
+    - Runtime?
+
+- Resources (RAM, CPU)
+
+Attributes
+- OS
+- Arch
+- CPU / Cores / Freq
+
+
+Node Events
+
+
+Client??!?! 
+Server?!?!
+

pilot/pilot/design/provision.md

@@ -0,0 +1,53 @@
+# Provision
+There's no way to do something like tofu.deploy() right now.
+
+This goes through two levels of abstractions?. JSII
+
+### CDKTF
+We write TerraformStack subclass and define our configration in init. It should something like this
+```python
+from cdktf import App, Fn, TerraformStack, TerraformVariable
+from cdktf_cdktf_provider_digitalocean.provider import DigitaloceanProvider
+from cdktf_cdktf_provider_digitalocean.vpc import Vpc
+from constructs import Construct
+
+
+class MyStack(TerraformStack):
+	def __init__(self, scope: Construct, id: str):
+		super().__init__(scope, id)
+		do_token_variable = TerraformVariable(self,"do_token", type="string")
+		DigitaloceanProvider(self, "digitalocean", token=do_token_variable.string_value)
+		vpc = Vpc(self, "example_vpc", name="vpc-1", region="blr-1", ip_range="ip_range")
+
+```
+
+Unfortunately, Pilot config isn't static. But good news is, this is actually implemented as following
+
+1. Define TerraformStack subclass, like we did before
+This is equivalent of writing a HCL file
+
+2. Define an app and call `synth` on it
+
+```python
+from cdktf import App, Fn, TerraformStack, TerraformVariable
+
+
+app = App()
+MyStack(app, "cdktf-demo")
+app.synth()
+```
+
+
+3. Apply generated plan
+`cdktf deploy`
+We can open up the implemntation and see what happens underneath.
+
+1. If the implementation is complicated then we can run `cdktf deploy` ourselves
+
+
+---
+
+We need to put some dynamic logic on Step 1.
+We can't write a class everytime. 
+Generating Python code is basically the same as writing HCL
+What we can do is build a class implementation and app implementation on the fly

pilot/pilot/design/workloads.md

@@ -0,0 +1,7 @@
+# System
+DaemonSet
+
+Run on all (or matching nodes at all times)
+
+# Service
+Typical Container

pilot/provision/README.wtf.md

@@ -0,0 +1,40 @@
+We want to do as much work in Python / JSON as possible
+
+Luckily Tofu helps out.
+
+Every plan / state can be stored as JSON or converted to JSON.
+
+Here's the rough idea
+
+```mermaid
+stateDiagram
+    code: Python Declaration
+    plan: Plan
+    json: Synthesized JSON
+    infra: Infrastructure
+    code --> json: app.synth()
+    json --> plan: tofu plan
+    plan --> infra: tofu deploy
+```
+
+
+Same thing in words
+
+1. Write Python code in `TerraformStack.__init__()` that describes the infra we need
+2. "Synthesize" this TerraformStack object `app.synth()`
+
+The synth actually executes the `__init__` so we can do whatever we want in Python (loops, conditionals etc)
+
+`synth` generates a `cdktf.out/stacks/<stack-name>/cdktf.json` file in the working directory (this is `frappe-bench/sites`)
+
+We will use `sites/<site>/stacks` for now. So the state moves with the site without any special handling. 
+
+TODO: Include this directory in the file backups.
+
+3. Store this synthesized JSON in some DocType Provision
+
+
+Note: Our Stack can have bugs or the code that defines what we need can have bugs. Have a way to prevent catastrophies at this stage. We need sanity checks in Production to guard against
+- Don't trigger anything that can cause data loss
+- Don't trigger massive changes ( More than n resources at a time)
+- Cross stack changes ?! (Don't delete someone other region?!)

pilot/provision/bootstrap.py

@@ -0,0 +1,42 @@
+# Copyright (c) 2024, Frappe and contributors
+# For license information, please see license.txt
+
+import frappe
+
+PROVIDER = "do"
+DO_ACCESS_TOKEN = ""
+CIDR = "10.10.0.0/24"
+REGION = "blr1"
+TITLE = "Bangalore"
+NAME = "do-bangalore-blr1"
+
+
+def create():
+	if frappe.db.exists("Region", NAME):
+		region = frappe.get_doc("Region", NAME)
+	else:
+		region = frappe.new_doc(
+			"Region",
+			**{
+				"name": NAME,
+				"access_token": DO_ACCESS_TOKEN,
+				"cloud_provider": PROVIDER,
+				"region_slug": REGION,
+				"title": TITLE,
+				"vpc_cidr_block": CIDR,
+			},
+		).insert()
+
+	region.provision()
+
+
+def destroy():
+	if frappe.db.exists("Region", NAME):
+		region = frappe.get_doc("Region", NAME)
+		region.destroy()
+
+
+def clear():
+	doctypes = ["Provision Declaration", "Provision Plan", "Provision State", "Provision Action"]
+	for doctype in doctypes:
+		frappe.db.delete(doctype)

pilot/provision/doctype/provision_action/provision_action.js

@@ -0,0 +1,8 @@
+// Copyright (c) 2024, Frappe and contributors
+// For license information, please see license.txt
+
+// frappe.ui.form.on("Provision Action", {
+// 	refresh(frm) {
+
+// 	},
+// });

pilot/provision/doctype/provision_action/provision_action.json

@@ -0,0 +1,105 @@
+{
+ "actions": [],
+ "autoname": "autoincrement",
+ "creation": "2024-07-29 11:26:21.416387",
+ "doctype": "DocType",
+ "engine": "MyISAM",
+ "field_order": [
+  "region",
+  "stack",
+  "column_break_payx",
+  "action",
+  "section_break_tioa",
+  "parsed_output",
+  "output",
+  "error"
+ ],
+ "fields": [
+  {
+   "fieldname": "stack",
+   "fieldtype": "Data",
+   "in_filter": 1,
+   "in_list_view": 1,
+   "in_standard_filter": 1,
+   "label": "Stack",
+   "read_only": 1,
+   "reqd": 1
+  },
+  {
+   "fieldname": "action",
+   "fieldtype": "Data",
+   "in_list_view": 1,
+   "in_preview": 1,
+   "in_standard_filter": 1,
+   "label": "Action",
+   "read_only": 1,
+   "reqd": 1
+  },
+  {
+   "default": "{}",
+   "fieldname": "output",
+   "fieldtype": "Code",
+   "label": "Output",
+   "read_only": 1
+  },
+  {
+   "fieldname": "error",
+   "fieldtype": "Code",
+   "label": "Error",
+   "read_only": 1
+  },
+  {
+   "fieldname": "region",
+   "fieldtype": "Link",
+   "in_filter": 1,
+   "in_list_view": 1,
+   "in_standard_filter": 1,
+   "label": "Region",
+   "options": "Region",
+   "read_only": 1,
+   "reqd": 1
+  },
+  {
+   "fieldname": "column_break_payx",
+   "fieldtype": "Column Break"
+  },
+  {
+   "fieldname": "section_break_tioa",
+   "fieldtype": "Section Break"
+  },
+  {
+   "default": "{}",
+   "fieldname": "parsed_output",
+   "fieldtype": "Code",
+   "label": "Parsed Output",
+   "read_only": 1
+  }
+ ],
+ "index_web_pages_for_search": 1,
+ "links": [],
+ "modified": "2024-08-09 16:33:22.536559",
+ "modified_by": "Administrator",
+ "module": "Provision",
+ "name": "Provision Action",
+ "naming_rule": "Autoincrement",
+ "owner": "Administrator",
+ "permissions": [
+  {
+   "create": 1,
+   "delete": 1,
+   "email": 1,
+   "export": 1,
+   "print": 1,
+   "read": 1,
+   "report": 1,
+   "role": "System Manager",
+   "share": 1,
+   "write": 1
+  }
+ ],
+ "show_title_field_in_link": 1,
+ "sort_field": "creation",
+ "sort_order": "DESC",
+ "states": [],
+ "title_field": "stack"
+}