Add sbctl integration proposal and move design directory into docs (#893

)

* add sbctl integration proposal and move design directory into docs

Co-authored-by: Evans Mungai <evans@replicated.com>

docs/design/proposal-sbctl-integration.md

@@ -0,0 +1,120 @@
+# Integrate SBCTL with Troubleshoot
+
+## Goals
+
+Make sbctl easier to maintain by consolidating code bases.
+
+Reduce code duplication.
+
+Improve end-user experience and improve discoverability of sbctl.
+
+## Non Goals
+
+## Background
+sbctl is our command line tool for examining K8s resources inside of Support Bundles generated by our troubleshoot project. With both projects having separate code bases we see the following problems:
+
+* Most users are familiar with the binaries from troubleshoot (support-bundle, preflight, etc.), but are unaware of the existince of sbctl because it's a separate project
+* sbctl and troubleshoot are naturally coupled together due to sbctl's reliance on support bundles to operate. Them being in separate repos leads to duplication of code in both projects increasing maintenance burden.
+
+## High-Level Design
+Because the functionality of sbctl and Troubleshoot are so tightly coupled together, they should live in the same repository. Moreover, as opposed to maintaing a separate binary for sbctl, all sbctl commands should become subcommands of `support-bundle`
+
+## Detailed Design
+The `serve` and `shell` commands become subcommands of the `support-bundle` CLI.
+
+```
+./bin/support-bundle --help
+A support bundle is an archive of files, output, metrics and state
+from a server that can be used to assist when troubleshooting a Kubernetes cluster.
+
+Usage:
+  support-bundle [url] [flags]
+  support-bundle [command]
+
+Available Commands:
+  analyze     analyze a support bundle
+  serve       Start API server
+  shell       Start interractive shell
+  completion  Generate the autocompletion script for the specified shell
+  ...
+```
+
+Similiar to the [Analyze](https://github.com/replicatedhq/troubleshoot/blob/main/cmd/troubleshoot/cli/analyze.go) subcommand, we add `serve.go` and `shell.go` to `cmd/troubleshoot/cli`
+
+### Standardize Cluster Resources Collector
+* In `pkg/collect` standardize the root directory location of the Cluster Resources collector with a variable instead of repeatedly specifying it as a string - https://github.com/replicatedhq/troubleshoot/blob/main/pkg/collect/cluster_resources.go#L121.
+* Import the directory the Cluster Resources collector uses from `pkg/collect` for usage in the APIs - https://github.com/replicatedhq/sbctl/blob/main/pkg/api/server.go#L220. This is an example of the way we can start making the sbctl code more efficient with this change.
+
+cluster_resources.go
+```go
+const ClusterResourcesPath = `cluster-resources`
+...
+
+path := filepath.Join(ClusterResourcesPath, "namespaces.json")
+output.SaveResult(c.BundlePath, path, bytes.NewBuffer(namespaces))
+```
+
+sbctl-server.go
+```go
+import (
+    ...
+    "github.com/replicatedhq/troubleshoot/pkg/collect"
+    ...
+)
+...
+
+dirName := filepath.Join(collect.ClusterResourcesPath, fmt.Sprintf("%s", resource))
+```
+
+### Package Structure
+```
+cmd
+    serve/
+    shell/
+pkg
+    serve/
+    shell/
+tests           # API tests, integration tests etc reside here.
+    serve/      # sbctl BDD tests.
+        # all tests and test fixtures moved here
+```
+
+### Makefile Changes
+
+We'll need to add make targets specific to these commands similar to the current ones
+```
+make serve run-serve
+make shell run-shell
+make ginkgo
+```
+
+### Docs 
+In troubleshoot.sh docs, add a new sub section under "Support Bundle" describing then new serve & shell subcommands.
+
+### Other Changes
+We would need to merge some of the support bundle functionality we see in both projects like so
+
+- Create a new sbutil module to store reusable support bundle functionality. At least `analyze` & `serve` modules will depend on this module
+
+```
+pkg
+    sbutil/
+```
+
+- Merge `troubleshoot/pkg/analyze/download.ExtractTroubleshootBundle` & `sbctl/pkg/sbctl/support-bundle.ExtractBundle` that duplicate support bundle archive extraction
+- Build on `sbctl/pkg/sbctl/support-bundle.ClusterData` to have it expose well known paths (ClusterResourcesDir, Version files, analysis results etc) and unmarshalled resources (this bit can be built on a need to have basis) in a support bundle especially once [Add generic kubernetes resource analyzer #780 PR](https://github.com/replicatedhq/troubleshoot/pull/780) is merged.
+- Build on `sbctl/pkg/sbctl/support-bundle.FindClusterData` to perform pre-validation of a support bundle to ensure it is valid before running any applications.
+
+## Limitations
+
+## Assumptions
+
+## Testing
+We will need to have a separate directory for sbctl (ginkgo BDD) tests. The test harness used is ginkgo cli instead of go test and there is no straight forward way of separating these tests. Also, from a directory structure point of view, such integration tests usually reside in modules in a root level tests directory. Changes to the test code need to be minimal, ideally no change, cause they will be used to ensure any refactored code does not break. We could now easily introduce tests which ensure that changes to collectors don't break APIs in sbctl.
+
+## Alternatives Considered
+* Have troubleshoot as a library dependency in sbctl keeping it as a separate binary and repo
+* Move sbctl to the Troubleshoot library "as is" and continue maintaing a separate binary
+* Leave things as they are
+
+## Security Considerations