Introduce dedicated documentation

.github/workflows/docs.yml

@@ -0,0 +1,76 @@
+# Based on https://imfing.github.io/hextra/docs/guide/deploy-site/
+name: Deploy docs
+
+on:
+  # Runs on pushes targeting the default branch
+  push:
+    branches: ["main"]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
+# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+# Default to bash
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  # Build job
+  build:
+    runs-on: ubuntu-latest
+    env:
+      HUGO_VERSION: 0.138.0
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # fetch all history for .GitInfo and .Lastmod
+          submodules: recursive
+      - name: Setup Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: '1.22'
+      - name: Setup Pages
+        id: pages
+        uses: actions/configure-pages@v4
+      - name: Setup Hugo
+        run: |
+          wget -O ${{ runner.temp }}/hugo.deb https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_linux-amd64.deb \
+          && sudo dpkg -i ${{ runner.temp }}/hugo.deb          
+      - name: Build with Hugo
+        env:
+          # For maximum backward compatibility with Hugo modules
+          HUGO_ENVIRONMENT: production
+          HUGO_ENV: production
+        run: |
+          cd docs && hugo \
+            --gc --minify \
+            --baseURL "${{ steps.pages.outputs.base_url }}/"          
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: docs/public
+
+  # Deployment job
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -152,5 +152,4 @@ dev/cleanup.py
 
 .python-version
 .databricks-login.json
-*.out
-foo
+*.out

Makefile

@@ -31,4 +31,12 @@ known:
 solacc:
 	hatch run python tests/integration/source_code/solacc.py
 
-.PHONY: all clean dev lint fmt test integration coverage known solacc
+docs:
+	cd docs && hugo server --buildDrafts --disableFastRender
+
+docs-test:
+	@echo 'Make sure to run `make docs` in another terminal'
+	cd docs && yarn linkinator http://localhost:1313/ucx/
+
+
+.PHONY: all clean dev lint fmt test integration coverage known solacc docs docs-test

docs/.gitignore

@@ -0,0 +1,3 @@
+.hugo_build.lock
+node_modules/
+public/

docs/archetypes/default.md

@@ -0,0 +1,5 @@
+---
+date: '{{ .Date }}'
+draft: true
+title: '{{ replace .File.ContentBaseName "-" " " | title }}'
+---

docs/assets/css/custom.css

@@ -0,0 +1,22 @@
+@import url('https://fonts.googleapis.com/css2?family=DM+Mono:ital,wght@0,300;0,400;0,500;1,300;1,400;1,500&family=DM+Sans:ital,opsz,wght@0,9..40,100..1000;1,9..40,100..1000&display=swap');
+
+.font-mono {
+    font-family: "DM Mono", monospace;
+    font-style: normal;
+}
+
+.font-sans {
+    font-family: "DM Sans", sans-serif;
+    font-style: normal;
+}
+
+.content {
+    @apply font-sans;
+}
+
+
+:root {
+    --primary-hue: 2;
+    --primary-saturation: 98%;
+    --primary-lightness: 41.8%;
+}

docs/assets/css/tailwind.css

@@ -0,0 +1 @@
+@import "tailwindcss";

docs/content/_index.md

@@ -0,0 +1,7 @@
+---
+date: '2024-11-15T15:13:18+01:00'
+draft: true
+title: ''
+description: 'Just a placeholder, actual content is in hextra-home.html'
+layout: 'hextra-home'
+---

docs/content/docs/_index.md

@@ -0,0 +1,43 @@
+---
+linkTitle: "Documentation"
+title: Introduction
+---
+
+👋 Hello! Welcome to the UCX documentation!
+
+## What is UCX?
+
+UCX is a companion toolkit to assist users with upgrading to Unity Catalog (UC). It is provided in a form of CLI tool that can be used to automate the process of upgrading to UC.
+
+
+Usual steps to upgrade to UC include infra-level configuration, which is followed by tables and code migration.
+
+Infra-level configuration includes:
+- Setting up a UC metastore
+- Assess the current state of your workspaces by running UCX assessment
+- Attach workspaces to UC metastore
+- Migrate workspace-level groups to account-level groups
+
+Tables and code migration includes:
+- Setting up cloud-level infra for UC (storage credentials, external locations, volumes)
+- Migrate tables to UC catalogs
+- Adjust code to be compatible with UC-enabled compute
+- Switch clusters to UC-enabled compute
+- Change downstream consumers (e.g. BI tools) to use UC-enabled compute and UC catalogs
+
+
+## Questions or Feedback?
+
+{{< callout emoji="💻" type="info" >}}
+  UCX is still in active development.\
+  Have a question or feedback? Feel free to [open an issue](https://github.com/databrickslabs/ucx/issues)
+{{< /callout >}}
+
+## Next
+
+Dive right into the following section to get started:
+
+{{< cards >}}
+  {{< card link="installation" title="Installation" icon="document-text" subtitle="Learn how to install UCX" >}}
+  {{< card link="process/overview" title="Overview" icon="document-text" subtitle="Check out the migration process overview" >}}
+{{< /cards >}}

docs/content/docs/dev/_index.md

@@ -0,0 +1,15 @@
+---
+linkTitle: "Developer Documentation"
+title: Introduction into developing UCX
+weight: 5
+---
+
+The sections here are dedicated mainly to developers who are interested in contributing to UCX. If you are a user looking for information on how to use UCX, please refer to the [User Documentation](docs/).
+
+## Next
+
+Dive right into the following section to get started:
+{{< cards >}}
+  {{< card link="contributing" title="Contributing" icon="document-text" subtitle="Learn how to contribute to UCX" >}}
+  {{< card link="docs" title="Write docs" icon="document-text" subtitle="Learn how to write documentation for UCX" >}}
+{{< /cards >}}

CONTRIBUTING.md → docs/content/docs/dev/contributing.md

@@ -1,4 +1,8 @@
-# Contributing
+---
+linkTitle: "Contributing to UCX"
+title: "Contributing to UCX"
+weight: 1
+---
 
 ## First Principles
 
@@ -27,7 +31,7 @@ or specialized functionality unavailable in standard libraries.
 
 ## Adding entries to `known.json` file
 
-When adding a new entry to the [`known.json` file](src/databricks/labs/ucx/source_code/known.json), ensure that the entry is unique and follows the correct format. The `known.json` file
+When adding a new entry to the [`known.json` file](https://github.com/databrickslabs/ucx/blob/main/src/databricks/labs/ucx/source_code/known.json), ensure that the entry is unique and follows the correct format. The `known.json` file
 stores information about known (in)compatibilities with Unity Catalog. The file speeds up static code analysis and prevents false positives.
 
 1. load into virtual environment
@@ -127,7 +131,7 @@ for the latest reference of environment variables related to authentication.
   indicate the cloud provider being used, such as "aws" for Amazon Web Services and "azure" for Microsoft Azure.
 - `DATABRICKS_ACCOUNT_ID`: This variable stores the unique identifier for your Databricks account.
 - `DATABRICKS_HOST`: This variable contains the URL of your Databricks workspace. It is the web address you use to access
-  your Databricks environment and typically looks like "https://dbc-....cloud.databricks.com."
+  your Databricks environment and typically looks like `https://dbc-....cloud.databricks.com`.
 - `TEST_DEFAULT_CLUSTER_ID`: This variable holds the identifier for the default cluster used in testing. The value
   resembles a unique cluster ID, like "0824-163015-tdtagl1h."
 - `TEST_DEFAULT_WAREHOUSE_DATASOURCE_ID`: This environment variable stores the identifier for the default warehouse data
@@ -214,7 +218,7 @@ with the [Python plugin (Community Edition)](https://plugins.jetbrains.com/plugi
 IntelliJ CE, then it would work in PyCharm. Debugging capabilities are essential for troubleshooting and diagnosing issues during
 development. Please make sure that your test setup allows for easy debugging by following best practices.
 
-![debugging tests](docs/debugging-tests.gif)
+![debugging tests](/images/debugging-tests.gif)
 
 Adhering to these guidelines ensures that our integration tests are robust, efficient, and easily maintainable. This,
 in turn, contributes to the overall reliability and quality of our software.
@@ -263,7 +267,7 @@ HATCH_PYTHON="$(which python3.10)" make clean dev test
 ```
 
 Configure your IDE to use `.venv/bin/python` from the virtual environment when developing the project:
-![IDE Setup](docs/hatch-intellij.gif)
+![IDE Setup](/images/hatch-intellij.gif)
 
 
 Verify installation with