Skip to content

Commit

Permalink
v1.0.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
nxBroome committed May 16, 2023
1 parent 8fc05c6 commit 7c1702b
Show file tree
Hide file tree
Showing 548 changed files with 67,291 additions and 23 deletions.
Binary file added .github/images/actions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/approve-pr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/logs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/new-branch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/new-pr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/run_wf.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/images/workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
57 changes: 57 additions & 0 deletions .github/workflows/delete_old_workflows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Delete old workflow runs
on:
workflow_dispatch:
inputs:
days:
description: 'Number of days.'
required: true
default: 30
minimum_runs:
description: 'The minimum runs to keep for each workflow.'
required: true
default: 6
delete_workflow_pattern:
description: 'The name or filename of the workflow. if not set then it will target all workflows.'
required: false
delete_workflow_by_state_pattern:
description: 'Remove workflow by state: active, deleted, disabled_fork, disabled_inactivity, disabled_manually'
required: true
default: "All"
type: choice
options:
- "All"
- active
- deleted
- disabled_inactivity
- disabled_manually
delete_run_by_conclusion_pattern:
description: 'Remove workflow by conclusion: action_required, cancelled, failure, skipped, success'
required: true
default: "All"
type: choice
options:
- "All"
- action_required
- cancelled
- failure
- skipped
- success
dry_run:
description: 'Only log actions, do not perform any delete operations.'
required: false

jobs:
del_runs:
runs-on: ubuntu-latest
steps:
- name: Delete workflow runs
uses: Mattraks/delete-workflow-runs@v2
with:
token: ${{ github.token }}
repository: ${{ github.repository }}
retain_days: ${{ github.event.inputs.days }}
keep_minimum_runs: ${{ github.event.inputs.minimum_runs }}
delete_workflow_pattern: ${{ github.event.inputs.delete_workflow_pattern }}
delete_workflow_by_state_pattern: ${{ github.event.inputs.delete_workflow_by_state_pattern }}
delete_run_by_conclusion_pattern: ${{ github.event.inputs.delete_run_by_conclusion_pattern }}
dry_run: ${{ github.event.inputs.dry_run }}
39 changes: 39 additions & 0 deletions .github/workflows/trigger-github-actions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Trigger Github Actions

on:
pull_request_review:
types: [submitted]

env:
GH_TOKEN: ${{ github.token }}
BRANCH_NAME: ${{ github.event.pull_request.head.ref }}

jobs:
call-python-workflow:
if: ${{ github.event.review.state == 'approved' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: tj-actions/changed-files@v35
id: changed_files
with:
json: "true"
- name: Print changed files
run: |
echo "Changed files: ${{ steps.changed_files.outputs.all_changed_files }}"
- name: Call imaging if new-site config file changed
if: contains(steps.changed_files.outputs.all_changed_files, 'config/new-site.json')
run: |
gh workflow run trigger-python-workflows.yml -f file=config/new-site.json -f workflow=imaging -f branch=${{ env.BRANCH_NAME }}
- name: Call cluster-config if pod-config file changed
if: contains(steps.changed_files.outputs.all_changed_files, 'config/pod-config.yml')
run: |
gh workflow run trigger-python-workflows.yml -f file=config/pod-config.yml -f workflow=pod-config -f branch=${{ env.BRANCH_NAME }}
- name: Call vm-workloads if create_vm_workloads file changed
if: contains(steps.changed_files.outputs.all_changed_files, 'config/create-vm-workloads.json')
run: |
gh workflow run trigger-python-workflows.yml -f file=config/create_vm_workloads.json -f workflow=create-vm-workloads -f branch=${{ env.BRANCH_NAME }}
- name: Call edge-ai-workloads if edge_ai file changed
if: contains(steps.changed_files.outputs.all_changed_files, 'config/edge-ai.json')
run: |
gh workflow run trigger-python-workflows.yml -f file=config/edge-ai.json -f workflow=calm-edgeai-vm-workload -f branch=${{ env.BRANCH_NAME }}
76 changes: 76 additions & 0 deletions .github/workflows/trigger-python-workflows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Trigger Python Workflows
on:
workflow_dispatch:
inputs:
file:
description: 'Input file'
required: true
branch:
description: 'Branch name where the config file resides'
required: true
workflow:
description: 'Workflow to run'
required: true
default: "pod-config"
type: choice
options:
- "pod-config"
- "imaging"
- "create-vm-workloads"
- "calm-edgeai-vm-workload"

jobs:
create-python-environment:
runs-on: self-hosted
steps:
- uses: actions/checkout@v3
with:
ref: ${{ github.event.inputs.branch }}
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
# gives you either a new virtualenv, or restores an old one based on
# the requirements*.txt -file.
- uses: syphar/restore-virtualenv@v1
id: cache-virtualenv
- uses: syphar/restore-pip-download-cache@v1
if: steps.cache-virtualenv.outputs.cache-hit != 'true'
# the package installation will only be executed when the
# requirements*.txt - file has changed.
- name: Install dependencies
run: |
python -m pip install --upgrade pip
cd framework
pip install -r requirements/dev.txt
if: steps.cache-virtualenv.outputs.cache-hit != 'true'
run-python-workflow:
needs: create-python-environment
runs-on: self-hosted
steps:
- uses: actions/checkout@v3
with:
ref: ${{ github.event.inputs.branch }}
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Restore cache
uses: syphar/restore-virtualenv@v1
id: cache-virtualenv
- name: Run python script
shell: bash
run: |
cd framework
python main.py --workflow ${{ github.event.inputs.workflow }} --file ${{ github.event.inputs.file }}
- name: Git add config and logs
run: |
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git add sites/*
git commit -a -m "Workflow trigger logs"
- name: Push the config and logs to the trigger branch
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ github.event.inputs.branch }}
97 changes: 97 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Mac OS
.DS_Store
.idea

# Development
publish_to_marketplace.log

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/

# Translations
*.mo
*.pot

# Django stuff
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# IPython Notebook
.ipynb_checkpoints

# pyenv
.python-version
pyvenv.cfg

# celery beat schedule file
celerybeat-schedule


# virtualenv
.venv/
venv/
ENV/

# Spyder project settings
.spyderproject

# Rope project settings
.ropeproject

# others
framework/script_log.log
11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
dev:
# Setup our python based virtualenv
# This step assumes python3 is installed on your dev machine as python
[ -f venv/bin/python ] || (python -m venv venv && \
venv/bin/pip install --upgrade pip setuptools)
cd framework && \
../venv/bin/pip install --no-cache -r requirements/dev.txt

test: dev
# In progress
pytest
68 changes: 45 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,39 @@
# Zero Touch Framework

A tool used to automate end to end deployment and configuration of Nutanix Cloud Platform without human intervention, hence the name Zero Touch. The tool can also be extended to manage Day-1 and Day-2 operations as well.
A tool used to automate end to end deployment and configuration of Nutanix Cloud Platform without human intervention,
hence the name Zero Touch. The tool can also be extended to manage Day-1 and Day-2 operations as well.

## Usage

### Prerequisites
- Foundation Central configured such that it can access the networks nodes are discoverable on.
- For how to set up Foundation Central and provide the API key via your DHCP server, refer to the [Foundation Central Guide](https://portal.nutanix.com/page/documents/details?targetId=Foundation-Central-v1_5:Foundation-Central-v1_5)
- GitHub Runner in the environment to run the workflows locally.

This tool can be used in two (2) modes:
1. **GitOps**
>This tool works as GitOps, i.e using this repository to manage Infrastructure as a Code (IaaC). Each operation is defined as a workflow, and these workflows can be directly triggered from Github through various actions.
Click [here](config/README.md) to read more about triggering Github pipelines.

2. **Dev Mode**
>This tool can also be setup locally in any popular OS and trigger the workflows manually. Look at the below section for Dev Setup.
1. **Dev Mode**
> This tool can also be setup locally in any popular OS and trigger the workflows manually. Look at the below section
for Dev Setup.

2. **GitOps**
> This tool works as GitOps, i.e using this repository to manage Infrastructure as a Code (IaaC). Each operation is
defined as a workflow, and these workflows can be directly triggered from Github through various actions.
Click [here](config/README.md) to read more about triggering Github pipelines.


## Dev Mode Setup

- Ensure Python version >= 3.10 (You can use [pyenv](https://realpython.com/intro-to-pyenv/) to manage multiple Python versions)
- If you are using pyenv, run this inside the project to use a specific version of Python.
```sh
> pyenv local 3.10.8
```
- `make dev` to create/ use python venv (virtual environment) in $TOPDIR/venv and setup dev environment. Activate it by calling source `venv/bin/activate`. Use deactivate to `deactivate` the venv.
- Ensure Python version >= 3.10 (You can use [pyenv](https://realpython.com/intro-to-pyenv/) to manage multiple Python
versions)
- If you are using pyenv, run this inside the project to use a specific version of Python.
```sh
> pyenv local 3.10.8
```
- `make dev` to create/ use python venv (virtual environment) in $TOPDIR/venv and setup dev environment. Activate it by
calling source `venv/bin/activate`. Use deactivate to `deactivate` the venv.

## Framework Usage
### Framework Usage

```sh
> cd framework && python main.py --help
Expand All @@ -36,28 +48,38 @@ options:
--debug
```

## Existing Workflows and Input Files

As you see above, the framework expects two parameters, `WORKFLOW` and `FILE`.
### Existing Workflows and Input Files

As mentioned above, the framework expects two parameters, `WORKFLOW` and `FILE`.

The framework is designed to be triggered by different **_functional workflows_**. These workflows call one or many scripts behind the scenes to accomplish the task.
The framework is designed to be triggered by different **_functional workflows_**. These workflows call one or many
scripts behind the scenes to accomplish the task.
The supported workflows are:

- `imaging` - This will trigger the Imaging of nodes using Foundation Central and creates Cluster/s.
Click [here](#prerequisites-for-imaging-workflow) to read prerequisite for imaging.
- `pod-config` - This will facilitate Pod configuration (including AZ and Cluster configurations) in parallel.
- `calm-vm-workloads` - This will use calm-dsl to create VM workloads in Self-Service from single or multiple calm-dsl file/s.
- `calm-edgeai-vm-workload` - This will use calm-dsl to create Edge-AI VM workload in Self-Service from single or multiple calm-dsl file/s.
- `calm-vm-workloads` - This will use calm-dsl to create VM workloads in Self-Service from single or multiple calm-dsl
file/s.
- `calm-edgeai-vm-workload` - This will use calm-dsl to create Edge-AI VM workload in Self-Service from single or
multiple calm-dsl file/s.

Along with the functional workflow, the tool also expects an input file to read the necessary configurations from. The
input files can either be a **json/ yaml** file.

Along with the functional workflow, the tool also expects an input file to read the necessary configurations from. The input files can either be a **json/ yaml** file.
- The global configurations reside in [global.json](config/global.json). The values defined here, will be inherited in all the functional workflows and can be overwritten in the corresponding input file/s.
- The global configurations reside in [global.json](config/global.json). The values defined here, will be inherited in
all the functional workflows and can be overwritten in the corresponding input file/s.
- For `imaging`, see the example input configuration [new-site.json](config/new-site.json).
- For `pod-config`, see the example input configuration [pod-config.yml](config/pod-config.yml).
- For `calm-vm-workloads`, see the example input configuration [create_vm_workloads.json](config/create-vm-workloads.json).
- For `calm-vm-workloads`, see the example input
configuration [create_vm_workloads.json](config/create-vm-workloads.json).
- For `calm-edgeai-vm-workload`, see the example input configuration [edge_ai.json](config/edge-ai.json).

## Trigger a workflow with an input file
### Trigger a workflow with an input file

```sh
> cd framework && python main.py --workflow pod-config -f config/pod-config.yml
```

> Note: The path to the file, should be defined relative to the root of the project, not to _framework_ directory
1 change: 1 addition & 0 deletions calm-dsl-bps/blueprints/LAMP/.local/MYSQL_PASSWORD
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
password
Loading

0 comments on commit 7c1702b

Please sign in to comment.