The Astronomer CLI can be used to build Airflow DAGs locally and run them via docker-compose, as well as to deploy those DAGs to Astronomer-managed Airflow clusters and interact with the Astronomer API in general.
astro is a command line interface for working with the Astronomer Platform.
Usage:
astro [command]
Available Commands:
auth Manage astronomer identity
cluster Manage Astronomer EE clusters
completion Generate autocompletions script for the specified shell (bash or zsh)
config Manage astro project configurations
deploy Deploy an airflow project
deployment Manage airflow deployments
dev Manage airflow projects
help Help about any command
upgrade Check for newer version of Astronomer CLI
user Manage astronomer user
version Astronomer CLI version
workspace Manage Astronomer workspaces
Flags:
-h, --help help for astro
Use "astro [command] --help" for more information about a command.
The Astronomer CLI is an open-source project. Installing it to your machine allows you to easily spin up a local instance of Apache Airflow and allows you to easily deploy code to remote Airflow environments if you are an Astronomer customer.
Note: If you are an Astronomer customer, your CLI version must match the version of Astronomer you are running. If you are using Astronomer Cloud, the latest version should always be correct. If you have a custom Astronomer Enterprise installation, you may be on a different version, which means you may need to ensure that your CLI and platform match up; you can check which version of Astronomer you're running by clicking the user icon in the top right corner of our UI.
brew install astronomer/tap/astroNote: If you are a Mac user on Catalina make sure you are using the latest version of
curlor you may receive certificate errors.
curl -sSL https://install.astronomer.io | sudo bash -s
To install a specific version Astro CLI use @major.minor.patch. For example, to install v0.13.1 run:
brew install astronomer/tap/astro@0.13.1To install a previous version of Astronomer, you can add the tag at the end of your curl command via the following syntax:
curl -sSL https://install.astronomer.io | sudo bash -s -- [TAGNAME]
ie:
curl -sSL https://install.astronomer.io | sudo bash -s -- v0.7.5
Note: If you get mkdir error during installation please download and run godownloader script locally.
$ cat godownloader.sh | bash -s -- -b /usr/local/bin
Note: Make sure you have Windows 10 and Docker installed
-
Download the latest release of the Astro CLI from this page
-
Extract the file ending in
windows_386.zipand copyastro.exesomewhere in your%PATH% -
Open cmd or PowerShell console and run the
astro versioncommand. Your output should look something like this:C:\Windows\system32>astro version Astro CLI Version: x.y.z Git Commit: 829e4702ca36dd725f1a98d82b6fdf889e5f4dc3
- Make sure you go through instruction to install Docker on windows properly https://docs.docker.com/docker-for-windows/install/
- Make sure you enabled Hyper-V, it's required for Docker and Linux Containers, also please review this document https://docs.docker.com/docker-for-windows/troubleshoot/
- Confirm the install worked:
$ astro
- Create a project:
$ mkdir hello-astro && cd hello-astro
$ astro dev init
This will generate a skeleton project directory:
.
├── dags
│ ├── example-dag.py
├── Dockerfile
├── include
├── packages.txt
├── plugins
└── requirements.txt
DAGs can go in the dags folder, custom Airflow plugins in plugins, python packages needed can go in requirements.txt, and OS level packages can go in packages.txt.
- Start airflow
Run astro dev start to start a local version of airflow on your machine. This will spin up a few locally running docker containers - one for the airflow scheduler, one for the webserver, and one for postgres.
(Run docker ps to verify)
The CLI includes a help command, descriptions, as well as usage info for subcommands.
To see the help overview:
$ astro help
Or for subcommands:
$ astro dev --help
$ astro deploy --help
How to get started as a developer:
- Build:
$ git clone git@github.com:astronomer/astro-cli.git
$ cd astro-cli
$ make build
- (Optional) Install to
$GOBIN:
$ make install
- Run:
$ astro
astro-cli is a single component of the much larger Astronomer Enterprise platform. In order to test locally you will need to
- setup both houston-api and astro-ui.
- edit your global or project config to enable local development
ex.
local:
enabled: true
houston: http://localhost:8871/v1
orbit: http://localhost:5000To run unit-tests you can run:
Note: Make sure you have running locally houston on http://localhost:8871/v1 it required for running tests
make testDocs (/docs) are generated using the github.com/spf13/cobra/doc pkg. Currently this pkg is broken with go vendoring, the following instructions include a workaround
- Remove the
/vendor/github.com/spf13/cobrapkg, forcing Go to search your go path for the package instead go run gendocs/gendocs.go- restore
/vendor/github.com/spf13/cobra
The Astronomer platform is under very active development. Because of this we cannot make backwards compatibility guarantees between versions.
The astro-cli is following a semantic versioning scheme
{MAJOR_RELEASE}.{MINOR_RELEASE}.{PATCH_RELEASE}
with all releases up until 1.0.0 considered beta.
astro-cli tightly tracks the platform release versioning, this means that compatibility is only guaranteed between matching minor versions of the platform and the astro-cli. ie. astro-cli v0.9.0 is guaranteed to be compatible with houston-api v0.9.x but not houston-api v0.10.x
astro-cli has a debug flag that allows to see queries and different helpful logs that are done internally. You can enable this by passing --verbosity=debug to your commands or you can edit your ~/.astro/config.yaml and add the following to it. This will turn on debug for all requests that are done until it is changed to info or removed from config.
verbosity: debugIf you need support, start with the Astronomer documentation.
If you still have questions, you can post on the Astronomer web forum or if you are a customer, you can open a support ticket.
Apache 2.0 with Commons Clause