diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2487385..b1d6b26 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,7 +26,7 @@ jobs: strategy: matrix: - python-version: [ 3.7, 3.8, 3.9 ] + python-version: [ "3.10", "3.11", "3.12", "3.13" ] steps: diff --git a/DEVELOP.md b/DEVELOP.md index f14c870..d637235 100644 --- a/DEVELOP.md +++ b/DEVELOP.md @@ -6,8 +6,8 @@ These instructions are for contributing to this repository. See main README.md f ```bash cd src/docker -docker-compose build -docker-compose up +docker compose build +docker compose up ``` ## Test diff --git a/README.md b/README.md index 9557f54..bc4751c 100644 --- a/README.md +++ b/README.md @@ -24,9 +24,9 @@ This open source effort puts together patterns the Graphistry team has reused ac ## What -* **Minimal core**: The barebones dashboard server. In provides a StreamLit docker-compose container with PyData ecosystem libraries and examples of visualizing data from various systems. Install it, plug in credentials to various web services like cloud databases and a free [Graphistry Hub](https://hub.graphistry.com) visualization account, and launch. It does not have GPU ETL and GPU AI libraries. +* **Minimal core**: The barebones dashboard server. In provides a StreamLit docker compose container with PyData ecosystem libraries and examples of visualizing data from various systems. Install it, plug in credentials to various web services like cloud databases and a free [Graphistry Hub](https://hub.graphistry.com) visualization account, and launch. It does not have GPU ETL and GPU AI libraries. -* **Full core**: Initially for AWS, the full core bundles adds to the docker-compose system: Accounts, Jupyter notebooks for authoring, serves StreamLit dashboards with both public + private zones, and runs Graphistry/RAPIDS locally on the same server. Launch with on click via the Cloud Formation template. +* **Full core**: Initially for AWS, the full core bundles adds to the docker compose system: Accounts, Jupyter notebooks for authoring, serves StreamLit dashboards with both public + private zones, and runs Graphistry/RAPIDS locally on the same server. Launch with on click via the Cloud Formation template. * **Full core + DB**: DB-specific variants are the same as minimal/full, and add simpler DB-specific quick launching/connecting. @@ -36,7 +36,7 @@ This open source effort puts together patterns the Graphistry team has reused ac **Note**: Base image includes Nvidia RAPIDS and AI dependencies so is quite large, see CPU alternative for a lightweight alternativve -**Note**: Use `sudo` for docker-compose commands if your configuration requires it and is giving permission error +**Note**: Use `sudo` for docker compose commands if your configuration requires it and is giving permission error ```bash # Minimal core @@ -44,18 +44,18 @@ git clone https://github.com/graphistry/graph-app-kit.git cd graph-app-kit/src/docker # Enable docker buildkit -# ... or run docker-compose via provided alias script `./dc` +# ... or run docker compose via provided alias script `./dc` export DOCKER_BUILDKIT=1 export COMPOSE_DOCKER_CLI_BUILD=1 # Build -docker-compose build +docker compose build # Optional: Edit src/docker/.env (API accounts), docker-compose.yml: Auth, ports, ... # Launch -docker-compose up -d -docker-compose logs -f -t --tail=100 +docker compose up -d +docker compose logs -f -t --tail=100 ``` => `http://localhost:8501/` @@ -65,7 +65,7 @@ To [add views](docs/views.md) and relaunch: ```bash # Add dashboards @ src/python/views//__init__.py -docker-compose up -d --force-recreate +docker compose up -d --force-recreate ``` ### Quick (Local code) - minimal CPU core + third-party connectors @@ -102,14 +102,14 @@ sudo docker ps sudo docker logs -f -t --tail=1 MY_CONTAINER # restart a graphistry container -cd graphistry && sudo docker-compose restart MY_CONTAINER +cd graphistry && sudo docker compose restart MY_CONTAINER # restart caddy (Caddy 1 override) -cd graphistry && sudo docker-compose -f docker-compose.gak.graphistry.yml up -d caddy +cd graphistry && sudo docker compose -f docker-compose.gak.graphistry.yml up -d caddy # run streamlit -cd graph-app-kit/public/graph-app-kit && docker-compose -p pub run -d --name streamlit-pub streamlit -cd graph-app-kit/private/graph-app-kit && docker-compose -p priv run -d --name streamlit-priv streamlit +cd graph-app-kit/public/graph-app-kit && docker compose -p pub run -d --name streamlit-pub streamlit +cd graph-app-kit/private/graph-app-kit && docker compose -p priv run -d --name streamlit-priv streamlit ``` **Minimal**: Open Streamlit, ssh to connect/add [free Graphistry Hub username/pass](https://www.graphistry.com/get-started): @@ -140,7 +140,7 @@ VIEW_PATH="`pwd`/views" streamlit run entrypoint.py * [Graphistry](https://www.graphistry.com/get-started) point-and-click GPU-accelerated visual graph analytics * Data frames: Data wrangling via [Pandas](https://pandas.pydata.org/)and [Apache Arrow](https://arrow.apache.org/), including handling formats such as CSV, XLS, JSON, Parquet, and more -* Standard Docker and docker-compose cross-platform deployment +* Standard Docker and docker compose cross-platform deployment ### GPU acceleration (optional) - Full diff --git a/docs/extend.md b/docs/extend.md index 1be28f6..b23ee91 100644 --- a/docs/extend.md +++ b/docs/extend.md @@ -8,7 +8,7 @@ Integration settings that deal with external systems such as TLS, accounts, and By default, `graph-app-kit` logging uses the Docker json file driver: -* Inspect recent activities: `cd src/docker` and then `sudo docker-compose logs -f -t --tail=100` +* Inspect recent activities: `cd src/docker` and then `sudo docker compose logs -f -t --tail=100` * Setup [alternative logging drivers](https://docs.docker.com/config/containers/logging/configure/) @@ -24,7 +24,7 @@ By default, `graph-app-kit` logging uses the Docker json file driver: ## TLS with Cadddy -* Auth: See [Caddy sample](src/caddy/Caddyfile) reverse proxy example for an authentication check against an account system, including the one shipping with your Graphistry server (requires `sudo docker-compose restart caddy` in your Graphistry server upon editing `/var/graphistry/data/config/Caddyfile`) +* Auth: See [Caddy sample](src/caddy/Caddyfile) reverse proxy example for an authentication check against an account system, including the one shipping with your Graphistry server (requires `sudo docker compose restart caddy` in your Graphistry server upon editing `/var/graphistry/data/config/Caddyfile`) ## Public+Private views * To simulatenously run 1 public and 1 private instance, create two `graph-app-kit` clones `public_dash` and `private_dash`, and for `src/docker/.env`, set: diff --git a/docs/neptune-manual.md b/docs/neptune-manual.md index d665bfa..8a8cbec 100644 --- a/docs/neptune-manual.md +++ b/docs/neptune-manual.md @@ -40,11 +40,11 @@ Reset and restart your `graph-app-kit` container: ```bash cd src/docker -sudo docker-compose down -v -sudo docker-compose up -d +sudo docker compose down -v +sudo docker compose up -d ``` -Watch logs with `sudo docker-compose logs -f -t --tail=1` +Watch logs with `sudo docker compose logs -f -t --tail=1` Access your Streamlit instance at http://the.public.ip.address:8501 diff --git a/docs/neptune.md b/docs/neptune.md index 19714c3..887f761 100644 --- a/docs/neptune.md +++ b/docs/neptune.md @@ -112,11 +112,11 @@ Launch using a button at the bottom of the [identity graph sample cloud formatio * **Streamlit: Public dashboards** * **http://[the.public.ip.address]/public/dash** * Installed at `/home/ubuntu/graph-app-kit/public/graph-app-kit` - * Run as `src/docker $ docker-compose -p pub run -d --name streamlit-pub streamlit` + * Run as `src/docker $ docker compose -p pub run -d --name streamlit-pub streamlit` * **Streamlit: Private dashboards** * **http://[the.public.ip.address]/private/dash** * Installed at `/home/ubuntu/graph-app-kit/private/graph-app-kit` - * Run as `src/docker $ docker-compose -p priv run -d --name streamlit-priv streamlit` + * Run as `src/docker $ docker compose -p priv run -d --name streamlit-priv streamlit` * **Jupyter: Data science notebooks + Streamlit dashboard live-editing** * **http://[the.public.ip.address]/notebook** * Live-edit `graph-app-kit` view folders `notebook/graph-app-kit/[public,private]/views` @@ -126,7 +126,7 @@ Launch using a button at the bottom of the [identity graph sample cloud formatio * **Streamlit: Public dashboards** * **http://[the.public.ip.address]/public/dash** * Installed at `/home/ubuntu/graph-app-kit/public/graph-app-kit` - * Run as `src/docker $ docker-compose up -d streamlit` + * Run as `src/docker $ docker compose up -d streamlit` ## 4. Next steps diff --git a/docs/setup-manual.md b/docs/setup-manual.md index 58602f5..b0d2c3e 100644 --- a/docs/setup-manual.md +++ b/docs/setup-manual.md @@ -10,7 +10,7 @@ For quick launchers, see the [AWS quick launch setup guide](setup.md). * Ubuntu 18.04 LTS is the most common choice for containerized GPU computing -* Install docker-ce and docker-compose +* Install docker-ce and docker compose * Optional: * GPU: If you have a [RAPIDS.ai](https://www.rapids.ai)-compatible GPU (see below), install the Nvidia docker runtime and set it as the default for Docker daemons @@ -57,9 +57,9 @@ GRAPHISTRY_PASSWORD=your_password `cd src/docker` and then: -* Start: `sudo docker-compose up -d` +* Start: `sudo docker compose up -d` * Use: Go to `http://localhost:8501/dashboard` (or whatever the public IP) -* Stop: `sudo docker-compose down -v` +* Stop: `sudo docker compose down -v` ## Graph! diff --git a/docs/setup.md b/docs/setup.md index 574bebd..9fad312 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -85,11 +85,11 @@ Go to your public Streamlit dashboard and start exploring: http://[the.public.ip * **Streamlit: Public dashboards** * **http://[the.public.ip.address]/public/dash** * Installed at `/home/ubuntu/graph-app-kit/public/graph-app-kit` - * Run as `src/docker $ docker-compose -p pub run -d --name streamlit-pub streamlit` + * Run as `src/docker $ docker compose -p pub run -d --name streamlit-pub streamlit` * **Streamlit: Private dashboards** * **http://[the.public.ip.address]/private/dash** * Installed at `/home/ubuntu/graph-app-kit/private/graph-app-kit` - * Run as `src/docker $ docker-compose -p priv run -d --name streamlit-priv streamlit` + * Run as `src/docker $ docker compose -p priv run -d --name streamlit-priv streamlit` * **Jupyter: Data science notebooks + Streamlit dashboard live-editing** * **http://[the.public.ip.address]/notebook** * Live-edit `graph-app-kit` view folders `notebook/graph-app-kit/[public,private]/views` @@ -99,7 +99,7 @@ Go to your public Streamlit dashboard and start exploring: http://[the.public.ip * **Streamlit: Public dashboards** * **http://[the.public.ip.address]/public/dash** * Installed at `/home/ubuntu/graph-app-kit/public/graph-app-kit` - * Run as `src/docker $ docker-compose up -d streamlit` + * Run as `src/docker $ docker compose up -d streamlit` ## 4. Optional - administer @@ -127,19 +127,19 @@ For more advanced Graphistry administration, so the [Graphistry admin docs repo] ```bash # restart a graphistry container -cd graphistry && sudo docker-compose restart MY_CONTAINER # do *not* run it's caddy (v2) +cd graphistry && sudo docker compose restart MY_CONTAINER # do *not* run it's caddy (v2) # restart caddy (Caddy 1 override over Graphistry's Caddy 2) -cd graphistry && sudo docker-compose -f docker-compose.gak.graphistry.yml up -d caddy +cd graphistry && sudo docker compose -f docker-compose.gak.graphistry.yml up -d caddy ``` ### Streamlit -Use `docker-compose` project names (`-p the_name`) to distinguish your public vs private dashboards: +Use `docker compose` project names (`-p the_name`) to distinguish your public vs private dashboards: ```bash -cd graph-app-kit/public/graph-app-kit && docker-compose -p pub run -d --name streamlit-pub streamlit -cd graph-app-kit/private/graph-app-kit && docker-compose -p priv run -d --name streamlit-priv streamlit +cd graph-app-kit/public/graph-app-kit && docker compose -p pub run -d --name streamlit-pub streamlit +cd graph-app-kit/private/graph-app-kit && docker compose -p priv run -d --name streamlit-priv streamlit ``` ## 5. Next steps diff --git a/docs/tigergraph.md b/docs/tigergraph.md index a1ab0b2..247667f 100644 --- a/docs/tigergraph.md +++ b/docs/tigergraph.md @@ -43,7 +43,7 @@ TIGERGRAPH_GRAPHNAME=AntiFraud TIGERGRAPH_SECRET=mysecret ``` -3. Restart your Streamlit container with the new creds: `cd src/docker && sudo docker-compose up -d --force-recreate` +3. Restart your Streamlit container with the new creds: `cd src/docker && sudo docker compose up -d --force-recreate` ## Explore diff --git a/docs/views.md b/docs/views.md index 8849ee3..03886a6 100644 --- a/docs/views.md +++ b/docs/views.md @@ -10,7 +10,7 @@ Not required but recommended, we recommend setting up Jupyter-based shared web a * Modify Python files in `src/python/views/[your dashboard]/__init__.py`, and in-tool, hit the `rerun` button that appears * Add new views by adding `views/[your dsahboard]/__init__.py` with methods `def info(): return {'name': 'x'}` and `def run(): None` -* Add new dependencies: modify `src/python/requirements-app.txt` and rerun `docker-compose build` and restart +* Add new dependencies: modify `src/python/requirements-app.txt` and rerun `docker compose build` and restart ## Toggle views diff --git a/src/bootstraps/core/graphistry.sh b/src/bootstraps/core/graphistry.sh index 1ffa2cf..98ad264 100755 --- a/src/bootstraps/core/graphistry.sh +++ b/src/bootstraps/core/graphistry.sh @@ -58,11 +58,11 @@ echo '----- Finish pub vs. priv .env specialization' echo '----- Launching graph-app-kit as streamlit-pub/priv:8501' ( \ cd "${GAK_PUBLIC}/src/docker" \ - && sudo docker-compose -p pub run -d --name streamlit-pub streamlit \ + && sudo docker compose -p pub run -d --name streamlit-pub streamlit \ ) ( \ cd "${GAK_PRIVATE}/src/docker" \ - && sudo docker-compose -p priv run -d --name streamlit-priv streamlit \ + && sudo docker compose -p priv run -d --name streamlit-priv streamlit \ ) ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/bootstraps/neptune/graphistry.sh b/src/bootstraps/neptune/graphistry.sh index 91b2b3d..4f63a9c 100755 --- a/src/bootstraps/neptune/graphistry.sh +++ b/src/bootstraps/neptune/graphistry.sh @@ -58,11 +58,11 @@ echo "BASE_PATH=private/dash/" | sudo tee -a "${GAK_PRIVATE}/src/docker/.env" echo '----- Launching graph-app-kit as streamlit-pub/priv:8501' ( \ cd "${GAK_PUBLIC}/src/docker" \ - && sudo docker-compose -p pub run -d --name streamlit-pub streamlit \ + && sudo docker compose -p pub run -d --name streamlit-pub streamlit \ ) ( \ cd "${GAK_PRIVATE}/src/docker" \ - && sudo docker-compose -p priv run -d --name streamlit-priv streamlit \ + && sudo docker compose -p priv run -d --name streamlit-priv streamlit \ ) ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/bootstraps/neptune/minimal.sh b/src/bootstraps/neptune/minimal.sh index a751b40..3d01f9e 100755 --- a/src/bootstraps/neptune/minimal.sh +++ b/src/bootstraps/neptune/minimal.sh @@ -49,7 +49,7 @@ echo '===== Configuring graph-app-kit with Graphistry account and Neptune =====' echo '----- Launching graph-app-kit as streamlit-pub:8501' ( \ cd "${GAK_PUBLIC}/src/docker" \ - && sudo /usr/local/bin/docker-compose up -d streamlit \ + && sudo /usr/local/bin/docker compose up -d streamlit \ ) ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/bootstraps/scripts/docker-aws.sh b/src/bootstraps/scripts/docker-aws.sh index 3678f7d..d938e2f 100755 --- a/src/bootstraps/scripts/docker-aws.sh +++ b/src/bootstraps/scripts/docker-aws.sh @@ -11,6 +11,6 @@ sudo chkconfig docker on sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose -docker-compose version +docker compose-version ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/bootstraps/scripts/docker-container-build.sh b/src/bootstraps/scripts/docker-container-build.sh index 3699f50..21089cc 100755 --- a/src/bootstraps/scripts/docker-container-build.sh +++ b/src/bootstraps/scripts/docker-container-build.sh @@ -2,7 +2,7 @@ set -ex SCRIPT="Build graph-app-kit docker" -DC_ALIAS=${DC_ALIAS:-docker-compose} +DC_ALIAS=${DC_ALIAS:-docker compose} ./hello-start.sh "$SCRIPT" ( cd ../../docker && $DC_ALIAS build ) diff --git a/src/bootstraps/scripts/graphistry-service-account.sh b/src/bootstraps/scripts/graphistry-service-account.sh index ce272fa..81fd1e7 100755 --- a/src/bootstraps/scripts/graphistry-service-account.sh +++ b/src/bootstraps/scripts/graphistry-service-account.sh @@ -10,7 +10,7 @@ SCRIPT="Get AWS Instance ID" cd "${GRAPHISTRY_HOME}" \ && ( \ until ( curl -fsS http://localhost/streamgl-gpu/secondary/gpu/health > /dev/null ); \ - do ( docker-compose ps && sleep 1 ); \ + do ( docker compose ps && sleep 1 ); \ done \ ) ) @@ -24,7 +24,7 @@ POST_SCRIPT="CELERY_BROKER_URL=zz python manage.py shell && echo done || { echo ( \ cd "${GRAPHISTRY_HOME}" \ - && docker-compose exec -T nexus \ + && docker compose exec -T nexus \ bash -c \ "source activate rapids && echo \"${ADD_USER_SCRIPT}; ${VERIFY_USER_SCRIPT}\" | ${POST_SCRIPT}" \ ) diff --git a/src/bootstraps/scripts/graphistry-wait-healthy.sh b/src/bootstraps/scripts/graphistry-wait-healthy.sh index 2d29bec..0dec80e 100755 --- a/src/bootstraps/scripts/graphistry-wait-healthy.sh +++ b/src/bootstraps/scripts/graphistry-wait-healthy.sh @@ -7,13 +7,13 @@ SCRIPT="Wait Graphistry docker containers healthy" ( \ cd "${GRAPHISTRY_HOME}" \ - && for i in `sudo docker-compose ps --services`; do ( \ + && for i in `sudo docker compose ps --services`; do ( \ ( \ until ( \ [[ '"healthy"' == $(sudo docker inspect "graphistry_${i}_1" --format "{{json .State.Health.Status}}") ]] \ ); do ( \ echo "waiting on $i (5s)" \ - && sudo docker-compose ps \ + && sudo docker compose ps \ && sleep 5 \ ); done \ ) && echo "healthy $i" \ @@ -21,6 +21,6 @@ SCRIPT="Wait Graphistry docker containers healthy" ) echo "--- Graphistry status after healthy waiting ---" -( cd "${GRAPHISTRY_HOME}" && sudo docker-compose ps ) +( cd "${GRAPHISTRY_HOME}" && sudo docker compose ps ) ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/bootstraps/scripts/swap-caddy.sh b/src/bootstraps/scripts/swap-caddy.sh index 30b6909..94b466b 100755 --- a/src/bootstraps/scripts/swap-caddy.sh +++ b/src/bootstraps/scripts/swap-caddy.sh @@ -2,7 +2,7 @@ set -ex ## Graphistry now uses Caddy2, and until Caddy 2 supports simple auth, g-a-k still sticks with Caddy1 -## ... So we stop Graphistry's Caddy 2 and put in a docker-compose of its old +## ... So we stop Graphistry's Caddy 2 and put in a docker compose of its old CADDY_FILENAME=${CADDY_FILENAME:-full.Caddyfile} CADDY_COMPOSE_FILENAME=${CADDY_COMPOSE_FILENAME:-docker-compose.gak.graphistry.yml} @@ -16,10 +16,10 @@ sudo cp "../../caddy/${CADDY_COMPOSE_FILENAME}" "${GRAPHISTRY_HOME}/${CADDY_COMP sudo cp "../../caddy/${CADDY_FILENAME}" "${GRAPHISTRY_HOME}/data/config/Caddyfile" ( \ cd "${GRAPHISTRY_HOME}" \ - && sudo docker-compose stop caddy \ - && sudo docker-compose -f "${CADDY_COMPOSE_FILENAME}" up -d caddy \ - && sudo docker-compose ps \ - && sudo docker-compose -f "${CADDY_COMPOSE_FILENAME}" ps \ + && sudo docker compose stop caddy \ + && sudo docker compose -f "${CADDY_COMPOSE_FILENAME}" up -d caddy \ + && sudo docker compose ps \ + && sudo docker compose -f "${CADDY_COMPOSE_FILENAME}" ps \ ) ./hello-end.sh "$SCRIPT" \ No newline at end of file diff --git a/src/caddy/Caddyfile b/src/caddy/Caddyfile index a5c3f79..0462927 100644 --- a/src/caddy/Caddyfile +++ b/src/caddy/Caddyfile @@ -8,7 +8,7 @@ ### # ### -- Relaunch: # ### # -### docker-compose stop caddy && docker-compose up -d # +### docker compose stop caddy && docker compose up -d # ### # ########################################################################################## diff --git a/src/caddy/docker-compose.gak.graphistry.yml b/src/caddy/docker-compose.gak.graphistry.yml index f06de01..22c31f9 100644 --- a/src/caddy/docker-compose.gak.graphistry.yml +++ b/src/caddy/docker-compose.gak.graphistry.yml @@ -1,4 +1,4 @@ -#Reuse graphistry's docker-compose setup.. except downgrade to caddy1 for auth support +#Reuse graphistry's docker compose setup.. except downgrade to caddy1 for auth support #Start _after_ graphistry version: '3.5' diff --git a/src/docker/docker-compose.yml b/src/docker/docker-compose.yml index a74018d..019202f 100644 --- a/src/docker/docker-compose.yml +++ b/src/docker/docker-compose.yml @@ -14,7 +14,7 @@ networks: name: grph_net #----------------------------------------------------------- -# Ex: Reuse network from another docker-compose +# Ex: Reuse network from another docker compose #----------------------------------------------------------- #networks: # grph_net: diff --git a/src/docker/hooks/build b/src/docker/hooks/build index 3fb181d..7a4b504 100755 --- a/src/docker/hooks/build +++ b/src/docker/hooks/build @@ -9,4 +9,4 @@ echo "pwd: `pwd`" echo "DOCKER_TAG: ${DOCKER_TAG}" # => src/docker echo "IMAGE_NAME: ${IMAGE_NAME}" # => graphistry/graph-app-kit-st -docker-compose -f docker-compose.yml -f override/cpu.override.yml build \ No newline at end of file +docker compose -f docker-compose.yml -f override/cpu.override.yml build \ No newline at end of file diff --git a/src/docker/override/docker-compose.override.yml b/src/docker/override/docker-compose.override.yml index 37bd61a..54397f9 100644 --- a/src/docker/override/docker-compose.override.yml +++ b/src/docker/override/docker-compose.override.yml @@ -14,7 +14,7 @@ version: "3.5" # # graphistry$ ln -s ../graph-app-kit/src/docker/override/docker-compose.override.yml docker-compose.override.yml # graphistry$ ln -s "`pwd`/../graph-app-kit/" data/config/graph-app-kit -# graphistry$ docker-compose up -d +# graphistry$ docker compose up -d # ######################################################################################################################## #