2.9.1

• Features
  • Introduce Slack notifications support for workflow running on Argo Workflows

Features

Introduce Slack notifications support for workflow running on Argo Workflows

With this release, Metaflow users can get notified on Slack when their workflows succeed or fail on Argo Workflows. Using this feature is quite straightforward

1. Follow these instructions on Slack to set up incoming webhooks for your Slack workspace.
2. You should now have a webhook URL that Slack provides. Here is an example webhook:

   https://hooks.slack.com/services/T0XXXXXXXXX/B0XXXXXXXXX/qZXXXXXX
   

3. To enable notifications on Slack when your Metaflow flow running on Argo Workflows succeeds or fails, deploy it using the --notify-on-error or --notify-on-success flags:

   python flow.py argo-workflows create --notify-on-error --notify-on-success --notify-slack-webhook-url <slack-webhook-url>
   

4. You can also set METAFLOW_ARGO_WORKFLOWS_CREATE_NOTIFY_SLACK_WEBHOOK_URL=<slack-webhook-url> in your environment instead of specifying --notify-slack-webhook-url on the CLI everytime.
5. Next time your workflow succeeds or fails on Argo Workflows, you will get a helpful notification on Slack.

FAQ

I deployed my workflow following the instructions above, but I haven’t received any notifications yet?

This issue may very well happen if you are running Kubernetes v1.24 or newer.

Since v1.24, Kubernetes stopped automatically creating a secret for every serviceAccount. Argo Workflows relies on the existence of these secrets to run lifecycle hooks responsible for the emission of these notifications.

Follow these steps for explicitly creating a secret for the service account that responsible for executing Argo Workflows steps:

1. Run the following command, replacing service-account.name with the serviceAccount in your deployment. Also change the name of the secret to correctly reflect the name of the _serviceAccount _for which this secret is

   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Secret
   metadata:
     name: default-sa-token #change according to the name of the sa
     annotations:
       kubernetes.io/service-account.name: default #replace with your sa
   type: kubernetes.io/service-account-token
   EOF
   

2. Edit the serviceAccount object so as to add the name of the above secret in it. You can use kubectl edit for this. The serviceAccount yaml should look like the following

   $ kubectl edit sa default -n mynamespace
   ...
   apiVersion: v1
   kind: ServiceAccount
   metadata:
     creationTimestamp: "2023-05-05T20:58:58Z"
     name: default
     namespace: jobs-default
     resourceVersion: "6739507"
     uid: 4a708eff-d6ba-4dd8-80ee-8fb3c4c1e1c7
   secrets:
   - name: default-sa-token # should match the secret above
   

3. That’s it! Try executing your workflow again on Argo Workflows. If you are still running into issues, reach out to us!

In case you need any assistance or have feedback for us, ping us at chat.metaflow.org or open a GitHub issue.

---

What's Changed

• feature: add argo events environment variables to metaflow configure kubernetes by @saikonen in #1405
• handle whitespaces in argo events parameters by @savingoyal in #1408
• Add back comment for argo workflows by @savingoyal in #1409
• Support ArgoEvent object with @kubernetes by @savingoyal in #1410
• Print workflow template location as part of argo-workflows create by @savingoyal in #1411

Full Changelog: 2.8.6...2.9.0