You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/modules/demos/pages/airflow-scheduled-job.adoc
+27-2Lines changed: 27 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -153,7 +153,7 @@ Note that the connection to Kafka (`kafka_conn`) is expected: this has been crea
153
153
154
154
image::airflow-scheduled-job/airflow_12.png[]
155
155
156
-
We can use the kafka-producer script bundled with Kafka to write to this topic:
156
+
We can use the kafka-producer script bundled with Kafka to write to this topic (note the namespace we chose initially is used consistently in this demo):
The triggerer logs will show that this DAG was fired (logging out the message that we wrote to the topic above):
168
+
167
169
image::airflow-scheduled-job/airflow_13.png[]
168
170
171
+
The DAG view will show that run was successful.
172
+
Note the `kafka_queue_asset` under `Schedule`.
173
+
This is an Airflow object (also defined in the DAG code) that wraps the actual trigger/wait mechanism in an generic way for use in DAG code.
174
+
169
175
image::airflow-scheduled-job/airflow_14.png[]
170
176
177
+
Clicking on the asset will show the triggers that have been fired, called `Asset Events`:
178
+
171
179
image::airflow-scheduled-job/airflow_15.png[]
172
180
181
+
=== `core_deferrable_sleep_demo` DAG
182
+
183
+
Now log out of the UI. This next section will illustrate user authorization with the Open Policy Agent (OPA) and the OPA Authorizer that is included in Stackable's Airflow product images.
184
+
Two other users - other than `admin` - are supplied with this demo: `jane.doe` and `richard.roe` (the passwords are the same as the user names).
185
+
`jane.doe` has permission to view all DAGs but only has permission to activate and run `core_deferrable_sleep_demo`.
186
+
`richard.roe` does not have permission to view or action any DAGs.
187
+
Login in as `jane.doe`.
188
+
You will see the list of 4 DAGs as usual, and can enable `core_deferrable_sleep_demo` (the only DAG not automatically enabled when the demo was deployed).
189
+
But if you try and run any other DAG, permission will be denied:
190
+
173
191
image::airflow-scheduled-job/airflow_16.png[]
174
192
193
+
Enable and run `core_deferrable_sleep_demo`:
194
+
175
195
image::airflow-scheduled-job/airflow_17.png[]
176
196
197
+
Click on the DAG, switching to the task view.
198
+
This DAG uses a deferrable operator which, in conjunction with the triggerer process, "offloads" the DAG from its worker for a specfic period of time, before being picked up and again and executed.
199
+
You will see the task cycle through the following states:
0 commit comments