Documenting the cluster policy feature

[mistercrunch]

This feature has been around for months but was never documented.

[r39132]

Hmn.. When does this policy get applied? Is a restart required across the cluster or is it automatically applied with the config file is reloaded? How does it get applied in a distributed deployment?

[aoen]

Policy gets applied on the scheduler when the dagbag is loaded, it is automatically picked up when the config file is modified. If by distributed deployment you mean not using local executors then it should still be run when the dagbag is loaded.

Not sure if those were questions or things you wanted to add to the PR comments, but they should be added either way.

[mistercrunch]

Yes, since this policy policy is meant to mutate tasks, it actually applies when the airflow run subprocess gets executed by the executor, meaning that all tasks that start after the settings file is changed should be impacted.

For the changes to be visible on the web server (most likely on the Task Details page), you'll need to refresh the DAG (rightmost button on the icon list in the DAGs view).

[mistercrunch]

If you were to mutate something that affect the scheduler (the queue for instance), by default it fully refreshes the DagBag every 10 runs, though refreshing from the UI should do it here and affect the very next scheduling cycle.