Allow Variable and XCom set and delete from trigger code

[karenbraganz]

This PR allows the usage of Variable and XCom set & delete methods from code that is run in the triggerer.

At the moment, only Variable and XCom get methods can be used.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[amoghrajesh]

@karenbraganz could you please update the PR title to reflect what is being done?

Tests are also needed in airflow-core/tests/unit/jobs/test_triggerer_job.py

[amoghrajesh]

@karenbraganz how's it going with this one?

[karenbraganz]

@amoghrajesh I have completed writing tests that are passing for Variable set and delete. However, I am having some trouble with XCom since there is no longer a XCom.get_one() method, which I was planning to use in the test to check whether the XCom was set. I am looking into alternative ways to test whether the XCom was set.

[karenbraganz]

I was able to make it work with get_many() instead.

[uranusjr]

I’m starting to wonder if it’d be a good idea to split the if-elses and maybe share duplicate parts between the triggerer and worker supervisor.

[karenbraganz]

@uranusjr could you please elaborate? I don't completely understand what you mean by splitting if-elses and which duplicate parts you are referring to.

[uranusjr]

It’s not spoecific to your PR. The trigger runner and task runner now has a lot of duplicate code handling api calls and they are increasingly difficult to manage. This probably shouldn’t be fixed in this PR but should be kept in mind.

[amoghrajesh]

@karenbraganz I will take a look at this today, the test timeouts could be intermittent. Let me close and open this PR to trigger the tests again.

[amoghrajesh]

One comment, rest looks good. Preemptively approving.

[amoghrajesh]

All good.

[github-actions]

Backport failed to create: v3-0-test. View the failure log Run details

Status	Branch	Result
❌	v3-0-test

You can attempt to backport this manually by running:

cherry_picker 13fa232 v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[amoghrajesh]

Since 3.0.4 is out in RC phase, we should target this for 3.0.5.
@karenbraganz could you help with a backport there (v3-0-test branch)?