Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for arrays of different data types in the Trigger Form UI #32734

Merged

Conversation

MatthieuBlais
Copy link
Contributor

Description

The new Trigger Form UI released in 2.6 is handy but doesn't support array params of data types other than string. For more flexibility, this PR proposes to generate a JSON field similar to the object param if the users specify the attribute items in schema.

How to reproduce the issue

Use an array param and require the items to be integers.

with DAG(
    dag_id=Path(__file__).stem,
    description=__doc__[0 : __doc__.find(".")],
    doc_md=__doc__,
    schedule=None,
    start_date=datetime.datetime(2022, 3, 4),
    catchup=False,
    tags=["example_ui"],
    params={
        "array_of_numbers": Param(
            [1, 2, 3],
            "Only integers are accepted in this array",
            type="array",
            title="Array of numbers",
            items={"type": "number"},
        ),
    },
) as dag:

    @task(task_id="show_params")
    def show_params(**kwargs) -> None:
        ti: TaskInstance = kwargs["ti"]
        dag_run: DagRun = ti.dag_run
        if not dag_run.conf:
            print("Uups, no parameters supplied as DagRun.conf, was the trigger w/o form?")
            raise AirflowSkipException("No DagRun.conf parameters supplied.")
        print(f"This DAG was triggered with the following parameters:\n{json.dumps(dag_run.conf, indent=4)}")

    show_params()

The Generated Conf JSON shows an array of strings

image

And we cannot trigger the DAG:

image

Workaround

We can manually update the Generated Configuration JSON box, but it is not user friendly (especially if we expect an array of objects!), and will become counter intuitive as users get used to the benefits of the new trigger form.

What do you think @jens-scheffler-bosch ?

@boring-cyborg boring-cyborg bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues kind:documentation labels Jul 21, 2023
@jscheffl jscheffl self-assigned this Jul 21, 2023
@jscheffl jscheffl added type:improvement Changelog: Improvements and removed kind:documentation labels Jul 21, 2023
Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small beautification proposals in documentation, otherwise - I like it! Thanks for the contribution!
(Take my approval emotional, I have no committer status)

docs/apache-airflow/core-concepts/params.rst Outdated Show resolved Hide resolved
airflow/example_dags/example_params_ui_tutorial.py Outdated Show resolved Hide resolved
@jscheffl jscheffl added this to the Airflow 2.7.0 milestone Jul 22, 2023
@MatthieuBlais MatthieuBlais force-pushed the feature/complex_arrays_for_trigger_forms branch from 9ed9cf4 to f029ad1 Compare July 23, 2023 15:00
@MatthieuBlais
Copy link
Contributor Author

Great, thanks for the feedback! I've updated the PR with your commits

@jscheffl
Copy link
Contributor

Great, thanks for the feedback! I've updated the PR with your commits

Uuups, sorry, my comments were not syntax checked :-( The URL is not correctly formed to be accepted by Sphinx.

Matthieu Blais and others added 6 commits July 24, 2023 23:00
@MatthieuBlais MatthieuBlais force-pushed the feature/complex_arrays_for_trigger_forms branch from 496b0a1 to 808d2bb Compare July 24, 2023 15:00
@eladkal eladkal merged commit da69315 into apache:main Aug 3, 2023
@ephraimbuddy ephraimbuddy removed this from the Airflow 2.7.1 milestone Aug 3, 2023
@ephraimbuddy ephraimbuddy added this to the Airflow 2.7.0 milestone Aug 3, 2023
ephraimbuddy pushed a commit that referenced this pull request Aug 3, 2023
…#32734)

* Add JSON field for complex arrays with types other than string

---------

Co-authored-by: Matthieu Blais <matthieu.blais@tech.jago.com>
Co-authored-by: Jens Scheffler <95105677+jens-scheffler-bosch@users.noreply.github.com>
Co-authored-by: eladkal <45845474+eladkal@users.noreply.github.com>
(cherry picked from commit da69315)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:improvement Changelog: Improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants