Export import for connections for airflowctl is broken in main

[potiuk]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

After #53973 where change in our security model has been implemented, to not allow anyone (including connection editing users) to be able to see sensitive data from the connection - effectively making connection sensitive data "write-only". This has an impact on export/import capability for airflowctl because airflowctl exposed export/import capability and retrieved connection data via API - and the current model (draft described in #54088) assumes that there is no access to sensitive connection credential via "public API". Only Tasks have access to those via "task-sdk" API.

However this means that exported files will contain "***" (unicode-d) mask instead of real sensitive data - which make them unusable for imports.

There are several ways it can be solved:

1. drop export/import functionality from airflowctl (technically speaking we could only drop export and assume that someone could export using airflow CLI to export data and airflowctl to import it, but it could be confusing).

2. encrypt the exported data (either with FERNET key or with a passphrase entered by the user) - the passphrase by the user is however violating the assumption that no "API" user shoud be able to see the password - we should rather use some security passphrase that is not available to the API user

3. relax expectation of the airflowctl user to be able to use "task-sdk" API - but this is also violating the assumptions that no API user should see the credentials.

Possibly there are other optiosns.

@ashb @pierrejeambrun (and of course @bugraoz93 should likely be aware of that) I wonder what your opinions on that are. We had no chance to discuss it before, because decision on changing the model has been taken very hastily without discussing the consequences, but maybe that is a good opportunity to discuss it here.

What you think should happen instead?

No response

How to reproduce

1. Install airflow
2. add connection
3. run airflowctl connection export
4. observe export file containing masked sensitive data that is not "importable" correctly

Operating System

Any

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Prasanth345]

Hi @potiuk I’d like to contribute towards resolving this issue.

From what I understand, the recent changes in the security model prevent exporting plaintext credentials, which breaks airflowctl connection export/import. I’d be happy to explore and propose a solution — whether that’s encrypted export, removing the feature, or adjusting the API access.

Would you prefer one direction over the others, or should I open a draft PR with options?

[potiuk]

I do not know - I was waiting mostly for comments of @ashb @pierrejeambrun @bugraoz93 as I think it's the consequence of the model we chose, but I am not sure if the good solution is to use different API user, specific permission of a user via public API, or encryption. I'd love to hear their opinion.

[ashb]

Import is unaffected right? It's just a "bulk create" operation, so I don't see what needs to change about that.

I'd say this should probably be an extra permission over-and-above "read connections".

[potiuk]

Import is unaffected right? It's just a "bulk create" operation, so I don't see what needs to change about that.

Yes. Import can be done without problems because we have "write-only" permission.

Most problems is with export - because our model assumes currently that we do not expose such credentials over API to the user (we only expose them via task-sdk to workloads and triggerer)

So my question is do we want to create another role in our model - someone who is able to read the passwords via public API. So you say we need to add such role? Do you think it's secure enough ? Previously the user who could edit connecton could do that - but we considered it not "Secure enough". And I think if we decide on adding it back, we need to be very specific about that role.

I'd say it's a bit not consistent as that user who logs in via the UI will also be able to see those passwords in the UI - so we are just shifting the problem we tried to solve to another role- and we need to add back to our security model that there are users who can read the credentials.

Or maybe we want to introduce a different class of the API and authentication - so far as I understand airflowctl uses the same API as the UI and we have no way to distinguish whether we are calling it from UI or airflowctl?

[ashb]

we have no way to distinguish whether we are calling it from UI or airflowctl?

We don't currently, correct, but we could probably very easily detect this via User-Agent header. Or the other option is to create a different UI-only endpoint (we already have other such endpoints)

I still think it should be a different permission though -- exporting all connections is a highly sensitive and risky approach -- so I'd feel much happier with export being a separate endpoint, even if it's not a separate permission. My main driver for this is auditability -- if it's a separate endpoint that exposes the password, you can then tell when people get those from acces logs alone.

So I'd probably say both:

1. UI only endpoint that always redacts (and possibly remove the hide_sensitive_var_conn_fields setting and make it mandatory?)
2. Add a new permission, ("Connections", "can_read_unredacted") or similar, that is required to either export, or to read a connection without redaction.
3. We could extend the UI views to add a "show senstive" fields button which would use the public API to populate things.

If you call the export endpoint and you don't have the export permission it could either return connections with redaction, or more likely return a 403. And if you call the normal connection GET endpoint without having that new perm it would return it redacted exactly as the UI?

[potiuk]

We don't currently, correct, but we could probably very easily detect this via User-Agent header. Or the other option is to create a different UI-only endpoint (we already have other such endpoints)

This can be spoofed by an attacker potentially (via injecting javascript etc.). I think the only way to prevent it, is to have completely separate authentication mechanism dedicated to the endpoint that returns the unredacted data.

How about just dropping export via "airflowctl" ? If we consider it really dangerous, we might only leave it for direct access via "airflow" command. And that even makes perfect sense because exporting/importing connections is almost like you export/import database and for that you need to have access to the database.

[pierrejeambrun]

I like the double endpoint approach. I would love to have a notion of technical user which isn't a real person, for machine to machine auth, maybe a new class of tokens or something like that, and some sensitive endpoint such as the retrieve connection without redacting could be exclusive to technical user. (basically there is no real 'user' in the token, that returns None, and the token might be a little bit different)

Or simply a new Role, that works too. And we can develop a custom dependency / decorator @require_technical_user that would allow such users to access an endpoint, and modify the basic dependency for permission so it would refuse access to technical users. This way technical user can only access endpoints with @require_technical_user which is export connection only at the moment.

[bugraoz93]

Yes, the UI and airflowctl are using the same endpoints on those entities. There are a few for UI for specific tasks. Since they are using the same endpoints, I would expect them to behave similarly.

If we reduct in UI, it makes sense to keep the same for it since the audience for GUI and CTL in general is the same.
We can add one more layer, like a new endpoint and role. New role could be better because then we can control for multiple endpoints while we need to consider GET and LIST endpoints of the reducted entities. I am not sure if these things are more secure since the Read Connection role can also be given carefully.

Yes, the connections are not a Secret Manager in general, but sometimes, while updating or validating, people tend to read from there.

I also agree that export may be more dangerous, but if you have read access without reduction, you can easily automate the export. I think we should evaluate the reduction in general, as it also encompasses both obtaining a single connection and acquiring all connections. Those are also endpoints and behave similarly. This is basically an export if you run airflowctl connections list -o yaml > a.yaml because, as in the airflow CLI, we are returning all in CTL, and the list can return yaml, json, etc...
This case also applies to Variables if it has some kind of password and needs a reduction, right?

I am also okay to drop this from airflowctl and keep it in airflow as an administrative operation to read database directly since the values are sensitive, but I would like to have similar functionality with the UI if we can, because if you like and get used to commands where you can do the same with things with the UI, for example, I generally use CLI in that case where you feel more control over what you call (of course GUI can be easier), my point is it would be great if we would be consistent in between UI and CTL.