OpenAPI: Add endpoint to retrieve valid credentials for a given table

[nastra]

This depends on the standardized credentials from #10722

[snazy]

I think that there should be a "top level" expiresAt field, so implementations do not have to peal out this common piece of information from the config property bag. If expiresAt is missing, that would mean the credentials do not expire.

Maybe also an optional refreshAt or refreshNotBefore timestamp "top level" field as well, so the server can tell clients when a refresh should happen and not have clients refresh too early (or too late).

[snazy]

Suggested change

	config:
	expiresAt:
	type: integer
	format: int64
	description: Timestamp in milliseconds since epoch (1970-01-01-00:00:00Z) when the credentials expire. If credentials do not expire, this field is absent.
	refreshNotBefore:
	type: integer
	format: int64
	description: If this field is present, clients must not refresh credentials before this timestamp in milliseconds since epoch (1970-01-01-00:00:00Z). This field must not be present for credentials that do not expire.
	config:

[nastra]

It could make sense to have a expires-at-ms field here, but I believe it's better if this isn't part of the OpenAPI spec and be rather specific to the underlying storage provider. That being said, I think it should be documented along with other properties related to that storage provider. The underlying refresh handler for the given storage provider can still mandate that the server sends the expiration field for a vended credential

[snazy]

rather specific to the underlying storage provider

The whole effort is about refreshing expiring credentials, so such an attribute is a first-class citizen, not something hidden in a rather generic property bag.

[bryanck]

It makes sense to me to have an "expires at" field of some sort. It might help make the refresh implementation logic more reusable across different providers?

[nastra]

we discussed this in yesterday's catalog sync and concluded that we don't want to introduce a separate expires-at-ms field to the spec.

[snazy]

Should have a definition here how prefixes must be described.
I can think of:

• schema: (including the colon :) to indicate that the credentials are valid for all FileIOs that use this schema
• schema://bucket/ (including the trailing slash /) for credentials for a specific bucket
• schema://bucket/some/path/ (including the trailing slash /) for credentials for a specific path in a specific bucket

I think it's important to define whether a trailing slash / must, should, or must not be included in the prefix to have a common definition across all implementations and behavior on the client side.

Mandating a trailing slash makes paths non-ambiguous (e.g. s3://foo/bar/ vs s3://foo/bar in case there are paths like s3://foo/bar/meep and s3://foo/barbaz/meep). WDYT?

[nastra]

I don't think we should be calling out specific things related to storage providers here on how pathing is used. This just makes it more difficult to evolve the OpenAPI spec

[snazy]

But where else should it be specified/documented?
Javadoc is rather a bad place, because it's Java-specific.

[nastra]

we can talk/think about what the best place would be for documenting this, but I don't think the OpenAPI spec is the right place to handle storage-specific paths

[snazy]

I suspect, it's worth to mention that both the URI's authority and path elements must be URL escaped.

[nastra]

it is up to the server to provide a valid prefix URL. As I mentioned further above, I don't think mentioning storage-specific paths in the OpenAPI spec is something that we'd want to do

[findepi]

cc @c-thiel