How to handle configuration options and namespace prefixes

[jhiemstrawisc]

Right now, the s3 plugin parses a namespace based on its configuration.

The configuration:

s3.service_name = my-service
s3.region = my-region
s3.service_url = https://my-url.com

results in the namespace /my-service/my-region

and getting object foo from bucket bar happens by issuing a GET request to the xrootd host at:
https://<xrootd host>/my-service/my-region/bar/foo

We are now looking to decouple the namespace from the existing s3 configuration options, as they're overloaded and cannot in general be set to arbitrary values (which is something we might need for creating arbitrary namespaces).

Discussions with @rw2 and @nkissebe landed at a possible configuration file of:

s3.service_name = my-service
s3.region = my-region
s3.service_url = https://my-url.com
s3.path_name my-magic-path

# Config that groups keys and buckets
s3.authz_map = /authz/map

where the authz map (or whatever we end up calling it) is however we decide to handle pairings of access key, secret key, and bucket.

All of the buckets exported by the xrootd instance should belong to the same service url, service name, and region. Then the Pelican federation namespace will be /my-magic-path/bar/foo, which gets converted into a request to https://<origin url>/my-magic-path/bar/foo where the origin url is the hostname of this instance of xrootd.

What I'm personally uncertain of is whether we need s3.path_name or if we should just handle this all via all.export, which is how we set the namespace prefix for POSIX origins.

[jhiemstrawisc]

@bbockelm, @matyasselmeci We wanted to open this up as a discussion to get your feedback in recognizing that we're probably not far away from being locked into our choices.

[rw2]

@jhiemstrawisc can there be multipel all.export(s)? If so, then that's an option. If not, then I think we need a path_name to distinguish the multiple configurations that are possible. The config file would have multiple path_name lines, one for each s3 configuration.

[jhiemstrawisc]

I'm pretty sure there can be, but I think if we go that route we'll always be locked into having the bucket name as a part of the namespace prefix. Can we ever envision a scenario in which we don't want that setup?

[rw2]

I think I'm back to advocating the proposal I made Friday.

s3.path_name my-magic-path
s3.bucket_name hubzero-private-rich
s3.service_name s3.amazonaws.com
s3.region us-east-1
s3.access_key_file /xrootd-dev/access-key
s3.secret_key_file /xrootd-dev/secret-key
s3.service_url https://s3.us-east-1.amazonaws.com

I didn't catch your external map file in the first post in this thread. What I like about my proposal is that everything is in one file and the bucket name can be hidden completely from the user if desired. And, of course, if an admin wants to expose it by putting it in my-magic-path they still can do that if they want.

[jhiemstrawisc]

I know this is probably blocking for you, but let's give this 1-2 more days to give @bbockelm and @matyasselmeci a chance to respond. After that, go ahead and make the judgement call, since ultimately you're the one implementing this (and have probably looked deeper in the source for what an implementation will look like than anyone else). I do worry a bit about multi-bucket setups. Would these blocks of s3 directives be delineated by an empty line? Or how does the plugin know the groupings? With most XRootD configurations I've seen, it feels like there's an assumption that "order doesn't matter," which wouldn't be the case here.

I do think that in general we should disallow a single origin to point at multiple regions or multiple service urls, if your intention was to define those per-grouping.

[rw2]

I don't think that multiline, order dependent, configs are the greatest thing in the world either. But I think if properly documented they can work. The implementation I have today treats the service_url as an end of block, it will be documented in the example as well as the doc and I am planning to have the error kicked out by the system be extra useful also.

[matyasselmeci]

If you must make it multiline, order-dependent then you should add explicit block markers.

s3.begin
  s3.path_name my-magic-path
  s3.bucket_name hubzero-private-rich
  s3.service_name s3.amazonaws.com
  s3.region us-east-1
  s3.access_key_file /xrootd-dev/access-key
  s3.secret_key_file /xrootd-dev/secret-key
  s3.service_url https://s3.us-east-1.amazonaws.com/
s3.end

multiline order-dependent knobs, to my knowledge, are not used anywhere else in the xrootd config, so if you must add them, make them stand out.

I just compared two xrootd installations by running cconfig on the configurations, running that through sort and opening the two in vimdiff. If I can no longer do that, I'd like to know just by reading the config file, without having to find and read the documentation.

[rw2]

Yeah, this seems like a fair compromise. That's what I'll do.