Support plugin configurations within config.properties

[tdcmeehan]

Hello folks,

Wanted to get any inputs on supporting configuring plugins directly within config.properties.

Firstly, why do this at all? When configuring multiple plugins, it can be a bit of a pain to see the at a glance configuration of Presto, because they're scattered across several properties. I think it would be easier to provide a default instance of a configuration if we allow inline plugin configuration in the main config file.

The idea would be to add a special prefix for plugins, such as plugin., then followed by a key indicating the plugin itself (such as hive), then a key indicating the instance of the plugin if applicable (such as hive_interactive1). There's a bit of disorganization in how we configure plugins, and I think this would clean it up a bit. Keys which are not prefixed by plugin. will continue to be strictly validated, and to prevent an accident where a core property is added which has this prefix, we can add a checkstyle rule which would fail the build if it's added at any point in the future.

We could still support the old way of configuring plugins. I have a couple ideas

1. Preferred. We could fallback to the old way if no property of the matching plugin and plugin instance are found. Perhaps we can throw if both are present simultaneously.
2. We could treat the plugin properties in config.properties as a default, and override when a dedicated etc config present.

Appreciate any thoughts.

[ZacBlanco]

I think this would be nice feature to have. Separate configuration files is also nice for users who might have lots of configuration items or complex setups where navigating a single file can lead to misunderstandings or errors. Here are a few of my thoughts regarding the configuration of plugins:

1. I think plugins should always be configured explicitly in config.properties

• Currently plugins are determined by looking at the list of files under the etc/ folder, requiring filesystem inspection in order to determine the plugins which might be deployed This can lead to ambiguity or confusion as to which plugins are deployed. Even if we don't take the approach of plugins configured directly in config.properties I think that at minimum we should force users to declare the plugins they wish to use inside of config.properties. This might take a form similar to what was suggested above where users would add a line in the config.properties which looks like plugin.<user-provided-name>=<plugin name (e.g. iceberg, hive, ...)>,

2. I think plugin config location outside of config.properties should be defined explicitly.

• Following with the example previously, users would add a line in their config which has an enum with values INLINE or FILESYSTEM that tells Presto where to look for the plugin's configuration. e.g. plugin.<user-provided-name>.location=FILESYSTEM. INLINE could be the default and allow users to configure plugins within config.properties directly. FILESYSTEM would then either search in etc/ automatically or allow the users to specify a particular path to their configuration with something like plugin.<user-provided-name>.config-location=<path>.

3. I have a concern that for complex configurations, we might want to provide a new type of configuration file, such as TOML which has sections built-in to its format.

• I have seen some user configurations for hive which have lots of configuration values. Having to prefix everything in config.properties with plugin.<name> might be cumbersome. It could be useful to have config sections like in TOML which would prevent needing to prefix plugin configurations. Example TOML configuration below.
• TOML might also be an interesting choice because it allows specifying structs in addition to simple values which could help simplify configuration/prevent errors when configuring subsystems (e.g. related caching parameters/timeouts), security, etc

plugin.hive_interactive1 = "hive"
plugin.hive_batch = "hive"

[plugin.hive_batch]
location = "FILESYSTEM" # plugin.<name>.config-file must be specified when this is present
config-file = "/opt/my-custom-location/plugin.config" 

[plugin.hive_interactive1]
hive.metastore = "thrift"
hive.metastore.uri = "thrift://example.net:9083"
hive.metastore-cache = { ttl = "1m", refresh = { interval = "20m", max-threads = 4 } }

[tdcmeehan]

Using TOML is an interesting suggestion. I like how it allows inline tables as in your example, and it seems like an easy migration from more basic properties. I also find the syntax to be intuitive.

I wrestled a bit with the idea of making plugins an array of tables, but I think the example above is better since plugin configurations have set semantics (no duplicates), and using an array to specify them would mean we'd have to validate those set semantics.

[hantangwangd]

I think it would be nice to support plugin configurations directly in config.properties, and keep the old way as a fallback configuration. And agree with @ZacBlanco 's suggestion about the config sections, which do not need to use prefixes to distinguish the same configuration items used by different plugins. In this way, we can easily identify the common configuration items which may be used in multiple plugins, and move them to appropriate common modules as appropriate. For example, moving some configuration items from presto-hive to presto-hive-common or presto-hive-metastore, since they may be configured and used in some other lakehouse connectors like Iceberg, Hudi or Delta.