As a sysadmin I want to use MicroProfile Config API to configure my installation #7000

poikilotherm · 2020-06-18T19:17:37Z

Now that Payara 5 is actually happening with the release of Dataverse 5 (thank you so much folks!!!), we should re-raise the topic (formerly #5187) of starting to use some of the goodies coming along with this.

As a system admin pushing to cloud, I would greatly benefit from having the possibility to get configuration inside my instance not via mangling with JVM options and database settings. Instead, I would like to see this being more flexible and combined with a hierarchy.

This will also benefit greatly any classic installation, as configuration will be easier than dealing with domain.xml and friends.

Introduce MicroProfile Config API and its support in Payara 5 for code variables and configuration variables, combining 1 & 2.

This will open up the following work items:

Replace all the System.getProperty().
Look for the database settings and how to replace them
Make all of this backward compatible
Do solid configuration validation
Look into replacing installers/container scripts configuration of JDBC, JMS, etc with annotations, which are then configured via Config API

Things going on:

Update to Payara 5.2021.3 #7700 updating to Payara 5.2021.4 to enable MPCONFIG 2.0 support
MPCONFIG: identify non-static settings #7639
Introduce using MicroProfile Config API #7457
Move Java Mail configuration to application #7424
Move database configuration into application #7418
As a developer, I want changes I make to the front end to be visible when I deploy from Netbeans #6953 / 6953 mpconfig context param #7695

The text was updated successfully, but these errors were encountered:

… variables in annotation options. IQSS#7418 IQSS#7000

…file on classpath. IQSS#7418 IQSS#7000

…ency to enable unit and integration testing without deployments. IQSS#7000

The microbean implementation of MicroProfile Config has not been updated to MPCONFIG 2.0. Switching to the smallrye implementation for testing scope.

To be able to have more control over JVM settings names, avoid typos, maybe create lists of settings and so on, this enum will provide the place to add any old and new settings that are destined to be made at the JVM level. Further extensions of this class include adding aliases when renaming settings, adding predicates for validation and offering injecting parameters into keys (as used with the files subsystem).

To provide built-time static settings, property files under src/main/resources are now filtered by maven to replace any variables ${propertyname} known by Maven with their values before storing the file under /target.

To simplify SystemConfig.getVersion, the MicroProfile Config property dataverse.version is set to the Maven project version via filtering the variable.

Instead of trying to read a built time file from Maven, use MicroProfile Config to retrieve the version and build number. The version is by default set via microprofile-config.properties (or overridden by an env var in a container). The build number is still read from either BuildNumber.properties or, if not present, from MicroProfile Config, defaulting to empty. This also avoids copying extra files into containers to retrieve the version string.

…#7000 By refactoring SystemConfig.getSolrHostColonPort, the Solr endpoint is not just configurable via a database setting, but also by all mechanisms of MicroProfile Config. - The database setting still has priority over the other mechanisms. - It's completely backward compatible, no config change necessary. - Tests have been added to ensure the behaviour - Default ("localhost:8983") for no setting given is now also done via MPCONFIG - Default for container usage ("solr:8983") possible via MPCONFIG profile "ct"

Describe the new options to set the Solr endpoint, crosslinking the old way and adding hints about MPCONFIG profiles.

Retrieving a setting as a simple String is a very often use case within the codebase. To make these lookups easier to write and comprehend, adding convenience methods here. These are simple wrappers around the MicroProfile Config lookup.

… MPCONFIG IQSS#7000 - Adding dataverse.files.directory equivalent to JvmSettings - Remove all System.getPropert("dataverse.files.directory") or similar - Add default with /tmp/dataverse via microprofile-config.properties as formerly seen at FileUtil and Dataset only - Refactor SwordConfigurationImpl to reuse the NoSuchElementException thrown by MPCONFIG - Refactor GoogleCloudSubmitToArchiveCommand to use the JvmSettings.lookup and create file stream in try-with-resources

- When renaming older JVM settings, these should be still retrievable from an old config - The AliasConfigSource now takes aliased settings from JvmSettings for this - This is just for pure renaming but could be further extended with data manipulation.

This commit introduces support for patterned settings, containing placeholders within names. These patterns allow to read object like structures from MicroProfile Config Sources, while offering control over where and how these variable arguments are inserted. It is also possible to create aliases with these placeholders, as the AliasConfigSource has support for this included. Lookups and building old aliases are now checked operations, avoiding wrong usage.

…ion IQSS#7000 As of this commit, tests can use the extension together with patterned setting names by providing the necessary arguments within the annotation.

Adding the most basic test to ensure the most basic functionality works.

…7000 To ease lookup when target variable is a String, the convenient lookup() and lookupOptional() have been added for placeholder-type settings, too. Adds JavaDocs for all the varargs lookup functions, too, easing usage in code.

…QSS#7000 The pattern did not check for the required $ sign after the positional index. Fixing that here.

Emphasize that the change is for devs for now. The idea is to migrate settings to the new framework over time.

IQSS#7000

…SS#9229 IQSS#7000

…#7000

As we are now using JvmSettings everywhere as an abstraction above MPCONFIG, no need to import on our own.

…SS#7000 When an invalid port was set, which cannot be transformed to an integer, the JSF page would kind of freeze: it leaves the user on the grayed out "action" screen and if you click on it, shows upload page with no files, so a silent error. By defaulting to 6311 also for this case and logging an error, this should improve the UX, but leave a hint for an admin in the logs.

…d compat IQSS#7000

To enable the AliasConfigSource to provide settings that also have a representation in a microprofile-config.properties file, it needs to have at least the same ordinal as that source. Thus the ordinal shall not be minimum integer, but 100. As the name is "Alias", it will likely to be used first when other sources are present with this ordinal (like that properties source).

…ies IQSS#7000

This adds the missing MPCONFIG option to configure the base URL for the permalink provider, which has been added after the initial work for providers happened.

poikilotherm added Feature: Admin Guide Component: Code Infrastructure formerly "Feature: Code Infrastructure" Type: Feature a feature request User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh labels Jun 18, 2020

poikilotherm mentioned this issue Sep 10, 2020

scripts for rebuilding a dev environment (with sample data) #7256

Closed

poikilotherm mentioned this issue Oct 23, 2020

Epic: small footprint container usable for development, testing and production purposes #5292

Closed

poikilotherm mentioned this issue Nov 17, 2020

Move database configuration into application #7418

Closed

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Nov 18, 2020

Make the DataSourceDefinition configureable using MicroProfile Config…

b92e465

… variables in annotation options. IQSS#7418 IQSS#7000

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Nov 18, 2020

Add defaults for the database connection using a MicroProfile Config …

3c92120

…file on classpath. IQSS#7418 IQSS#7000

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 1, 2020

Introduce Eclipse MicroProfile Config API to the project. IQSS#7000

4f5a6b9

poikilotherm mentioned this issue Dec 4, 2020

Introduce using MicroProfile Config API #7457

Closed

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 4, 2020

Introduce Eclipse MicroProfile Config API to the project. IQSS#7000

74f35d8

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 4, 2020

Add MicroBean MicroProfile Config API implementation as a test depend…

13447f9

…ency to enable unit and integration testing without deployments. IQSS#7000

poikilotherm mentioned this issue Dec 7, 2020

db_password_alias changed to dataverse.db.password #7418 #7422 #7459

Merged

poikilotherm mentioned this issue Jun 24, 2021

As an admin and developer, I'd benefit from a simplified configuration mechanism #5293

Closed

poikilotherm self-assigned this Jun 24, 2021

poikilotherm mentioned this issue Jun 20, 2022

7000 mpconfig for JVM settings #8810

Closed

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Jun 21, 2022

docs(settings): add Solr MPCONFIG options to guides IQSS#7000

7c98cf1

Describe the new options to set the Solr endpoint, crosslinking the old way and adding hints about MPCONFIG profiles.

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Jun 22, 2022

refactor(settings): use new JvmSettings.lookup() IQSS#7000

983e49f

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Jun 23, 2022

docs(settings): mark :SolrHostColonPort with @deprecated IQSS#7000

a2edcf0

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

docs(settings): add basic release note for IQSS#7000

3f3c318

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

test(settings): add simple setting test for JvmSettings IQSS#7000

063883b

Adding the most basic test to ensure the most basic functionality works.

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

docs(dev): update for JvmSettings and deleted DB config source IQSS#7000

1ec792a

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

fix(settings): correct JvmSettings pattern for old names placeholders I…

f423f51

…QSS#7000 The pattern did not check for the required $ sign after the positional index. Fixing that here.

poikilotherm pushed a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

tweak docs and release note IQSS#7000

c86517d

Emphasize that the change is for devs for now. The idea is to migrate settings to the new framework over time.

poikilotherm pushed a commit to poikilotherm/dataverse that referenced this issue Sep 20, 2022

"IllegalStateException: Recursive update" went away with Payara upgrade

ee426d7

IQSS#7000

mreekie added the bk2211 label Nov 1, 2022

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 6, 2022

refactor(settings): remove unused Config var in SystemConfig IQSS#7000

4b60983

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 14, 2022

feat(settings): add feature gates functionality for the first time IQ…

f5862e3

…SS#9229 IQSS#7000

pdurbin added a commit to poikilotherm/dataverse that referenced this issue Dec 16, 2022

cross link some docs IQSS#7000

4ed1013

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 19, 2022

fix(rserve): use correct setting key in mp-c.props for user IQSS#7000

ef7030f

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 19, 2022

fix(rserve): align docs and code for Rserve temporary file store IQSS…

a01fd58

…#7000

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Dec 20, 2022

style(solr): remove unused MPCONFIG import IQSS#7000

fcfd9bf

As we are now using JvmSettings everywhere as an abstraction above MPCONFIG, no need to import on our own.

mreekie removed the bk2211 label Jan 11, 2023

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Mar 1, 2023

feat(settings): add alias names to Handle.net JvmSettings for backwar…

661cb2e

…d compat IQSS#7000

poikilotherm added a commit to poikilotherm/dataverse that referenced this issue Mar 1, 2023

test(settings): add tests for alias JvmSettings with DataCite propert…

803e11f

…ies IQSS#7000

pdurbin added a commit to poikilotherm/dataverse that referenced this issue Mar 31, 2023

list new settings in release note IQSS#7000

c92a635

pdurbin added a commit to poikilotherm/dataverse that referenced this issue Mar 31, 2023

various tweaks to PID provider docs IQSS#7000

b3b0b5d

This was referenced Oct 11, 2023

Migrate storage configuration to MPCONFIG (milestone D) #9998

Open

Migrate leftover hardcoded JVM options to be MPCONFIG enabled #9999

Open

MPCONFIG: move JvmSettings to SPI (was: allow to retrieve all values used for a placeholder scoped setting) #10000

Open

poikilotherm mentioned this issue Jul 14, 2024

Feature Request: Enabling configuration of Dataverse using simple file(s) #10684

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

As a sysadmin I want to use MicroProfile Config API to configure my installation #7000

As a sysadmin I want to use MicroProfile Config API to configure my installation #7000

poikilotherm commented Jun 18, 2020 •

edited

Loading

As a sysadmin I want to use MicroProfile Config API to configure my installation #7000

As a sysadmin I want to use MicroProfile Config API to configure my installation #7000

Comments

poikilotherm commented Jun 18, 2020 • edited Loading

Things going on:

poikilotherm commented Jun 18, 2020 •

edited

Loading