Search configuration cache #1625
Comments
|
Try to avoid concurrency issues, particularly with locking. Probably worth considering a ThreadLocal cache implementation for this stuff as long as the cached content isn't big (which seems to be the case here). |
This is a big, end-user-visible change to the way we load search parameters and apply filtering from fhir-server-config. Previously, we loaded the built-in search parameters from the registry, but then applied tenant-specific extension-search-parameters on top of that. Now, we have a new ExtensionSearchParametersResourceProvider that reads the same tenant-specific extension-search-parameters (for backwards compatibility) and provides those resources to the registry in a tenant-aware manner. This is #1596. Additionally, we used to apply search filtering on each and every request. Now, the ParametersUtil will loop through all configured tenants and load their search parameters from the registry during initialization. This tenant-aware search parameter map will then be consulted anytime we need a search parameter or set of search parameters in the context of a particular request (for a particular tenant). Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
This is a big, end-user-visible change to the way we load search parameters and apply filtering from fhir-server-config. Previously, we loaded the built-in search parameters from the registry, but then applied tenant-specific extension-search-parameters on top of that. Now, we have a new ExtensionSearchParametersResourceProvider that reads the same tenant-specific extension-search-parameters (for backwards compatibility) and provides those resources to the registry in a tenant-aware manner. This is #1596. Additionally, we used to apply search filtering on each and every request. Now, the ParametersUtil will loop through all configured tenants and load their search parameters from the registry during initialization. This tenant-aware search parameter map will then be consulted anytime we need a search parameter or set of search parameters in the context of a particular request (for a particular tenant). Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
This is a big, end-user-visible change to the way we load search parameters and apply filtering from fhir-server-config. Previously, we loaded the built-in search parameters from the registry, but then applied tenant-specific extension-search-parameters on top of that. Now, we have a new ExtensionSearchParametersResourceProvider that reads the same tenant-specific extension-search-parameters (for backwards compatibility) and provides those resources to the registry in a tenant-aware manner. This is #1596. Additionally, we used to apply search filtering on each and every request. Now, the ParametersUtil will loop through all configured tenants and load their search parameters from the registry during initialization. This tenant-aware search parameter map will then be consulted anytime we need a search parameter or set of search parameters in the context of a particular request (for a particular tenant). Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
This is a big, end-user-visible change to the way we load search parameters and apply filtering from fhir-server-config. Previously, we loaded the built-in search parameters from the registry, but then applied tenant-specific extension-search-parameters on top of that. Now, we have a new ExtensionSearchParametersResourceProvider that reads the same tenant-specific extension-search-parameters (for backwards compatibility) and provides those resources to the registry in a tenant-aware manner. This is #1596. Additionally, we used to apply search filtering on each and every request. Now, the ParametersUtil will loop through all configured tenants and load their search parameters from the registry during initialization. This tenant-aware search parameter map will then be consulted anytime we need a search parameter or set of search parameters in the context of a particular request (for a particular tenant). Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
* issues #1625 and #1596 - apply search parameter filters on init This is a big, end-user-visible change to the way we load search parameters and apply filtering from fhir-server-config. Previously, we loaded the built-in search parameters from the registry, but then applied tenant-specific extension-search-parameters on top of that. Now, we have a new ExtensionSearchParametersResourceProvider that reads the same tenant-specific extension-search-parameters (for backwards compatibility) and provides those resources to the registry in a tenant-aware manner. This is #1596. Additionally, we used to apply search filtering on each and every request. Now, the ParametersUtil will loop through all configured tenants and load their search parameters from the registry during initialization. This tenant-aware search parameter map will then be consulted anytime we need a search parameter or set of search parameters in the context of a particular request (for a particular tenant). Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * JDBC test cleanup LocationParmBehaviorUtilTest wasn't restoring the FHIRRequestContext after running. Additionally, I removed the 'static' modifier from the before and after methods...its not needed for TestNG and, in fact, TestNG warns against using static methods for this. Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * Handle missing config more gracefully The CQL SearchParameterResolverTest was calling SearchUtil.getSearchParameters without any fhir-server-config files in the home directory. This was causing a NullPointerException while initializing ParametersUtil. The updated logic will now load all parameters in such cases...just as if fhir-server-config was there but without a `resources` PropertyGroup. Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * updated documentation to reflect new behavior Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * General cleanup Extracted private helpers from ParametersUtil.computeTenantSPs and added some comments. Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * Update SearchUtil.getSearchParameters method sig This one now returns a map from code to SearchParameter instead of just a the list of SearchParameter. This way, we can support cases where the explicitly-configured code differs from the code in the SearchParameter. Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * Update ParametersMap to store disambiguated results Previously, ParametersMap had to store all the built-in parameters and make those available to SearchUtil for filtering/disambiguation. Now that we apply filtering before populating ParametersMap, we can select a "winner" in cases where there are multiple parameter with the same code (or multiple parameters with the same url / canonical). For now, I chose a "last insert wins" approach...which means that the order we load the map in will matter. Note: this change uncovered an issue in our fhir-persistence search tests where we defined a parameter with code "code" that conflicts with a base spec parameter. The fix was to disambiguate via fhir-server-config. Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com> * minor updates per review feedback mostly copyright dates... Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
|
The "cache" was implemented as a revision to the existing ParameterUtils class. Now, upon initialization, that will loop over all configured tenants (i.e. all directories under the fhir-server's config dir) and apply the search filtering rules therein to the search parameters loaded from the registry. |
|
QA Complete Looks good Test 1. Server Startup with a single Parameter[11/12/21, 15:52:14:076 UTC] 00000022 FHIRConfigura 1 Returning list of tenant ids: [default] Test 2. Added SearchParameter post startupWaited 1 minute, No evidence of reloading Test 3. Restart and Rerun{ Shows no error / invalid which is what is expected Test 4. Restart and Run with tenant1 and tenant2[11/12/21, 16:12:48:696 UTC] 00000027 FHIRConfigura 1 Returning list of tenant ids: [default, tenant1, tenant2] The three tenants are loaded and their parameters are available. Test 5. Startup and Check FS Events during queryPS=$(ps -ef | grep -i fhir-server | grep -v grep | awk '{print $2}') I did not see any FS events that showed a reloading for the search-parameters json |
Is your feature request related to a problem? Please describe.
SearchUtil.getFilteredBuiltinSearchParameters()reads in tenant-specific configuration and uses that config to filter the built-in search parameters down to a list of enabled search parameters on a per-tenant basis.The fhir-config module already caches fhir-server-config.json and so reading the config should be relatively fast (no file read if the file hasn't changed), but we're still doing some extra work with each request.
Namely, if the config hasn't changed then there should be no need to recompute the set of search parameters.
Describe the solution you'd like
During server startup, we could read the fhir-server-config.json for all the tenants and pre-construct a ParametersMap for each one.
Then, we should only ever rebuild the ParametersMap if/when the fhir-server-config.json is updated for this tenant (and only if the dynamic behavior is required).
Describe alternatives you've considered
Instead of server startup, we could build the tenant-specific ParametersMap on the first request from the tenant.
Additional context
This change is highly related to #1596 because, if we get the "built-in" search parameters from the registry, then the caching here would be complicated by the fact that the registry could return different SearchParameters over time (at least when
serverRegistryResourceProviderEnabledis set to true).The text was updated successfully, but these errors were encountered: