Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL on FHIR view runner #1775

Open
wants to merge 333 commits into
base: main
Choose a base branch
from
Open

SQL on FHIR view runner #1775

wants to merge 333 commits into from

Conversation

piotrszul
Copy link
Collaborator

@piotrszul piotrszul commented Jan 16, 2024

This pull request adds the ability to execute SQL on FHIR view definitions against data sources within the library API.

It also includes a comprehensive refactoring that integrates this new functionality with the rest of the code base and makes some changes that will improve performance across all of the different FHIRPath query operations.

Resolves #1438 and #1759.

Implementation of 'union' and 'forEach' clauses.
Separation of 'v1' FHIRView test from our custom enhanced tests.
Implemented 'getResourceKey()' function.
Implementing 'collection' and singular value selection semantics for DirectSelection.
…ons.

Implementing 'r-keys' functions: getReferenceKey() and getResourceKey().
Cleaning up our extended FhirView tests.
…to allow FHIRView paths like: forEach = 'where(gender='male').name,
…lemented as functions in (StandardFunctions).
POC: Implementation of ExtractQueryExecutor
Reverted changes in ExtractQueryTests results to match current input resources.
Refactoring: Making DatasetResult generic with regard to the result type.
# Conflicts:
#	fhirpath/src/main/java/au/csiro/pathling/sql/boundary/DecimalBoundaryFunction.java
# Conflicts:
#	deployment/helm/pom.xml
#	encoders/pom.xml
#	encoders/src/main/java/au/csiro/pathling/config/EncodingConfiguration.java
#	fhir-server/pom.xml
#	fhir-server/src/main/java/au/csiro/pathling/aggregate/AggregateExecutor.java
#	fhir-server/src/main/java/au/csiro/pathling/async/AsyncAspect.java
#	fhir-server/src/main/java/au/csiro/pathling/async/JobProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/caching/EntityTagInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/config/AuthorizationConfiguration.java
#	fhir-server/src/main/java/au/csiro/pathling/config/ServerConfiguration.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/DiagnosticContext.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/ErrorHandlingInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/errors/ErrorReportingInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/extract/ResultProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/ConformanceProvider.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/FhirServer.java
#	fhir-server/src/main/java/au/csiro/pathling/fhir/SmartConfigurationInterceptor.java
#	fhir-server/src/main/java/au/csiro/pathling/search/SearchExecutor.java
#	fhir-server/src/main/java/au/csiro/pathling/security/SecurityConfiguration.java
#	fhir-server/src/test/java/au/csiro/pathling/aggregate/DrillDownBuilderTest.java
#	fhir-server/src/test/java/au/csiro/pathling/extract/ExtractRequestBuilder.java
#	fhir-server/src/test/java/au/csiro/pathling/fhir/EntityTagInterceptorTest.java
#	fhir-server/src/test/java/au/csiro/pathling/fhirpath/parser/ParserTest.java
#	fhir-server/src/test/java/au/csiro/pathling/jmh/SpringBootJmhContext.java
#	fhir-server/src/test/java/au/csiro/pathling/test/IntegrationTestDependencies.java
#	fhirpath/pom.xml
#	fhirpath/src/main/java/au/csiro/pathling/QueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/QueryHelpers.java
#	fhirpath/src/main/java/au/csiro/pathling/aggregate/AggregateQueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/aggregate/AggregateRequest.java
#	fhirpath/src/main/java/au/csiro/pathling/config/StorageConfiguration.java
#	fhirpath/src/main/java/au/csiro/pathling/extract/ExtractQueryExecutor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/FhirPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Materializable.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/NonLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Numeric.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Referrer.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/ResourcePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/Temporal.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/TerminologyUtils.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/UntypedResourcePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/comparison/CodingComparator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/definition/ResourceDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/BooleanPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/CodingPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DatePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DateTimePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/DecimalPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ElementDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ElementPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ExtensionPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/IntegerPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/QuantityPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ReferenceExtensionDefinition.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/ReferencePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/StringPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/element/TimePath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/AggregateFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/Arguments.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/BooleansTestFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/CountFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/EmptyFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ExistsFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ExtensionFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/FirstFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/IifFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NamedFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NamedFunctionInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/NotFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/OfTypeFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ResolveFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/ReverseResolveFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/SumFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/UntilFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/WhereFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/DesignationFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/DisplayFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/MemberOfFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/PropertyFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/SubsumesFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/function/terminology/TranslateFunction.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/BooleanLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/CodingLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DateLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DateTimeLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/DecimalLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/IntegerLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/LiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/NullLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/QuantityLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/StringLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/literal/TimeLiteralPath.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BinaryOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BinaryOperatorInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/BooleanOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/CombineOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/Comparable.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/ComparisonOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/DateArithmeticOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/MathOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/MembershipOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/Operator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/PathTraversalInput.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/operator/PathTraversalOperator.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/InvocationVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/LiteralTermVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/ParserContext.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/TermVisitor.java
#	fhirpath/src/main/java/au/csiro/pathling/fhirpath/parser/Visitor.java
#	fhirpath/src/test/java/au/csiro/pathling/UnitTestDependencies.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/CanBeCombinedWithTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/BooleansTestFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/ResolveFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/UntilFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/DesignationFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/PropertyFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/SubsumesFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/function/terminology/TranslateFunctionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/ComparisonOperatorTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/DateArithmeticTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/MathOperatorQuantityTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/MathOperatorTest.java
#	fhirpath/src/test/java/au/csiro/pathling/fhirpath/operator/QuantityOperatorsPrecisionTest.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/Assertions.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/BaseFhirPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/ElementPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/FhirPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/LiteralPathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/assertions/ResourcePathAssertion.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ElementPathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ParserContextBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/ResourcePathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/builders/UntypedResourcePathBuilder.java
#	fhirpath/src/test/java/au/csiro/pathling/test/helpers/SparkHelpers.java
#	lib/R/pom.xml
#	lib/import/pom.xml
#	lib/js/pom.xml
#	lib/python/Dockerfile
#	lib/python/pom.xml
#	library-api/pom.xml
#	library-api/src/main/java/au/csiro/pathling/library/PathlingContext.java
#	library-api/src/main/java/au/csiro/pathling/library/io/sink/DataSinkBuilder.java
#	library-api/src/main/java/au/csiro/pathling/library/io/source/AbstractSource.java
#	library-api/src/main/java/au/csiro/pathling/library/io/source/QueryableDataSource.java
#	library-api/src/main/java/au/csiro/pathling/library/query/QueryDispatcher.java
#	library-api/src/test/java/au/csiro/pathling/library/PathlingContextTest.java
#	library-runtime/pom.xml
#	pom.xml
#	site/pom.xml
#	terminology/pom.xml
#	terminology/src/main/java/au/csiro/pathling/config/HttpClientCachingConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/HttpClientConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/TerminologyAuthConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/config/TerminologyConfiguration.java
#	terminology/src/main/java/au/csiro/pathling/terminology/DefaultTerminologyService.java
#	terminology/src/main/java/au/csiro/pathling/terminology/TerminologyServiceFactory.java
#	terminology/src/main/java/au/csiro/pathling/terminology/caching/CachingTerminologyService.java
#	terminology/src/test/java/au/csiro/pathling/test/helpers/TerminologyServiceHelpers.java
#	utilities/pom.xml
#	utilities/src/main/java/au/csiro/pathling/utilities/Lists.java
#	utilities/src/main/java/au/csiro/pathling/validation/ValidationUtils.java
@johngrimes
Copy link
Member

We need to make a corresponding change on this branch for c6b06e7, so that we can properly handle the encoding of arrays of complex elements as null.

johngrimes and others added 22 commits July 15, 2024 10:53
…fixing import to make the project compile (moslty javax -> jakarta migration).
Adding (disabled) test cases for infix aggregation in reverseResolve.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request refactoring Improving the design of existing code
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

SQL on FHIR view layer - POC implementation
2 participants