From 79e1739207625ac92eaf7fbac6f0d3dda230d475 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 7 Jan 2025 12:57:32 -0500 Subject: [PATCH 1/3] Bump com.google.api-client:google-api-client-gson from 2.7.0 to 2.7.1 (#1262) --- pipelines/common/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/common/pom.xml b/pipelines/common/pom.xml index 0b18da999..0d1e42412 100644 --- a/pipelines/common/pom.xml +++ b/pipelines/common/pom.xml @@ -50,7 +50,7 @@ com.google.api-client google-api-client-gson - 2.7.0 + 2.7.1 From 95f956500df87e7b415faa019aaeb2ee6ac3eafd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 8 Jan 2025 00:01:51 -0500 Subject: [PATCH 2/3] Bump com.google.guava:guava from 33.3.1-jre to 33.4.0-jre (#1269) --- pipelines/common/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/common/pom.xml b/pipelines/common/pom.xml index 0d1e42412..28ce509e8 100644 --- a/pipelines/common/pom.xml +++ b/pipelines/common/pom.xml @@ -29,7 +29,7 @@ 0.10.1 - 33.3.1-jre + 33.4.0-jre From a11b7128e28bb1333971aaf2b64b1e883d58f99b Mon Sep 17 00:00:00 2001 From: Bashir Sadjad Date: Wed, 8 Jan 2025 00:56:23 -0500 Subject: [PATCH 3/3] Fixed some doc links (#1274) --- doc/docs/concepts/concepts.md | 18 +++++++++--------- doc/docs/tutorials/add_dashboard.md | 2 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/docs/concepts/concepts.md b/doc/docs/concepts/concepts.md index bc456df54..eaf6b9153 100644 --- a/doc/docs/concepts/concepts.md +++ b/doc/docs/concepts/concepts.md @@ -56,7 +56,7 @@ FHIR Resources are transformed into a "Parquet on FHIR" format: * Configurable support for FHIR profiles and extensions * (Optional) In-pipeline 'flattening' of FHIR data using [ViewDefinition](https://build.fhir.org/ig/FHIR/sql-on-fhir-v2/StructureDefinition-ViewDefinition.html) - resources - [read more](views#viewdefinition-resource) + resources - [read more](../views#viewdefinition-resource) ### Loading @@ -78,7 +78,7 @@ full", "incremental", and "merger" pipelines together. * The Pipelines Controller is built on top of pipelines and shares many of the same settings * Using the controller module you can schedule periodic incremental updates or - use the [Web Control Panel](../additional#web-control-panel) to start the + use the [Web Control Panel](../../additional/#web-control-panel) to start the pipeline manually @@ -90,13 +90,13 @@ Choosing the right approach, comes down to a number of factors including the specific use-case, infrastructure constraints of the project, future scalability requirements, and expertise of the team. -| Scenario | Approach | Considerations | -|----------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------| -| Simple relational database to power dashboards or reporting | Custom schema defined as ViewDefinition Resources - [see tutorial](tutorial_lossy_example/) | By design, this will provide a subset of FHIR data in flat tables | -| Horizontally scalable query architecture with lossless FHIR data | Parquet based DWH and distributed query engine (e.g. SparkSQL) - [see tutorial](tutorial_add_dashboard/) | This requires a distributed file system for Parquet files and a distributed query engine. | -| Lossless FHIR DWH with a single process query engine | Parquet based DWH with non-distributed query engine set-up (e.g. single-node Spark or DuckDB) | Scalability and performance (e.g. SparkSQL >> DuckDB) | -| Exploratory data science or ML use cases | Use the generated Parquet files which as _"near lossless"_ for enhanced data science workflows | Can either use the Parquet or custom schema to power dashboards or reports | -| Push FHIR data to a central FHIR-store (e.g., for a Shared Health Record system) | Use the Pipelines Controller to push from a FHIR source to a FHIR sink | Management of the intermediate Parquet files created as part of the pipelines | +| Scenario | Approach | Considerations | +|----------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------| +| Simple relational database to power dashboards or reporting | Custom schema defined as ViewDefinition Resources - [see Views](../views/#query-simplification) | By design, this will provide a subset of FHIR data in flat tables | +| Horizontally scalable query architecture with lossless FHIR data | Parquet based DWH and distributed query engine (e.g. SparkSQL) - [see tutorial](../../tutorials/add_dashboard/) | This requires a distributed file system for Parquet files and a distributed query engine. | +| Lossless FHIR DWH with a single process query engine | Parquet based DWH with non-distributed query engine set-up (e.g. single-node Spark or DuckDB) | Scalability and performance (e.g. SparkSQL >> DuckDB) | +| Exploratory data science or ML use cases | Use the generated Parquet files which as _"near lossless"_ for enhanced data science workflows | Can either use the Parquet or custom schema to power dashboards or reports | +| Push FHIR data to a central FHIR-store (e.g., for a Shared Health Record system) | Use the Pipelines Controller to push from a FHIR source to a FHIR sink | Management of the intermediate Parquet files created as part of the pipelines | ## Query Simplification with Flat Views diff --git a/doc/docs/tutorials/add_dashboard.md b/doc/docs/tutorials/add_dashboard.md index c2c15f11e..99b4a7dcc 100644 --- a/doc/docs/tutorials/add_dashboard.md +++ b/doc/docs/tutorials/add_dashboard.md @@ -148,7 +148,7 @@ that as a bar chart over time. The query is provided below. database to "Single Machine Test Data" or the name you chose. Enter the following as the query as either a raw SQL-on-FHIR query or using -the [pre-defined](../../concepts/#sql-virtual-views) observation_flat views: +the [pre-defined](../../concepts/views) observation_flat views: === "Using SQL-on-FHIR Query"