DQAstats

• DQAstats
  • Installation
    • CRAN Version
    • Development Version
  • Configuration of the tool
  • Example
  • Demo Usage / Deployment Examples
  • Citation
  • More Infos

The R package ‘DQAstats’ provides core functionalities to perform data quality assessment (DQA) of electronic health record data (EHR).

Currently implemented features are:

• descriptive (univariate) analysis of categorical and continuous variables of a source database and a target database
• checks of the extract-transform-load (ETL) pipeline:
  • comparing of distinct values and valid values between the source database and the target database
  • identification of missing/duplicate Data based on a comparison of timestamps
• value conformance checks by comparing the resulting statistics to value constraints (given in a meta data repository (MDR))
• ‘atemporal plausibility’ checks (multivariate)
• ‘uniqueness plausibility’ checks (multivariate)

The tool provides one main function, dqa(), to create a comprehensive PDF document, which presents all statistics and results of the data quality assessment.

Currently supported input data formats / databases:

• CSV files (via R package data.table)
• PostgreSQL (via R package RPostgres)
• ORACLE (via R package RJDBC)

Installation

CRAN Version

DQAstats can be installed directly from CRAN with:

install.packages("DQAstats")

Development Version

You can install the latest development version of DQAstats with:

install.packages("remotes")
remotes::install_github("miracum/dqa-dqastats")

Note: A working LaTeX installation is a prerequisite for using this software (e.g. using the R package tinytex)!

💡 If you want to run this in a dockerized environment you can use the rocker/verse image which has TeX already installed.

Configuration of the tool

The configuration of databases, be it CSV files or SQL-based databases, is done with environment variables, which can be set using the base R command Sys.setenv().

A detailed description, which environment variables need to be set for the specific databases can be found here.

Example

The following code example is intended to provide a minimal working example on how to apply the DQA tool to data. Example data and a corresponding MDR are provided with the R package DQAstats (a working LaTeX installation is a prerequisite for using this software, e.g. by using the R package tinytex; please refer to the DQAstats wiki for further installation instructions).

• Example data: https://github.com/miracum/dqa-dqastats/tree/master/inst/demo_data
• Example MDR: https://github.com/miracum/dqa-dqastats/blob/master/inst/demo_data/utilities/MDR/mdr_example_data.csv

# Load library DQAstats:
library(DQAstats)

# Set environment vars to demo files paths:
Sys.setenv("EXAMPLECSV_SOURCE_PATH" = system.file("demo_data",
                                                  package = "DQAstats"))
Sys.setenv("EXAMPLECSV_TARGET_PATH" = system.file("demo_data",
                                                  package = "DQAstats"))
# Set path to utilities folder where to find the mdr and template files:
utils_path <- system.file("demo_data/utilities",
                          package = "DQAstats")

# Execute the DQA and generate a PDF report:
results <- DQAstats::dqa(
  source_system_name = "exampleCSV_source",
  target_system_name = "exampleCSV_target",
  utils_path = utils_path,
  mdr_filename = "mdr_example_data.csv",
  output_dir = "output/",
  parallel = FALSE
)

# The PDF report is stored at "./output/"

Demo Usage / Deployment Examples

You can test the package without needing to install anything except docker. 💡 For further details, see the Wiki: https://github.com/miracum/dqa-dqastats/wiki/Deployment.

Citation

L.A. Kapsner, J.M. Mang, S. Mate, S.A. Seuchter, A. Vengadeswaran, F. Bathelt, N. Deppenwiese, D. Kadioglu, D. Kraska, and H.-U. Prokosch, Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository, Appl Clin Inform. 12 (2021) 826–835. doi:10.1055/s-0041-1733847.

@article{kapsner2021,
  title = {Linking a {{Consortium}}-{{Wide Data Quality Assessment Tool}} with the {{MIRACUM Metadata Repository}}},
  author = {Kapsner, Lorenz A. and Mang, Jonathan M. and Mate, Sebastian and Seuchter, Susanne A. and Vengadeswaran, Abishaa and Bathelt, Franziska and Deppenwiese, Noemi and Kadioglu, Dennis and Kraska, Detlef and Prokosch, Hans-Ulrich},
  year = {2021},
  month = aug,
  journal = {Applied Clinical Informatics},
  volume = {12},
  number = {04},
  pages = {826--835},
  issn = {1869-0327},
  doi = {10.1055/s-0041-1733847},
  language = {en}
}

More Infos

• about MIRACUM: https://www.miracum.org/
• about the Medical Informatics Initiative: https://www.medizininformatik-initiative.de/index.php/de
• the precursor’s version of the MIRACUM DQA tool (publication): https://gitlab.miracum.org/miracum/dqa/dqa-p21-i2b2