Skip to content

Commit

Permalink
explorer to teva
Browse files Browse the repository at this point in the history
  • Loading branch information
svittoz committed Apr 23, 2024
1 parent 81d4bdb commit b440c6f
Show file tree
Hide file tree
Showing 9 changed files with 61 additions and 60 deletions.
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# OMOP Explorer - Config
# OMOP Teva - Config

## Table configuration

All plots generated by ```eds_scikit.plot.generate_omop_explorer``` are based on the configuration file ```eds_scikit.plot.default_omop_explorer_config```.
All plots generated by ```eds_scikit.plot.generate_omop_teva``` are based on the configuration file ```eds_scikit.plot.default_omop_teva_config```.

A table configuration is defined by :

- category columns list
- date column
- category columns mapping

=== "Default condition explorer configuration"
=== "Default condition teva configuration"

```python
"condition_occurrence": {
Expand All @@ -33,7 +33,7 @@ A table configuration is defined by :
},
```

=== "Custom diabete condition explorer configuration"
=== "Custom diabete condition teva configuration"

```python
"condition_occurrence": {
Expand All @@ -59,27 +59,27 @@ A table configuration is defined by :

## Specifying table configuration

To specify configuration, simply update ```default_omop_explorer_config``` and pass it to ```generate_omop_explorer```.
To specify configuration, simply update ```default_omop_teva_config``` and pass it to ```generate_omop_teva```.

```python
from eds_scikit.plot import generate_omop_explorer
from eds_scikit.io.omop_explorer_default_config import default_omop_explorer_config
from eds_scikit.plot import generate_omop_teva
from eds_scikit.io.omop_teva_default_config import default_omop_teva_config

omop_explorer_config = default_omop_explorer_config
omop_teva_config = default_omop_teva_config

condition_mapper = {
"condition_source_value": {"has_diabete": r"^E10|^E11|^E12|^E13|^E14|O24"}
}

omop_explorer_config["condition_occurrence"]["mapper"].update(condition_mapper)
omop_teva_config["condition_occurrence"]["mapper"].update(condition_mapper)

start_date, end_date = "2021-01-01", "2021-12-01"
generate_omop_explorer(data=data,
generate_omop_teva(data=data,
start_date=start_date,
end_date=end_date,
explorer_config=omop_explorer_config)
teva_config=omop_teva_config)
```

!!! warning "Adding a new table in default_omop_explorer_config"
For now, if you want to generate a dashboard for a table which do not exists in default_omop_explorer_config you must redefine and update ```generate_omop_explorer```.
!!! warning "Adding a new table in default_omop_teva_config"
For now, if you want to generate a dashboard for a table which do not exists in default_omop_teva_config you must redefine and update ```generate_omop_teva```.

Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Custom Explorer
# Custom Teva

OMOP-explorer module can also be applied to any dataframe. User must use ```reduce_table``` and ```visualize_table``` from ```eds_scikit.plot.table_viz```.
OMOP-Teva module can also be applied to any dataframe. User must use ```reduce_table``` and ```visualize_table``` from ```eds_scikit.plot.table_viz```.

!!! warning "Make sure to specify categorical columns with less then 50 values."
Use the function ```eds_scikit.plot.table_viz.map_column``` to reduce columns volumetry.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# OMOP Explorer
# OMOP Teva

The OMOP Explorer module of [eds-scikit](https://github.com/aphp/eds-scikit) supports data scientists working on OMOP data. Its main objective is to allow quick OMOP tables exploration by displaying __cartesian product values volumetry__ over time.
The OMOP Teva module of [eds-scikit](https://github.com/aphp/eds-scikit) supports data scientists working on OMOP data.
OMOP Teva generates an interactive dashboard for each OMOP table, allowing timely visualization of the volumes associated with each combination of values. This provides a general overview of the possible values, their relative importance and allows to detect quickly possible bias.

```vegalite
{
Expand Down Expand Up @@ -866,46 +867,46 @@ The OMOP Explorer module of [eds-scikit](https://github.com/aphp/eds-scikit) sup
}
```

<!-- --8<-- [start:omop-explorer] -->
<!-- --8<-- [start:omop-teva] -->

=== card {: href=/functionalities/omop-explorer/quick-use-omop }
=== card {: href=/functionalities/omop-teva/quick-use-omop }

:fontawesome-solid-file-medical:
**OMOP Explorer - Quick use**
**OMOP Teva - Quick use**

---

Quick use for OMOP dataset exploration.

=== card {: href=/functionalities/omop-explorer/configuration-omop }
=== card {: href=/functionalities/omop-teva/configuration-omop }

:fontawesome-solid-gears:
**OMOP Explorer - Config**
**OMOP Teva - Config**

---

See how your can configurate your own OMOP dashboards .

=== card {: href=/functionalities/omop-explorer/custom-explorer }
=== card {: href=/functionalities/omop-teva/custom-teva }

:fontawesome-solid-file:
**Custom Explorer**
**Custom Teva**

---

For any dataframe exploration.

=== card {: href=/functionalities/omop-explorer/omop-explorer-example }
=== card {: href=/functionalities/omop-teva/omop-teva-example }

:fontawesome-solid-chart-line:
**OMOP Explorer - Example**
**OMOP Teva - Example**

---

A toy example of what can be obtained with this module.



<!-- --8<-- [end:omop-explorer] -->
<!-- --8<-- [end:omop-teva] -->

See [Adjusting for the progressive digitization of health records](https://www.medrxiv.org/content/10.1101/2023.08.17.23294220v1) for a better understanding about temporal bias in EHR data and [EDS-Teva](https://github.com/aphp/edsteva) for a more complete library on how to handle those bias.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# OMOP Explorer example
# OMOP Teva example

OMOP Explorer generates an interactive dashboard for each table, allowing visualization of the volumes associated with each combination of values. This provides a general overview of the possible values, their relative importance and allows to detect quickly possible bias.
OMOP Teva generates an interactive dashboard for each table, allowing visualization of the volumes associated with each combination of values. This provides a general overview of the possible values, their relative importance and allows to detect quickly possible bias.

The dashboard below provides an example for the __visit_occurrence__ table.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Quick use - OMOP Explorer
# OMOP Teva - Quick use

This tutorial demonstrates how the OMOP explorer module can be quickly used to generate OMOP tables dashboard.
This tutorial demonstrates how the OMOP teva module can be quickly used to generate OMOP tables dashboard.

Simply apply ```generate_omop_explorer``` function after loading the data. It will create a directory with one HTML per OMOP table.
Simply apply ```generate_omop_teva``` function after loading the data. It will create a directory with one HTML per OMOP table.

??? tip "Loading dataset"

Expand All @@ -27,10 +27,10 @@ Simply apply ```generate_omop_explorer``` function after loading the data. It wi
```

```python
from eds_scikit.plot import generate_omop_explorer
from eds_scikit.plot import generate_omop_teva

start_date, end_date = "2021-01-01", "2021-12-01"
generate_omop_explorer(data=data, start_date=start_date, end_date=end_date)
generate_omop_teva(data=data, start_date=start_date, end_date=end_date)
```

=== "Visit occurrence"
Expand Down Expand Up @@ -17505,7 +17505,7 @@ generate_omop_explorer(data=data, start_date=start_date, end_date=end_date)
=== "Condition occurrence (diabete)"

!!! warning "In this example ```condition_source_value``` is splited between diabetic and non-diabetic conditions."
You can modify dashboard configuration by importing ```eds_scikit.plot.default_omop_explorer_config``` and customizing it. See next section for details on how to do it.
You can modify dashboard configuration by importing ```eds_scikit.plot.default_omop_teva_config``` and customizing it. See next section for details on how to do it.

```vegalite
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
default_omop_explorer_config = {
default_omop_teva_config = {
"visit_occurrence": {
"category_columns": [
"care_site_short_name",
Expand Down
4 changes: 2 additions & 2 deletions eds_scikit/plot/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
from eds_scikit.plot.table_viz import map_column, reduce_table, visualize_table
from eds_scikit.plot.omop_explorer import generate_omop_explorer
from eds_scikit.io.omop_explorer_default_config import default_omop_explorer_config
from eds_scikit.plot.omop_teva import generate_omop_teva
from eds_scikit.io.omop_teva_default_config import default_omop_teva_config
28 changes: 14 additions & 14 deletions eds_scikit/plot/omop_explorer.py → eds_scikit/plot/omop_teva.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,28 @@
from loguru import logger

from eds_scikit.io import HiveData
from eds_scikit.io.omop_explorer_default_config import default_omop_explorer_config
from eds_scikit.io.omop_teva_default_config import default_omop_teva_config
from eds_scikit.plot.table_viz import reduce_table, visualize_table


def generate_omop_explorer(
def generate_omop_teva(
data: HiveData,
start_date: str,
end_date: str,
explorer_config: dict = default_omop_explorer_config,
output_dir="omop_explorer",
teva_config: dict = default_omop_teva_config,
output_dir="omop_teva",
):
"""Generate OMOP explorer folder.
"""Generate OMOP teva folder.
Parameters
----------
data : HiveData
start_date : str
end_date : str
explorer_config : dict, optional
OMOP explorer configuration, by default default_omop_explorer_config
teva_config : dict, optional
OMOP teva configuration, by default default_omop_teva_config
output_dir : str, optional
Output directory path, by default "omop_explorer"
Output directory path, by default "omop_teva"
"""

if not os.path.exists(f"{output_dir}/"):
Expand All @@ -38,7 +38,7 @@ def generate_omop_explorer(
visit_occurrence = visit_occurrence.merge(
data.care_site[["care_site_id", "care_site_short_name"]], on="care_site_id"
)
config = explorer_config["visit_occurrence"]
config = teva_config["visit_occurrence"]
visit_columns = [
*config["category_columns"],
config["date_column"],
Expand Down Expand Up @@ -69,7 +69,7 @@ def generate_omop_explorer(
logger.info("Starting procedure_occurrence processing.")

procedure_occurrence = data.procedure_occurrence
config = explorer_config["procedure_occurrence"]
config = teva_config["procedure_occurrence"]
procedure_occurrence = procedure_occurrence.merge(
visit_occurrence, on="visit_occurrence_id", how="left"
)
Expand Down Expand Up @@ -100,7 +100,7 @@ def generate_omop_explorer(
logger.info("Starting condition_occurrence processing.")

condition_occurrence = data.condition_occurrence
config = explorer_config["condition_occurrence"]
config = teva_config["condition_occurrence"]
condition_occurrence = condition_occurrence.merge(
visit_occurrence, on="visit_occurrence_id", how="left"
)
Expand Down Expand Up @@ -131,7 +131,7 @@ def generate_omop_explorer(
logger.info("Starting notes processing.")

note = data.note
config = explorer_config["note"]
config = teva_config["note"]
note = note.merge(visit_occurrence, on="visit_occurrence_id", how="left")
note["visit_occurrence_id"] = note["visit_occurrence_id"].astype(str)

Expand All @@ -154,7 +154,7 @@ def generate_omop_explorer(
logger.info("Starting drug exposure prescription processing.")

drg_exposure = data.drug_exposure_prescription
config = explorer_config["drug_exposure_prescription"]
config = teva_config["drug_exposure_prescription"]
drg_exposure = drg_exposure.merge(
visit_occurrence, on="visit_occurrence_id", how="left"
)
Expand Down Expand Up @@ -183,7 +183,7 @@ def generate_omop_explorer(
logger.info("Starting drug exposure administration processing.")

drg_exposure = data.drug_exposure_administration
config = explorer_config["drug_exposure_administration"]
config = teva_config["drug_exposure_administration"]
drg_exposure = drg_exposure.merge(
visit_occurrence, on="visit_occurrence_id", how="left"
)
Expand Down
16 changes: 8 additions & 8 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,14 @@ nav:
- Emergency Units: functionalities/patients-course/is_emergency.md
- Visit merging: functionalities/patients-course/visit_merging.md
- Consultation dates: functionalities/patients-course/consultation_dates.md
- OMOP explorer:
- functionalities/omop-explorer/index.md
- functionalities/omop-explorer/quick-use-omop.md
# - functionalities/omop-explorer/quick-use-general.md
- functionalities/omop-explorer/configuration-omop.md
- functionalities/omop-explorer/custom-explorer.md
- functionalities/omop-explorer/omop-explorer-example.md
# - functionalities/omop-explorer/detailed-use-omop.md
- OMOP Teva:
- functionalities/omop-teva/index.md
- functionalities/omop-teva/quick-use-omop.md
# - functionalities/omop-teva/quick-use-general.md
- functionalities/omop-teva/configuration-omop.md
- functionalities/omop-teva/custom-teva.md
- functionalities/omop-teva/omop-teva-example.md
# - functionalities/omop-teva/detailed-use-omop.md
- Biology:
- functionalities/biology/index.md
- functionalities/biology/quick-use.md
Expand Down

0 comments on commit b440c6f

Please sign in to comment.