Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
thomasyu888 committed Oct 29, 2024
1 parent 983aeb6 commit 568cc87
Showing 1 changed file with 49 additions and 21 deletions.
70 changes: 49 additions & 21 deletions scripts/table_updates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,28 @@ BPC Table Update
[![automated](https://img.shields.io/docker/cloud/automated/sagebionetworks/genie-bpc-pipeline-table-updates)](https://hub.docker.com/r/sagebionetworks/genie-bpc-pipeline-table-updates)
![status](https://img.shields.io/docker/cloud/build/sagebionetworks/genie-bpc-pipeline-table-updates)

Installation and Setup
----------------------
### Python version
Make sure you have Python 3.8 installed.


### Install the required packages
This folder contains multiple scripts to update the tables required for the BPC pipeline. Note there are two separate update scripts right now
that may require different synapseclient versions. We encourage using different conda environments for each script.

```
pip install -r requirements.txt
```
# Service catalog instance
Use a t3.2xlarge ec2 instance for large memory requirement.

### Synapse Credential
# Synapse Credential
Please make sure you have the [.synapseConfig file](https://help.synapse.org/docs/Client-Configuration.1985446156.html)

### Service catalog instance
Use a t3.2xlarge ec2 instance for large memory requirement.

Putting it all together.
# Update data element catalog + table schema

## Installation and Setup

### Python version
Make sure you have Python 3.8 and conda installed.

### Install the required packages
```
pip install 'synapseclient[pandas] == 2.7.2'
```

> [!NOTE]
> Due to this tool using an older version of the python client, until there is bandwidth to update
Expand All @@ -31,26 +34,51 @@ Putting it all together.
> AttributeError: 'dict' object has no attribute 'endswith'
> ```

```
# Make sure you have anaconda installed
conda create -n genie-table-update python=3.8
conda activate genie-table-update
pip install -r requirements.txt
conda create -n genie-table-update-precursor python=3.8
conda activate genie-table-update-precursor
pip install 'synapseclient[pandas] == 2.7.2'
```

Usage
-----
### Prepare the Synapse tables to be updated
### Usage

Prepare the Synapse tables to be updated
> **_NOTE:_** ONLY need to be executed when there is a new version of PRISSMM data dictionary
##### Step 1. Update the Data Catalog
python update_data_element_catalog.py -v [prissmm_version_number]
##### Step 2. Update the table schema
python update_table_schema.py

### Update the Synapse Tables with data

# Update Data Table

The `update_data_table.py` script is used to update the BPC internal tables.

## Installation and Setup

### Python version
Make sure you have Python 3.9+ and conda installed.

### Install the required packages

```
pip install -r requirements.txt
```

```
conda create -n genie-table-update python=3.10
conda activate genie-table-update
pip install -r requirements.txt
```

### Usage
Update the Synapse Tables with data

#### Primary Case Tables
python update_data_table.py -m [version_comment] primary
#### IRR Case Tables
python update_data_table.py -m [version_comment] irr

This is to run the script manually, there is a nextflow workflow associated with this script.

0 comments on commit 568cc87

Please sign in to comment.