Merge branch 'main' into il-improve-l2g-tracking

.github/workflows/docs.yaml

@@ -11,7 +11,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.10.8
       - name: Install and configure Poetry

.github/workflows/pytest.yaml

@@ -8,7 +8,7 @@ jobs:
         with:
           fetch-depth: 1
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.10.8
       - name: Install and configure Poetry

CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+- The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.

config/datasets/gcp.yaml

@@ -30,25 +30,20 @@ eqtl_catalogue_paths_imported: ${datasets.inputs}/preprocess/eqtl_catalogue/tabi
 gene_index: ${datasets.outputs}/gene_index
 variant_annotation: ${datasets.outputs}/variant_annotation
 variant_index: ${datasets.outputs}/variant_index
+study_index: ${datasets.outputs}/study_index
 study_locus: ${datasets.outputs}/study_locus
 credible_set: ${datasets.outputs}/credible_set
+summary_statistics: ${datasets.outputs}/summary_statistics
 study_locus_overlap: ${datasets.outputs}/study_locus_overlap
 colocalisation: ${datasets.outputs}/colocalisation
 v2g: ${datasets.outputs}/v2g
 ld_index: ${datasets.outputs}/ld_index
-catalog_study_index: ${datasets.outputs}/catalog_study_index
+catalog_study_index: ${datasets.study_index}/catalog
 catalog_study_locus: ${datasets.study_locus}/catalog_study_locus
-finngen_study_index: ${datasets.outputs}/finngen_study_index
-finngen_summary_stats: ${datasets.outputs}/finngen_summary_stats
+finngen_study_index: ${datasets.study_index}/finngen
+finngen_summary_stats: ${datasets.summary_statistics}/finngen
 from_sumstats_study_locus: ${datasets.study_locus}/from_sumstats
 from_sumstats_pics: ${datasets.credible_set}/from_sumstats
-ukbiobank_study_index: ${datasets.outputs}/ukbiobank_study_index
+ukbiobank_study_index: ${datasets.study_index}/ukbiobank
 l2g_model: ${datasets.outputs}/l2g_model
 l2g_predictions: ${datasets.outputs}/l2g_predictions
-eqtl_catalogue_study_index_out: ${datasets.outputs}/preprocess/eqtl_catalogue/study_index
-eqtl_catalogue_summary_stats_out: ${datasets.outputs}/preprocess/eqtl_catalogue/summary_stats
-
-# Constants
-finngen_release_prefix: FINNGEN_R9
-finngen_sumstat_url_prefix: gs://finngen-public-data-r9/summary_stats/finngen_R9_
-finngen_sumstat_url_suffix: .gz

config/step/finngen.yaml

@@ -1,7 +1,3 @@
 _target_: otg.finngen.FinnGenStep
-finngen_phenotype_table_url: ${datasets.finngen_phenotype_table_url}
-finngen_release_prefix: ${datasets.finngen_release_prefix}
-finngen_sumstat_url_prefix: ${datasets.finngen_sumstat_url_prefix}
-finngen_sumstat_url_suffix: ${datasets.finngen_sumstat_url_suffix}
 finngen_study_index_out: ${datasets.finngen_study_index}
 finngen_summary_stats_out: ${datasets.finngen_summary_stats}

config/step/gwas_catalog.yaml → config/step/gwas_catalog_ingestion.yaml

@@ -4,6 +4,5 @@ catalog_ancestry_files: ${datasets.catalog_ancestries}
 catalog_associations_file: ${datasets.catalog_associations}
 catalog_sumstats_lut: ${datasets.catalog_sumstats_lut}
 variant_annotation_path: ${datasets.variant_annotation}
-ld_index_path: ${datasets.ld_index}
 catalog_studies_out: ${datasets.catalog_study_index}
 catalog_associations_out: ${datasets.catalog_study_locus}

config/step/locus_to_gene.yaml

@@ -11,7 +11,7 @@ predictions_path: ${datasets.l2g_predictions}
 credible_set_path: ${datasets.credible_set}
 variant_gene_path: ${datasets.v2g}
 colocalisation_path: ${datasets.colocalisation}
-study_index_path: ${datasets.catalog_study_index}
+study_index_path: ${datasets.study_index}
 study_locus_overlap_path: ${datasets.study_locus_overlap}
 gold_standard_curation_path: ${datasets.l2g_gold_standard_curation}
 gene_interactions_path: ${datasets.gene_interactions}

config/step/pics.yaml

@@ -1,3 +1,3 @@
 _target_: otg.pics.PICSStep
-study_locus_ld_annotated_in: ${datasets.from_sumstats_study_locus}
-picsed_study_locus_out: ${datasets.from_sumstats_pics}
+study_locus_ld_annotated_in: ???
+picsed_study_locus_out: ???

config/step/study_locus_overlap.yaml

@@ -1,4 +1,4 @@
 _target_: otg.overlaps.OverlapsIndexStep
-study_locus_path: ${datasets.outputs}/catalog_study_locus
-study_index_path: ${datasets.outputs}/catalog_study_index
+study_locus_path: ${datasets.outputs}/credible_set
+study_index_path: ${datasets.outputs}/study_index
 overlaps_index_out: ${datasets.outputs}/study_locus_overlap

config/step/variant_index.yaml

@@ -1,4 +1,4 @@
 _target_: otg.variant_index.VariantIndexStep
 variant_annotation_path: ${datasets.variant_annotation}
-study_locus_path: ${datasets.study_locus}
+credible_set_path: ${datasets.study_locus}
 variant_index_path: ${datasets.variant_index}

docs/python_api/datasource/gwas_catalog/associations.md

@@ -2,4 +2,8 @@
 title: Associations
 ---
 
-::: otg.datasource.gwas_catalog.associations.GWASCatalogAssociations
+::: otg.datasource.gwas_catalog.associations.GWASCatalogCuratedAssociationsParser
+
+---
+
+::: otg.datasource.gwas_catalog.associations.StudyLocusGWASCatalog

docs/python_api/datasource/gwas_catalog/study_index.md

@@ -2,4 +2,8 @@
 title: Study Index
 ---
 
-::: otg.datasource.gwas_catalog.study_index.GWASCatalogStudyIndex
+::: otg.datasource.gwas_catalog.study_index.StudyIndexGWASCatalogParser
+
+---
+
+::: otg.datasource.gwas_catalog.study_index.StudyIndexGWASCatalog

docs/python_api/step/gwas_catalog_ingestion.md

@@ -0,0 +1,5 @@
+---
+title: GWAS Catalog
+---
+
+::: otg.gwas_catalog_ingestion.GWASCatalogIngestionStep