First release of orchestration pipeline

This release contains the logic to run the unified pipeline with that was used for running the 24.12 data release for the Open Targets Platform

What's Changed

feat: support for dependabot by @d0choa in #3
feat: initial config and airflow layer from gentropy by @project-defiant in #1
feat: fetch manifest from github by @project-defiant in #4
feat: add interface with batch operators and refactor the utils by @ireneisdoomed in #7
chore: start Poetry shell at the end of environment configuration by @tskir in #6
docs: update and migrate from Gentropy by @tskir in #5
refactor: code clenaup by @project-defiant in #9
fix: pre-commit syntax in Makefile by @tskir in #8
feat: platform pis stage first prototype by @javfg in #11
feat: finngen ingestion by @project-defiant in #13
feat: pis stage for the platform orchestration dag by @javfg in #14
feat: extract config for finngen ingestion to yaml file by @project-defiant in #15
feat: Allow gcs paths without prefix by @project-defiant in #16
feat: move ukb_ppp_eur datasource ingestion to orchestration by @project-defiant in #17
docs: fix README formatting by @tskir in #19
docs: describe gwas catalog data bucket content by @project-defiant in #20
feat: synced gentropy dags and latest configs to orchestration by @project-defiant in #21
feat: add FinnGen UKB meta-analysis ingestion DAG by @tskir in #22
feat: add validation to genetics etl by @project-defiant in #18
feat: include LD_CLUMPED QC flag in credible set validation by @d0choa in #23
feat: finemapping template and DAG for UKB PPP by @tskir in #10
feat: add ETL stage to the platform pipeline by @javfg in #24
fix: platform becomes pipeline dag by @javfg in #25
refactor: genetic dags by @project-defiant in #26
feat: allow for force reinstall dependencies on cluster when set up by @project-defiant in #32
feat: GWAS catalog top-hit DAG by @d0choa in #33
feat: gwas_catalog_sumstats_pics DAG and harmonised paths by @d0choa in #35
feat: variant-index refactoring by @project-defiant in #29
refactor: clarify paths by @d0choa in #36
fix(vep_operator): change dtype for pos to int for correct sorting by @project-defiant in #38
feat(ukb_ppp_eur) dags refactoring by @project-defiant in #39
chore(finngen): params and documentation update by @project-defiant in #44
feat: working genetics_etl including L2G by @d0choa in #45
chore(eqtl_catalogue): params and documentation update by @project-defiant in #41
fix(variant_to_vcf): increase batch job size by @project-defiant in #46
fix(yarn): add dataproc parameter to hydra comamnd parsing function in gentropy step by @project-defiant in #40
chore(finngen_ingestion): update configuration by @project-defiant in #47
feat(susie_finemapper): ensure proper output paths by @project-defiant in #48
feat(add_efo_biosample): add input to orchestration by @Tobi1kenobi in #52
feat: l2g evidence generation step added to genetics etl by @DSuveges in #53
fix(vep): change vep outputs extension to .json by @ireneisdoomed in #56
feat: locus to gene evidence step - polished by @DSuveges in #58
feat(gwas catalog sumstats): finemapping by @project-defiant in #51
feat(credible set qc dag): added dag and docs by @project-defiant in #59
feat(freeze_5): november data release freeze5 by @project-defiant in #60
refactor: reorder tasks in genetics_etl by @project-defiant in #62
fix: prevent code from listing buckets at compile time by @project-defiant in #61
perf(efm): enhanced flexibility mode in genetics etl cluster by @project-defiant in #63
chore: name l2g run with release id and move download_from_hub param by @ireneisdoomed in #65
feat: add task to genetics etl to generate association data by @vivienho in #64
feat: updating invalid qc reasons for validation by @DSuveges in #66
feat: refactoring vep annotation step by @project-defiant in #67
chore: configure hf repo in l2g predicting step by @ireneisdoomed in #68
feat: freeze 6 improvements by @project-defiant in #69
fix: changing to the 99% file and using a different column for fine mapping ingestion by @Daniel-Considine in #70
chore: improve versioning of l2g model in training step by @ireneisdoomed in #72
feat: configuration for etl run with gentropy by @project-defiant in #71
chore: set l2g_threshold to 0.05 by @ireneisdoomed in #74
chore: drop old dags by @project-defiant in #75
chore: request additional vep fields for variant index by @DSuveges in #77
fix: gnomad variant index and ld preprocess by @DSuveges in #78
feat(curation update): genetics curation update 24.11.20 by @project-defiant in #80
feat(docs): add command to update the documentation in buckets by @project-defiant in #76
feat: changing to 95% file for finngen ingestion by @Daniel-Considine in #83
feat: adding conservation VEP plugin for variant annotation by @DSuveges in #85
fix: small fixes for the unified pipeline by @javfg in #86