v1.1.10

Recent Updates to the Harmonisation Pipeline

1. Harmonisation Step Changes

• Z-Score Harmonisation: The pipeline now harmonises z-scores in the input file, ensuring consistency for effect size calculations.
• Automatic Conversion of neg_Log_10_P_Value: neg_log_10_p_value fields are now automatically converted to p_values for easier cross-study comparison.

2. Workflow Modifications

• Customisable Input File Names: The pipeline now accepts customised input file names.
• Consistent Naming for GCST Files: If the input file name follows the pattern GCST[0-9]+, the output files will continue to use the GCST naming convention.

3. Output Folder Structure Adjustments

• Reorganisation of Intermediate Files: The intermediate files folder is now numbered for easier navigation.
• Final Output Files: The final output folder includes only the essential files: h.tsv.gz, h.tsv.gz.tbi, h.tsv.gz-meta.yaml, and the running.log.

4. Configuration File Isolation

• Executor Configuration Isolation: The executor configuration file is now isolated, making it easier for users to adapt the pipeline for various executors.

5. Removal of Obsolete Steps

• Rsid Validation Removal: As current rsid values come directly from VCF files, the separate rsid validation step has been removed.

6. Automatic Version Logging

• Dynamic Versioning in Running Log: The hardcoded version entry in the running log has been replaced with an automated version retrieval from the pipeline parameters.

7. Error Handling Strategy

• Pipeline Termination on Error: Users may now terminate the pipeline if an error occurs. However, this is not recommended due to the impact on parallelised jobs.

8.Failed File Management

• Failed Logs in FTP: Previously, files where all variants failed harmonisation (containing only headers) were moved to an internal "failed" folder. Now, the failed log is moved to the FTP directory, enabling users to view data that did not pass harmonisation.

What's Changed

• Fix typo in palindromes by @smlmbrt in #106
• Dev 1.9 by @jiyue1214 in #111
• Updates to a new version v1.1.10 (GWAS_SSF) by @jiyue1214 in #112

New Contributors

• @smlmbrt made their first contribution in #106

Full Changelog: v1.1.9...v1.1.10

Harmonisation pipeline for gwas-ssf

Revise the output of the ci_upper and ci_lower

Full Changelog: v1.1.8...v1.1.9

PRE_GWAS_SSF harmonisation minor revise

What's Changed

Revise the output of the ci_upper and ci_lower

Full Changelog: v1.0.8...v1.0.9

v1.1.8 Slurm harmonisation pipeline for GWAS_SSF

1. Change the cluster setting from LSF to SLURM
2. Convert neg_log_10_p_value to normal p_value
3. Update the gwas-sumstats-tools version to 1.0.6 since the metadata field is updated, especially the field date_metadata_last_modified is renamed
4. Add procps in the docker image to allow nf tower to generate the matrix
5. Tag each process so that we can know the GCST id within one process even though their output name is the same.
6. Update the test.config file to recognise the test data are in the $NXF_ASSETS folder or $HOME/.nextflow/asset folder
7. Update the test yaml data since we updated the field date_last_modified as date_metadata_last_modified
8. Allow the yaml file to be accessible by the count the strand direction step.
9. Combine the harmonisation channel with unmapped channel to generate the log file

What's Changed

• Master fix yue 20240610 by @jiyue1214 in #91

Full Changelog: v1.0.7...v1.1.8

v1.0.8

Updates:

1. Read either variant_id column or rsid column to map variants
2. Update gwas-sumstat-tools version to 1.0.6
3. Add procps in the docker image to allow nf tower to generate the matrix
4. Setting Slurm not LSF as the executor
5. Converting neg_log_10_p_value to normal p_value
6. Tag each process so that we can know the GCST id within one process even though their output name is the same.
7. Update the test.config file to recognise the test data are in the $NXF_ASSETS folder instead of the $HOME/.nextflow/asset folder
8. Update the test yaml data since we updated the field date_last_modified as date_metadata_last_modified
9. Allow the yaml file to be accessible by the count the strand direction step.
10. Pre-gwas-ssf cannot use the latest branch. Update the version with the upcoming release version v1.0.8

What's Changed

• Pre_gwas_ssf_fix by @jiyue1214 in #81
• feat: Migrate from LSF to SLURM (Pre-GWAS SSF) by @karatugo in #83
• modifications on yaml fields and id by @jiyue1214 in #85
• Pre gwas ssf modifications by @jiyue1214 in #88
• update to pre_gwas_ssf harmonisation pipeline by @jiyue1214 in #90

Full Changelog: v1.1.3...v1.0.8

v1.1.7

Full Changelog: v1.1.6...v1.1.7

v1.1.6

Full Changelog: v1.1.5...v1.1.6

v1.0.7

Full Changelog: v1.0.6...v1.0.7

v1.1.5

What's Changed

• change on yaml and id range by @jiyue1214 in #86

Full Changelog: v1.1.4...v1.1.5

v1.0.6

What's Changed

• change on yaml and id range by @jiyue1214 in #86

Full Changelog: v1.0.5...v1.0.6