feat: include LD_CLUMPED QC flag in credible set validation

[d0choa]

We were missing LD_CLUMPING from the credible set invalid flags creating artificially large number of credible sets:

In [36]: cs.df.filter(f.col("studyLocusId") == "6629262360041376681").select("studyLocusId", "variantId", "studyId", "beta", "pValueMantissa", "pValueExponent", f.size("locus").alias("locusSize"), "qualityControls").show()
+-------------------+---------------+----------+-----------+--------------+--------------+---------+--------------------+
|       studyLocusId|      variantId|   studyId|       beta|pValueMantissa|pValueExponent|locusSize|     qualityControls|
+-------------------+---------------+----------+-----------+--------------+--------------+---------+--------------------+
|6629262360041376681|17_28367840_G_A|GCST006585|  1.2908332|           9.0|         -1306|        1|                  []|
|6629262360041376681|17_28367840_G_A|GCST006585|  1.2268604|           5.0|         -1071|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|   1.174808|           8.0|          -883|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|   1.168157|           1.0|          -859|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|   1.131848|           8.0|          -806|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  1.1367371|           7.0|          -760|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|   1.114209|           1.0|          -722|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  1.0570763|           3.0|          -614|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  1.0583832|           3.0|          -600|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585| -0.9495574|           1.0|          -463|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  0.9657885|           8.0|          -436|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|-0.90764076|           1.0|          -422|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585| -0.8891719|           2.0|          -395|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  0.9070528|           4.0|          -387|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  0.9075735|           5.0|          -385|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|  0.8684036|           3.0|          -349|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585|-0.85740745|           4.0|          -349|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585| 0.83947664|           4.0|          -343|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585| 0.85441273|           2.0|          -342|       -1|[Explained by a m...|
|6629262360041376681|17_28367840_G_A|GCST006585| 0.84799707|           1.0|          -323|       -1|[Explained by a m...|
+-------------------+---------------+----------+-----------+--------------+--------------+---------+--------------------+
only showing top 20 rows


In [37]: cs.validate_lead_pvalue(pvalue_cutoff=1e-5).valid_rows(["AMBIGUOUS_STUDY", "FAILED_STUDY", "MISSING_STUDY", "NO_GENOMIC_LOCATION_FLAG", "COMPOSITE
    ...: _FLAG", "INCONSISTENCY_FLAG", "PALINDROMIC_ALLELE_FLAG", "SUBSIGNIFICANT_FLAG"]).df.groupBy("studyLocusId").count().sort(f.col("count").desc()).sh
    ...: ow()
+--------------------+-----+
|        studyLocusId|count|
+--------------------+-----+
| 6629262360041376681|  589|
| 4626398603852685151|  128|
|-2592912666549793099|  128|
|-7802876325096777266|  128|
|-3947655064423442357|  128|
|-2404281503146502601|  128|
|-5678344386350504285|  128|
|-8707989057387840331|  128|
|-7101912826718209864|  128|
|-7887587128982974677|  128|
| 9070000386217697685|  128|
| 2654418611062674803|  128|
| 6936480237625281064|  124|
| 3811852029307275816|  114|
|  -62926400198760996|  113|
| 6582716589660682537|  105|
| 8160639039739847201|   98|
| 7805902697984171135|   97|
|-1872181628327223997|   93|
|-6397465531938250860|   77|
+--------------------+-----+
only showing top 20 rows