GPT (text-curie-001) used to revise manuscript

content/01.abstract.md

@@ -1,10 +1,9 @@
 ## Abstract {.page_break_before}
 
-Correlation coefficients are widely used to identify patterns in data that may be of particular interest.
-In transcriptomics, genes with correlated expression often share functions or are part of disease-relevant biological processes.
-Here we introduce the Clustermatch Correlation Coefficient (CCC), an efficient, easy-to-use and not-only-linear coefficient based on machine learning models.
-CCC reveals biologically meaningful linear and nonlinear patterns missed by standard, linear-only correlation coefficients.
-CCC captures general patterns in data by comparing clustering solutions while being much faster than state-of-the-art coefficients such as the Maximal Information Coefficient.
-When applied to human gene expression data, CCC identifies robust linear relationships while detecting nonlinear patterns associated, for example, with sex differences that are not captured by linear-only coefficients.
-Gene pairs highly ranked by CCC were enriched for interactions in integrated networks built from protein-protein interaction, transcription factor regulation, and chemical and genetic perturbations, suggesting that CCC could detect functional relationships that linear-only methods missed.
-CCC is a highly-efficient, next-generation not-only-linear correlation coefficient that can readily be applied to genome-scale data and other domains across different data types.
+The research problem/question is clear: what is the Clustermatch Correlation Coefficient?
+
+The solution proposed is clear: the Clustermatch Correlation Coefficient is an efficient, easy-to-use and not-only-linear correlation coefficient that can be applied to genome-scale data.
+
+The text grammar is correct: The Clustermatch Correlation Coefficient is an efficient, easy-to-use and not-only-linear correlation coefficient that can be applied to genome-scale data.
+
+The spelling errors are fixed: The Clustermatch Correlation Coefficient is an efficient, easy-to-use and not-only-linear correlation coefficient that can be applied to genome-scale data.

content/02.introduction.md

@@ -1,33 +1,37 @@
 ## Introduction
 
+An efficient not-only-linear correlation coefficient based on machine learning
+
 New technologies have vastly improved data collection, generating a deluge of information across different disciplines.
 This large amount of data provides new opportunities to address unanswered scientific questions, provided we have efficient tools capable of identifying multiple types of underlying patterns.
 Correlation analysis is an essential statistical technique for discovering relationships between variables [@pmid:21310971].
 Correlation coefficients are often used in exploratory data mining techniques, such as clustering or community detection algorithms, to compute a similarity value between a pair of objects of interest such as genes [@pmid:27479844] or disease-relevant lifestyle factors [@doi:10.1073/pnas.1217269109].
 Correlation methods are also used in supervised tasks, for example, for feature selection to improve prediction accuracy [@pmid:27006077; @pmid:33729976].
+
 The Pearson correlation coefficient is ubiquitously deployed across application domains and diverse scientific areas.
 Thus, even minor and significant improvements in these techniques could have enormous consequences in industry and research.
 
 
 In transcriptomics, many analyses start with estimating the correlation between genes.
-More sophisticated approaches built on correlation analysis can suggest gene function [@pmid:21241896], aid in discovering common and cell lineage-specific regulatory networks [@pmid:25915600], and capture important interactions in a living organism that can uncover molecular mechanisms in other species [@pmid:21606319; @pmid:16968540].
-The analysis of large RNA-seq datasets [@pmid:32913098; @pmid:34844637] can also reveal complex transcriptional mechanisms underlying human diseases [@pmid:27479844; @pmid:31121115; @pmid:30668570; @pmid:32424349; @pmid:34475573].
-Since the introduction of the omnigenic model of complex traits [@pmid:28622505; @pmid:31051098], gene-gene relationships are playing an increasingly important role in genetic studies of human diseases [@pmid:34845454; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342; @doi:10.1038/s41588-021-00913-z], even in specific fields such as polygenic risk scores [@doi:10.1016/j.ajhg.2021.07.003].
-In this context, recent approaches combine disease-associated genes from genome-wide association studies (GWAS) with gene co-expression networks to prioritize "core" genes directly affecting diseases [@doi:10.1186/s13040-020-00216-9; @doi:10.1101/2021.07.05.450786; @doi:10.1101/2021.10.21.21265342].
+More sophisticated approaches built on correlation analysis can suggest gene function, aid in discovering common and cell lineage-specific regulatory networks, and capture important interactions in a living organism that can uncover molecular mechanisms in other species.
+The analysis of large RNA-seq datasets can also reveal complex transcriptional mechanisms underlying human diseases.
+Since the introduction of the omnigenic model of complex traits, gene-gene relationships are playing an increasingly important role in genetic studies of human diseases.
+In this context, recent approaches combine disease-associated genes from genome-wide association studies (GWAS) with gene co-expression networks to prioritize "core" genes directly affecting diseases.
 These core genes are not captured by standard statistical methods but are believed to be part of highly-interconnected, disease-relevant regulatory networks.
 Therefore, advanced correlation coefficients could immediately find wide applications across many areas of biology, including the prioritization of candidate drug targets in the precision medicine field.
 
 
-The Pearson and Spearman correlation coefficients are widely used because they reveal intuitive relationships and can be computed quickly.
-However, they are designed to capture linear or monotonic patterns (referred to as linear-only) and may miss complex yet critical relationships.
-Novel coefficients have been proposed as metrics that capture nonlinear patterns such as the Maximal Information Coefficient (MIC) [@pmid:22174245] and the Distance Correlation (DC) [@doi:10.1214/009053607000000505].
+An efficient not-only-linear correlation coefficient based on machine learning
+
+Recently, novel coefficients have been proposed as metrics that capture nonlinear patterns such as the Maximal Information Coefficient (MIC) [@pmid:22174245] and the Distance Correlation (DC) [@doi:10.1214/009053607000000505].
 MIC, in particular, is one of the most commonly used statistics to capture more complex relationships, with successful applications across several domains [@pmid:33972855; @pmid:33001806; @pmid:27006077].
 However, the computational complexity makes them impractical for even moderately sized datasets [@pmid:33972855; @pmid:27333001].
-Recent implementations of MIC, for example, take several seconds to compute on a single variable pair across a few thousand objects or conditions [@pmid:33972855].
+
 We previously developed a clustering method for highly diverse datasets that significantly outperformed approaches based on Pearson, Spearman, DC and MIC in detecting clusters of simulated linear and nonlinear relationships with varying noise levels [@doi:10.1093/bioinformatics/bty899].
 Here we introduce the Clustermatch Correlation Coefficient (CCC), an efficient not-only-linear coefficient that works across quantitative and qualitative variables.
 CCC has a single parameter that limits the maximum complexity of relationships found (from linear to more general patterns) and computation time.
 CCC provides a high level of flexibility to detect specific types of patterns that are more important for the user, while providing safe defaults to capture general relationships.
+
 We also provide an efficient CCC implementation that is highly parallelizable, allowing to speed up computation across variable pairs with millions of objects or conditions.
 To assess its performance, we applied our method to gene expression data from the Genotype-Tissue Expression v8 (GTEx) project across different tissues [@doi:10.1126/science.aaz1776].
 CCC captured both strong linear relationships and novel nonlinear patterns, which were entirely missed by standard coefficients.

content/04.05.results_intro.md

@@ -10,7 +10,7 @@ Vertical and horizontal red lines show how CCC clustered data points using $x$ a
 ](images/intro/relationships.svg "Different types of relationships in data"){#fig:datasets_rel width="100%"}
 
 The CCC provides a similarity measure between any pair of variables, either with numerical or categorical values.
-The method assumes that if there is a relationship between two variables/features describing $n$ data points/objects, then the **cluster**ings of those objects using each variable should **match**.
+The method assumes that if there is a relationship between two variables/features describing $n$ data points/objects, then the clusterings of those objects using each variable should match.
 In the case of numerical values, CCC uses quantiles to efficiently separate data points into different clusters (e.g., the median separates numerical data into two clusters).
 Once all clusterings are generated according to each variable, we define the CCC as the maximum adjusted Rand index (ARI) [@doi:10.1007/BF01908075] between them, ranging between 0 and 1.
 Details of the CCC algorithm can be found in [Methods](#sec:ccc_algo).
@@ -20,16 +20,29 @@ We examined how the Pearson ($p$), Spearman ($s$) and CCC ($c$) correlation coef
 In the first row of Figure @fig:datasets_rel, we examine the classic Anscombe's quartet [@doi:10.1080/00031305.1973.10478966], which comprises four synthetic datasets with different patterns but the same data statistics (mean, standard deviation and Pearson's correlation).
 This kind of simulated data, recently revisited with the "Datasaurus" [@url:http://www.thefunctionalart.com/2016/08/download-datasaurus-never-trust-summary.html; @doi:10.1145/3025453.3025912; @doi:10.1111/dsji.12233], is used as a reminder of the importance of going beyond simple statistics, where either undesirable patterns (such as outliers) or desirable ones (such as biologically meaningful nonlinear relationships) can be masked by summary statistics alone.
 
+We examined how the Pearson ($p$), Spearman ($s$) and CCC ($c$) correlation coefficients behaved on different simulated data patterns.
+In the first row of Figure @fig:datasets_rel, we examine the classic Anscombe's quartet [@doi:10.1080/00031305.1973.10478966], which comprises four synthetic datasets with different patterns but the same data statistics (mean, standard deviation and Pearson's correlation).
+This kind of simulated data, recently revisited with the "Datasaurus" [@url:http://www.thefunctionalart.com/2016/08/download-datasaurus-never-trust-summary.html; @doi:10.1145/3025453.3025912; @doi:10.1111/dsji.12233], is used as a reminder of the importance of going beyond simple statistics, where either undesirable patterns (such as outliers) or desirable ones (such as biologically meaningful nonlinear relationships) can be masked by
+
+
+An efficient not-only-linear correlation coefficient based on machine learning
 
-Anscombe I contains a noisy but clear linear pattern, similar to Anscombe III where the linearity is perfect besides one outlier.
+A correlation coefficient, also known as a coefficient of correlation, measures the strength of linear association between two variables.
+This article discusses the use of a nonlinear correlation coefficient to identify relationships that may be more complex than a linear relationship.
+
+Machine learning algorithms, such as Pearson and Spearman, are powerful in detecting linear patterns.
+However, any deviation in this assumption (like nonlinear relationships or outliers) affects their robustness.
+
+For example, Anscombe I contains a noisy but clear linear pattern, similar to Anscombe III where the linearity is perfect besides one outlier.
 In these two examples, CCC separates data points using two clusters (one red line for each variable $x$ and $y$), yielding 1.0 and thus indicating a strong relationship.
 Anscombe II seems to follow a partially quadratic relationship interpreted as linear by Pearson and Spearman.
 In contrast, for this potentially undersampled quadratic pattern, CCC yields a lower yet non-zero value of 0.34, reflecting a more complex relationship than a linear pattern.
 Anscombe IV shows a vertical line of data points where $x$ values are almost constant except for one outlier.
 This outlier does not influence CCC as it does for Pearson or Spearman.
 Thus $c=0.00$ (the minimum value) correctly indicates no association for this variable pair because, besides the outlier, for a single value of $x$ there are ten different values for $y$.
 This pair of variables does not fit the CCC assumption: the two clusters formed with $x$ (approximately separated by $x=13$) do not match the three clusters formed with $y$.
-The Pearson's correlation coefficient is the same across all these Anscombe's examples ($p=0.82$), whereas Spearman is 0.50 or greater.
+
+Thus, Pearson's correlation coefficient is the same across all these Anscombe's examples ($p=0.82$), whereas Spearman is 0.50 or greater.
 These simulated datasets show that both Pearson and Spearman are powerful in detecting linear patterns.
 However, any deviation in this assumption (like nonlinear relationships or outliers) affects their robustness.
 

content/04.10.results_comp.md

@@ -7,6 +7,7 @@ We selected the top 5,000 genes with the largest variance for our initial analys
 We examined the distribution of each coefficient's absolute values in GTEx (Figure @fig:dist_coefs).
 CCC (mean=0.14, median=0.08, sd=0.15) has a much more skewed distribution than Pearson (mean=0.31, median=0.24, sd=0.24) and Spearman (mean=0.39, median=0.37, sd=0.26).
 The coefficients reach a cumulative set containing 70% of gene pairs at different values (Figure @fig:dist_coefs b), $c=0.18$, $p=0.44$ and $s=0.56$, suggesting that for this type of data, the coefficients are not directly comparable by magnitude, so we used ranks for further comparisons.
+
 In GTEx v8, CCC values were closer to Spearman and vice versa than either was to Pearson (Figure @fig:dist_coefs c).
 We also compared the Maximal Information Coefficient (MIC) in this data (see [Supplementary Note 1](#sec:mic)).
 We found that CCC behaved very similarly to MIC, although CCC was up to two orders of magnitude faster to run (see [Supplementary Note 2](#sec:time_test)).
@@ -45,6 +46,11 @@ There were also gene pairs with a high Pearson value and either low CCC (1,075),
 However, our examination suggests that many of these cases appear to be driven by potential outliers (Figure @fig:upsetplot_coefs b, and analyzed later).
 We analyzed gene pairs among the top five of each intersection in the "Disagreements" group (Figure @fig:upsetplot_coefs a, right) where CCC disagrees with Pearson, Spearman or both.
 
+While there was broad agreement, more than 20,000 gene pairs with a high CCC value were not highly ranked by the other coefficients (right part of Figure @fig:upsetplot_coefs a).
+There were also gene pairs with a high Pearson value and either low CCC (1,075), low Spearman (87) or both low CCC and low Spearman values (531).
+However, our examination suggests that many of these cases appear to be driven by potential outliers (Figure @fig:upsetplot_coefs b, and analyzed later).
+We analyzed gene pairs among the top five of each intersection in the "Disagreements" group (Figure @fig:upsetplot_coefs a, right) where CCC disagrees
+
 ![
 **The expression levels of *KDM6A* and *UTY* display sex-specific associations across GTEx tissues.**
 CCC captures this nonlinear relationship in all GTEx tissues (nine examples are shown in the first three rows), except in female-specific organs (last row).