Compare two datasets
These functions will perform a point-by-point comparison of two datasets :
###Compare two datasets (t-test)
Compute a point-by-point t-test using two datasets. The function also implements permutation testing to identify significant clusters in the obtained statistical time-courses or maps.
To use this function, you must create two merged datasets, where each epoch corresponds to a measure (e.g. the average waveform of a given subject). For a paired-sample t-test, each epoch of the two datasets must correspond to the same subject.
-
Test type. The function can be used to perform a paired-sample t-test, or a two samples t-test.
-
Select whether you want to perform a two-tailed test, a right-tailed test, or a left-tailed test.
-
Reference dataset. Choose one reference dataset, to which the other dataset(s) should be compared.
-
Alpha level. The minimum p-value to consider the result of the statistical test as “significant”.
-
Clustersize-based permutation testing. Follow this link for details.
###Compare two datasets (non parametric)
Compute a point-by-point Wilcoxon test using two datasets. The function also implements permutation testing to identify significant clusters in the obtained statistical time-courses or maps.
To use this function, you must create two merged datasets, where each epoch corresponds to a measure (e.g. the average waveform of a given subject). For a paired-sample t-test, each epoch of the two datasets must correspond to the same subject.
-
Test type. The function can be used to perform a signed rank test, a rank sum test, or a sign test.
-
Select whether you want to perform a two-tailed test, a right-tailed test, or a left-tailed test.
-
Reference dataset. Choose one reference dataset, to which the other dataset(s) should be compared.
-
Alpha level. The minimum p-value to consider the result of the statistical test as “significant”.
-
Clustersize-based permutation testing. Follow this link for details.
###Clustersize based permutation testing
The main issue of point-by-point statistical testing is the correction for multiple comparisons. Permutation testing can be a mean to address this issue (link). Permutation testing computes a distribution of the clusters of significant p values obtained when computing the statistical tests after permuting the datasets. The “strength” of each cluster is estimated using the sum of the test statistic within each significant cluster obtained using the permuted datasets. This distribution of clusters is then used to define a cluster threshold. The cluster threshold is applied to the result of the statistical test applied on the non-permuted datasets, such as to discard all clusters which are below the cluster threshold.
- Clustersize-based permutation testing. Perform a clustersize-based permutation testing. This can be very time consuming.
- Number of permutations. The number of permutations to be used to build the clustersize distribution.
Several methods are proposed to define the threshold distinguishing between “significant” and “non significant” clusters:
- Standard deviation of mean cluster sum. The threshold is defined as a z-score using the mean and standard deviation of the distribution of all significant clusters obtained using permutation testing (e.g. Z>2).
- Standard deviation of maximum cluster sum. The threshold is defined as a z-score using the mean and standard deviation of the distribution of the most significant cluster obtained for each permutation testing (e.g. Z>2).
- Percentile of mean cluster sum. The threshold is defined as a percentile using the distribution z of the distribution of all significant clusters obtained using permutation testing (e.g. >95%; this is equivalent to setting the cluster threshold to p <0.05).
- Percentile of mean cluster sum. The threshold is defined as a percentile using the distribution z of the distribution of the most significant cluster obtained for each permutation testing (e.g. >95%; this is equivalent to setting the cluster threshold to p <0.05).