I'm having exactly the same doubt to detect group differences in SC and FC. In our group we're using exactly those 2 techniques: NBS or FDR. It seems there is no well-established technique. There is also the option to use the Bonferroni correction, but it is very conservative and hardly anything passes the test.

After quantifications, we are now using JMP software that can handle multiple group statistical analysis such as one-way and two-way ANOVA and interaction analysis. For voxel-wise analysis, we used Freesurfer, SPM and FSL before. These neuroimaging toolbox all share some similar features and its good to install all of them and choose the one that best suits your data.

I think it is invalid to assume that each element in a connectivity matrix is an independent observation (FDR, Bonferroni). I wonder if there are alternatives to NBS or other, perhaps computationally more efficient implementations? 

In terms of condensing data, general principal or independent component analysis methods, as well as some feature selection algorithms in data mining (e.g. weka software) and strategies such as Maximal relevance and minimal redundancy (mRMR) could be applied.

I am a strong proponent of using a multivariate machine learning approach to identify group differences. Support Vector Machine and multi-variate pattern analysis are the particularly popular one the field of neuroscience (matlab code is easily accessible for both).  This allows you to probe your data and try to optimize group separability in an unbiased way. The best approach to do this with limited sample sizes is to leave one subject out, train a function on the rest of the data (group labels + connectiivty data) and then predict group membership of the left out subject. You can determine prediction accuracy and you can also go back and see which connections carry the most predictive power (usually called beta weight).  The first paper that comes to mind is a Dosenbach 2010 Science paper, but many have been published with this approach, and it is increasing in popularity because of aforementioned advantages.

In order to reduce the dimensionality (and mutual information) of the data before running the algorithm, I typically run PCA on my connectivity matrices and use component scores to train the classifier. I can then easily convert results back to the native space if needed.

Dear Joshua Siegel,

I have a basic understanding of using machine learning techniques you mentioned, and  I also find them useful in many ways. Although currently I do not see how that would help to tackle the huge multiple comparison problem (maybe 1000*1000 non-independent observations) that we meet when analyzing connectomes. Having a million observations ("features") per patient will always lead to optimal classification, as that is the criteria of the training, and it is not clear how to control the level of significance. I am also not convinced that having smaller case numbers (typical clinical study: 30-40 vs controls) and lot of features is an optimal scenario to have non-linear classifiers for revealing relevant differences in the connectome. What do you think?

András

Andras, 

Thank you for your reply. I agree that small case numbers and huge number of features is not optimal no matter what approach you take, but there are a few ways to mine that data in an unbiased way to find meaningful relationships. Here is what I mean (in response to your concerns):

"Having a million observations ("features") per patient will always lead to optimal classification, as that is the criteria of the training"

I am not sure what you mean by optimal, but if you are doing Leave One Out (LOO) prediction, then it makes no difference how many samples you have, you should not identify spurious correlations by chance. That said, having a million observations, the vast majority of which are noise, will in many cases make it less likely that a machine learning algorthing (MLA) will seperate your groups with high accuracy.  This point is addressed below.

"I do not see how that would help to tackle the huge multiple comparison problem (maybe 1000*1000 non-independent observations)"

Doing principle component analysis on your correlation matrix (or independent component analysis on your timecourses) before hand enables you to greatly reduce the number of observation that you are giving to your MLA. Functional connectivity matrices are highly degenerate (http://www.pnas.org/content/96/6/3257.full). You may have a million values, but they are not independent samples and there is much mutual information. In other words, you can capture almost all of the information in that million connections using maybe 40 components. 

So the approach that I take to group differences is to run PCA on the FC matrices across all of my samples, and then plug the components in to a machine learning algorithm to determine separability and identify meaningful features (as mentioned above). If no real group difference exists, then no separability will be found.  I can't reference any of my own publications on this matter (give me a few months on that), but the Dosenbach 2010 paper was my jumping off point. Does that adequately address your concerns?

Josh

Dear Joshua, Thanks for the elaborated answer.