Title: Reliability and comparability of human brain structural covariance networks

Abstract: Structural covariance analysis is a widely used structural MRI analysis method which characterises the co-relations of morphology between brain regions over a group of subjects. To our knowledge, little has been investigated in terms of the comparability of results between different data sets or the reliability of results over the same subjects in different rescan sessions, image resolutions, or FreeSurfer versions.
In terms of comparability, our results show substantial differences in the structural covariance matrix between data sets of age- and sex-matched healthy human adults. These differences persist after site correction, they are exacerbated by low sample sizes, and they are most pronounced when using average cortical thickness as a morphological measure. Down-stream graph theoretic analyses further show statistically significant differences.
In terms of reliability, substantial differences were also found when comparing repeated scan sessions of the same subjects, and image resolutions and FreeSurfer versions of the same image. We could further estimate the relative measurement error and showed that it is largest when using thickness. With simulated data, we argue that cortical thickness is least reliable because of larger relative measurement errors.
Practically, we make the following recommendations (1) pooling subjects across sites into one group should be avoided, particularly if sites differ in image resolutions, demographics, or preprocessing; (2) surface area and volume should be preferred as morphological measures over cortical thickness; (3) a large number of subjects should be used to estimate structural covariance; (4) measurement error should be assessed where repeated measurements are available; (5) if combining sites is critical, univariate site-correction is insufficient, but error covariance should be explicitly measured and modelled.