References & Citations
Statistics > Methodology
Title: Consilience: A Holistic Measure of Goodness-of-Fit
(Submitted on 23 Oct 2017 (v1), last revised 20 Oct 2018 (this version, v4))
Abstract: We describe an apparently new measure of multivariate goodness-of-fit between sets of quantitative results from a model (simulation, analytical, or multiple regression), paired with those observed under corresponding conditions from the system being modeled. Our approach returns a single, integrative measure, even though it can accommodate complex systems that produce responses of M types. For each response-type, the goodness-of-fit measure, which we label "Consilience" (C), is maximally 1, for perfect fit; near 0 for the large-sample case (number of pairs, N, more than about 25) in which the modeled series is a random sample from a quasi-normal distribution with the same mean and variance as that of the observed series (null model); and, less than 0, toward minus-infinity, for progressively worse fit. In addition, lack-of-fit for each response-type can be apportioned between systematic and non-systematic (unexplained) components of error. Finally, for statistical assessment of models relative to the equivalent null model, we offer provisional estimates of critical C vs. N, and of critical joint-C vs. N and M, at various levels of Pr(type-I error). Application of our proposed methodology requires only MS Excel (2003 or later); we provide Excel XLS and XLSX templates that afford semi-automatic computation for systems involving up to M = 5 response types, each represented by up to N = 1000 observed-and-modeled result pairs. N need not be equal, nor response pairs in complete overlap, over M.
Submission history
From: William Neill [view email][v1] Mon, 23 Oct 2017 01:21:40 GMT (513kb)
[v2] Thu, 2 Nov 2017 15:56:59 GMT (579kb)
[v3] Wed, 11 Apr 2018 02:36:28 GMT (581kb)
[v4] Sat, 20 Oct 2018 21:42:52 GMT (578kb)
Link back to: arXiv, form interface, contact.