Title: Combinatorial Information Theory: I. Philosophical Basis of Cross-Entropy and Entropy

Abstract: In this review and synthesis, the three main theoretical bases of the concepts of entropy and cross-entropy - information-theoretic, axiomatic and combinatorial - are critically examined. It is shown that the combinatorial basis, as promulgated by Boltzmann and Planck, is the most fundamental (most primitive) basis of these concepts. Not only does it provide (i) a derivation of the Kullback-Leibler cross-entropy and Shannon entropy functions, as simplified forms of the multinomial distribution subject to the Stirling approximation; the combinatorial approach also yields (ii) an explanation for the need to maximize entropy (or minimize cross-entropy) to find the most probable realization; and (iii) the means to derive entropy and cross-entropy functions for systems which do not satisfy the multinomial distribution, i.e. which fall outside the domain of the Kullback-Leibler and Shannon measures. The information-theoretic and axiomatic bases of cross-entropy and entropy - whilst of tremendous importance and utility - are therefore seen as secondary viewpoints, which lack the breadth of the combinatorial approach. Appreciation of this reasoning would permit development of a powerful body of "combinatorial information theory", as a tool for statistical inference in all fields (inside and outside science). For completeness, the essential features of Jaynes' analysis of entropy and cross-entropy - reinterpreted in light of the combinatorial approach - are outlined, including derivation of probability distributions, ensemble theory, Jaynes relations, a generalized free energy (or ``free information'') function, generalized Gibbs-Duhem relation, phase rule, fluctuation theory, Jaynes' entropy concentration theorem, and generalized definitions of entropy and cross-entropy.