Title: On the Capacity of Face Representation

Abstract: Face recognition is a widely used technology with numerous large-scale applications, such as surveillance, social media and law enforcement. There has been tremendous progress in face recognition accuracy over the past few decades, much of which can be attributed to deep learning based approaches during the last five years. Indeed, automated face recognition systems are now believed to surpass human performance in some scenarios. Despite this progress, a crucial question still remains unanswered: given a face representation, how many identities can it resolve? In other words, what is the capacity of the face representation? A scientific basis for estimating the capacity of a given face representation will not only benefit the evaluation and comparison of different face representation methods, but will also establish an upper bound on the scalability of an automatic face recognition system. We cast the face capacity estimation problem under the information theoretic framework of capacity of a Gaussian noise channel. By explicitly accounting for two sources of representational noise: epistemic (model) uncertainty and aleatoric (data) variability, our approach is able to estimate the capacity of any given face representation. To demonstrate the efficacy of our approach, we estimate the capacity of a 128-dimensional state-of-the-art deep neural network based face representation, FaceNet. Our numerical experiments indicate that, (a) our capacity estimation model yields a capacity upper bound of $1\times10^{12}$ for the FaceNet representation at a false acceptance rate (FAR) of 5%, (b) the capacity reduces drastically as you lower the desired FAR with an estimate of $2\times10^{7}$ and $6\times10^{3}$ at FAR of 0.1% and 0.001%, respectively), and (c) the performance of the FaceNet representation is significantly below the theoretical limit.