Title: Diversity metric for evaluation of quantum annealing

Abstract: Solving discrete NP-hard problems is an important part of scientific discoveries and operations research as well as many commercial applications. A commonly used metric to compare meta-heuristic solvers is the time required to obtain an optimal solution, known as time to solution. However, for some applications it is desirable to have a set of high-quality and diverse solutions, instead of a single optimal one. For these applications, time to solution may not be informative of the performance of a solver, and another metric would be necessary. In particular, it is not known how well quantum solvers sample the configuration space in comparison to their classical counterparts. Here, we apply a recently introduced collective distance measure in solution space to quantify diversity by Mohseni et. al. and, based on that, we employ time-to-diversity as a metric for evaluation of meta-heuristics solvers. We use this measure to compare the performance of the D-Wave quantum annealing processor with a few classical heuristic solvers on a set of synthetic problems and show that D-Wave quantum annealing processor is indeed a competitive heuristic, and on many instances outperforms state-of-the-art classical solvers, while it remains on par on other instances. This suggests that a portfolio solver that combines quantum and classical solutions may win over all solvers.