Title: Tropical Density Estimation of Phylogenetic Trees

Abstract: In 2004, Speyer and Sturmfels showed that a space of phylogenetic trees with $m$ leaves is a tropical Grassmanian, which is a tropicalization of the set of all solutions for a system of certain linear equations under the max-plus arithmetic. In this research we apply the "tropical metric," a well-defined metric over the space of phylogenetic trees under the max-plus algebra, to non-parametric estimation of gene trees distribution over the tree space. Kernel density estimator (KDE) is one of the most popular non-parametric estimation of a distribution from a given sample and we mimic KDE using the tropical metric which measures the length of an intrinsic geodesic between trees over the tree space. We estimate the probability of an observed tree by empirical frequencies of nearby trees, with the level of influence determined by the tropical metric. Then, with simulated data generated from the multispecies coalescent model, we show that the non-parametric estimation of gene tree distribution using the tropical metric performs better than one using the Billera-Holmes-Vogtman (BHV) metric developed by Huggins et al.~and Weyenberg et al.~in terms of computational times and accuracy. We then apply it to Apicomplexa data.