Title: Collective excitations in trapped boson-fermion mixtures: from demixing to collapse

Abstract: We calculate the spectrum of low-lying collective excitations in a gaseous cloud formed by a Bose-Einstein condensate and a spin-polarized Fermi gas over a range of the boson-fermion coupling strength extending from strongly repulsive to strongly attractive. Increasing boson-fermion repulsions drive the system towards spatial separation of its components (``demixing''), whereas boson-fermion attractions drive it towards implosion (``collapse''). The dynamics of the system is treated in the experimentally relevant collisionless regime by means of a Random-Phase approximation and the behavior of a mesoscopic cloud under isotropic harmonic confinement is contrasted with that of a macroscopic mixture at given average particle densities. In the latter case the locations of both the demixing and the collapse phase transitions are sharply defined by the same stability condition, which is determined by the softening of an eigenmode of either fermionic or bosonic origin. In contrast, the transitions to either demixing or collapse in a mesoscopic cloud at fixed confinement and particle numbers are spread out over a range of boson-fermion coupling strength, and some initial decrease of the frequencies of a set of collective modes is followed by hardening as evidenced by blue shifts of most eigenmodes. The spectral hardening can serve as a signal of the impending transition and is most evident when the number of bosons in the cloud is relatively large. We propose physical interpretations for these dynamical behaviors with the help of suitably defined partial compressibilities for the gaseous cloud under confinement.