Title: RG analysis and Magnetic instability in gapless superconductors

Abstract: We study the magnetic instability of gapless superconductors. The instability arises due to the infrared divergence associated with the gapless modes of the superconductor. When the Fermi-surface mismatch between pairing fermions is close to the gap, the gapless modes have a quadratic energy dispersion relation at low energy and open a secondary gap at the Fermi surface, which is only power-suppressed by the coupling. On the other hand, for a large mismatch, we find the gapless superconductor does not open a secondary gap, but instead makes transition to a new phase by forming the condensate of supercurrents. We calculate the condensate of supercurrents by minimizing the effective potential. In the new phase, the Meissner mass is positive for the magnetic fields orthogonal to the direction of the condensate but zero in the parallel direction.