Recently published in Physical Review B:
The magnetic honeycomb lattice with competing interactions can give rise to unconventional quantum phases. We present observations of highly frustrated quasi two-dimensional (2D) magnetic correlations in the honeycomb lattice layers of the Seff = 1/2 compound BaCo2(PO4)2. Specific heat lacks a sharp anomaly down to 2K although all spin entropy is recovered, indicating the absence of a transition to long range order in this material. However, ac susceptibility shows that spins do not freeze into a spin-glass like state. The spin correlations measured by neutron powder diffraction reveal short range quasi-2D order with two separate, incompatible, short range magnetic orders onset. We compare inelastic neutron scattering data from this state with linear spin wave theory, and two separate parameter regions of the XXZ J1-J2-J3 model with ferromagnetic nearest-neighbor exchange J1 are favored, both near regions of high classical degeneracy. These data show that BaCo2(PO4)2 is a rare highly frustrated, quasi-2D Seff = 1/2 honeycomb lattice material which resists long range magnetic order and spin freezing, and the nature of its quantum ground state is not yet understood.