Competing ferromagnetic and antiferromagnetic phases on the frustrated Ising honeycomb lattice

We investigate the frustrated J₁–J₂–J₃ Ising model on the honeycomb lattice, featuring first- and second-neighbor ferromagnetic couplings (J₁>0 and J₂>0) and third-neighbor antiferromagnetic interactions (J₃<0). Using the cluster mean-field method, we analyze the phase transitions in the regime 1/2<J₂/J₁≤1, where ferromagnetic and antiferromagnetic phases compete. Our results reveal that near the strongly frustrated limit J₃/J₁=−1, the system exhibits order-by-disorder state selection, tricritical and bicritical behavior, critical endpoints, and two successive phase transitions. The ferromagnetic–paramagnetic transition remains second order across the entire interaction range, whereas the antiferromagnetic–paramagnetic boundary shows a richer behavior, including both first- and second-order transitions as well as tricriticality. Increasing the second-neighbor coupling J₂/J₁ narrows the range of J₃/J₁ where first-order antiferromagnetic–paramagnetic transitions occur; beyond a certain threshold, only second-order order–disorder transitions persist. Consequently, the tricritical point shifts toward J₃/J₁≈−1 as J₂/J₁ increases, culminating in a bicritical point where the antiferromagnetic, ferromagnetic, and paramagnetic phases meet.