Atomic norm-based DOA estimation with sum and difference co-arrays in coexistence of circular and non-circular signals

Sparse arrays can increase the array aperture and degrees of freedom through the construction of either sum or difference co-arrays or both. In order to exploit the advantages of sparse arrays while estimating directions of arrival (DOAs) of a mixture of circular and non-circular signals, in this paper, a gridless DOA estimation method is proposed by employing a recently introduced enhanced nested array, whose virtual arrays have no holes. The virtual signals derived from both sum and difference co-arrays are constructed based on atomic norm minimization. It is shown that the proposed method also works when the circular and non-circular signals come from the same set of directions. Simulation results are provided to demonstrate the performance of the proposed method.