On the Role of Entropy-based Loss for Learning Causal Structures with
  Continuous Optimization
  Causal discovery from observational data is an important but challenging task
in many scientific fields. Recently, NOTEARS [Zheng et al., 2018] formulates
the causal structure learning problem as a continuous optimization problem
using least-square loss with an acyclicity constraint. Though the least-square
loss function is well justified under the standard Gaussian noise assumption,
it is limited if the assumption does not hold. In this work, we theoretically
show that the violation of the Gaussian noise assumption will hinder the causal
direction identification, making the causal orientation fully determined by the
causal strength as well as the variances of noises in the linear case and the
noises of strong non-Gaussianity in the nonlinear case. Consequently, we
propose a more general entropy-based loss that is theoretically consistent with
the likelihood score under any noise distribution. We run extensive empirical
evaluations on both synthetic data and real-world data to validate the
effectiveness of the proposed method and show that our method achieves the best
in Structure Hamming Distance, False Discovery Rate, and True Positive Rate
matrices.
