Parameter Estimation with Dense and Convolutional Neural Networks
  Applied to the FitzHugh-Nagumo ODE
  Machine learning algorithms have been successfully used to approximate
nonlinear maps under weak assumptions on the structure and properties of the
maps. We present deep neural networks using dense and convolutional layers to
solve an inverse problem, where we seek to estimate parameters of a
FitzHugh-Nagumo model, which consists of a nonlinear system of ordinary
differential equations (ODEs). We employ the neural networks to approximate
reconstruction maps for model parameter estimation from observational data,
where the data comes from the solution of the ODE and takes the form of a time
series representing dynamically spiking membrane potential of a biological
neuron. We target this dynamical model because of the computational challenges
it poses in an inference setting, namely, having a highly nonlinear and
nonconvex data misfit term and permitting only weakly informative priors on
parameters. These challenges cause traditional optimization to fail and
alternative algorithms to exhibit large computational costs. We quantify the
prediction errors of model parameters obtained from the neural networks and
investigate the effects of network architectures with and without the presence
of noise in observational data. We generalize our framework for neural
network-based reconstruction maps to simultaneously estimate ODE parameters and
parameters of autocorrelated observational noise. Our results demonstrate that
deep neural networks have the potential to estimate parameters in dynamical
models and stochastic processes, and they are capable of predicting parameters
accurately for the FitzHugh-Nagumo model.
