Variational inference for large-scale models of discrete choice Jon McAuliffe Statistics Department University of Pennsylvania Wharton School Philadelphia, PA 19104 Michael Braun MIT Sloan School of Management Massachusetts Institute of Technology Cambridge, MA 02142 6 January 2008 GLYPH<3> Abstract Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact inference is often intractable. Markov chain Monte Carlo techniques make approximate infer- ence possible, but the computational cost is prohibitive on the large data sets now becoming routinely available. Variational methods provide a deterministic alternative for approximation of the posterior distribution. We derive variational procedures for empirical Bayes and fully Bayesian inference in the mixed multinomial logit model of discrete choice. The algorithms require only that we solve a sequence of unconstrained optimization problems, which are shown to be convex. Extensive simulations demon- strate that variational methods achieve accuracy competitive with Markov chain Monte Carlo, at a small fraction of the computational cost. Thus, variational methods permit inferences on data sets that otherwise could not be analyzed without bias-inducing modifications to the underlying model. Discrete choice models have a long history in statistical analysis, appearing in applications as varied as the analysis of consumer choice data (Guadagni and Little 1983; Fader and Hardie 1996), transportation planning (Theil 1969; McFadden 1974; Ben-Akiva and Ler- man 1985), economic demand estimation (Train et al. 1987; Revelt and Train 1998), new product development (Moore et al. 1999), portfolio analysis (Uhler and Cragg 1971) and $^{GLYPH<3>}$The authors gratefully acknowledge research assistance from Sandy Spicer and Liz Theurer. 1 Introduction 1 arXiv:0712.2526v3 [stat.ME] 15 Jan 2008