{"action":"created","issue":{"url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1896","repository_url":"https://api.github.com/repos/pymc-devs/pymc3","labels_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1896/labels{/name}","comments_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1896/comments","events_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1896/events","html_url":"https://github.com/pymc-devs/pymc3/issues/1896","id":213993446,"number":1896,"title":"ADVI behaving in BVAR","user":{"login":"tomaskrehlik","id":1689944,"avatar_url":"https://avatars1.githubusercontent.com/u/1689944?v=3","gravatar_id":"","url":"https://api.github.com/users/tomaskrehlik","html_url":"https://github.com/tomaskrehlik","followers_url":"https://api.github.com/users/tomaskrehlik/followers","following_url":"https://api.github.com/users/tomaskrehlik/following{/other_user}","gists_url":"https://api.github.com/users/tomaskrehlik/gists{/gist_id}","starred_url":"https://api.github.com/users/tomaskrehlik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomaskrehlik/subscriptions","organizations_url":"https://api.github.com/users/tomaskrehlik/orgs","repos_url":"https://api.github.com/users/tomaskrehlik/repos","events_url":"https://api.github.com/users/tomaskrehlik/events{/privacy}","received_events_url":"https://api.github.com/users/tomaskrehlik/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2017-03-14T07:59:08Z","updated_at":"2017-03-15T12:29:17Z","closed_at":null,"body":"I am trying to get Bayesian VAR estimation work in PyMC3 and ADVI has been behaving weirdly. The data I use are [here](https://gist.github.com/tomaskrehlik/b94454a7df73eb7d1977976d5e3c05bf). So I use the following code that should be rather self-explanatory.\r\n\r\n````{python3}\r\n%matplotlib inline\r\nimport numpy as np\r\nimport pymc3 as pm\r\nfrom scipy import optimize\r\nfrom theano import tensor as tt\r\n\r\ndata = np.genfromtxt('data.csv', delimiter=',')\r\n\r\nk = 7\r\nwindow = 500\r\nlags = 2\r\n\r\ndef get_lags(data, lags):\r\n    return([data[(1 + lags - i):(window + lags - i),0:k] for i in range(0,lags+1)])\r\n\r\ndata_new = get_lags(data, lags)\r\n\r\n# Init values from OLS and its residuals\r\nX = np.concatenate((np.ones((499,1)),np.array(data_new[1]), np.array(data_new[2])), axis = 1)\r\nbet=np.linalg.inv(np.transpose(X)@X)@np.transpose(X)@data_new[0]\r\ninit_sigma = np.std(data_new[0] - X@bet, axis=0)\r\nA = np.corrcoef(data_new[0] - X@bet, rowvar=0)\r\ninit_corr = A[np.triu_indices_from(A,1)]\r\n\r\ntri_ind = np.zeros((k,k), dtype = np.int64)\r\ntri_ind[np.triu_indices(k, 1)] = range(0,(k*(k-1)//2))\r\ntri_ind[np.tril_indices(k, -1)] = range(0,(k*(k-1)//2))\r\nwith pm.Model() as model:\r\n    sigma = pm.Lognormal('sigma', np.zeros(k), np.ones(k), shape=k, testval=init_sigma)\r\n    C_triu = pm.LKJCorr('C_triu', n=60, p=k, testval=init_corr)\r\n    C = pm.Deterministic('C', tt.fill_diagonal(C_triu[tri_ind], 1.))\r\n    sigma_diag = pm.Deterministic('sigma_mat', tt.nlinalg.diag(sigma))\r\n    cov = pm.Deterministic('cov', pm.math.matrix_dot(sigma_diag, C, sigma_diag))\r\n    \r\n    alpha = pm.Normal('alpha', mu=0, sd=10, shape=k, testval=bet[0,:])\r\n    beta1 = pm.Normal('beta1', mu=0, sd=0.5, shape=(k,k), testval=bet[1:k+1])\r\n    beta2 = pm.Normal('beta2', mu=0, sd=0.5, shape=(k,k), testval=bet[1+k:2*k+1])   \r\n    mu = tt.transpose(alpha) + pm.math.matrix_dot(data_new[1],beta1) + pm.math.matrix_dot(data_new[2],beta2)\r\n    Y_obs = pm.MvNormal('Y_obs', mu=mu, cov=cov, observed=data_new[0])\r\n````\r\n\r\nNow, if I just sample by NUTS I get estimates that converged to reasonable values (the initial values are pretty close so I probably don't really need a starting procedure). But when I run ADVI, it gives reasonable values at the beginning and then breaks. It gets worse with more variables in the model. Any indication why this could be?"},"comment":{"url":"https://api.github.com/repos/pymc-devs/pymc3/issues/comments/286727158","html_url":"https://github.com/pymc-devs/pymc3/issues/1896#issuecomment-286727158","issue_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1896","id":286727158,"user":{"login":"aseyboldt","id":1882397,"avatar_url":"https://avatars0.githubusercontent.com/u/1882397?v=3","gravatar_id":"","url":"https://api.github.com/users/aseyboldt","html_url":"https://github.com/aseyboldt","followers_url":"https://api.github.com/users/aseyboldt/followers","following_url":"https://api.github.com/users/aseyboldt/following{/other_user}","gists_url":"https://api.github.com/users/aseyboldt/gists{/gist_id}","starred_url":"https://api.github.com/users/aseyboldt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aseyboldt/subscriptions","organizations_url":"https://api.github.com/users/aseyboldt/orgs","repos_url":"https://api.github.com/users/aseyboldt/repos","events_url":"https://api.github.com/users/aseyboldt/events{/privacy}","received_events_url":"https://api.github.com/users/aseyboldt/received_events","type":"User","site_admin":false},"created_at":"2017-03-15T12:29:17Z","updated_at":"2017-03-15T12:29:17Z","body":"Yes, but sadly it is not implemented yet. #1898 is a (very much unfinished) start."}}
{"actor":{"display_login":"aseyboldt"}}