

$AIC =2k -2ln(L)$

$k$ = number of parameters in the model

$L$ = likelihood


Edit: Here is an example to think and discuss:


           method="qr", data=Prostate,se.fit = TRUE, x=TRUE, y=TRUE)
    out <- p$results.all
      xlab= expression(paste(lambda))


enter image description here



  • 2
    $\begingroup$ Great practical interpretation, and also makes sense in the Bayesian context... "theoretic"-based likelihood ratio vs. "atheoretic" prediction error. $\endgroup$ – shadowtalker Jul 20 '14 at 16:26
  • 3
    $\begingroup$ It would probably help to elaborate on how "effective degrees of freedom" for a regularized solution can be computed and used in AIC. $\endgroup$ – Brian Borchers Jul 20 '14 at 16:33
  • 2
    $\begingroup$ See the code in the R rms package effective.df function and my book Regression Modeling Strategies. The main idea, from Robert Gray, is that you consider the covariance matrix without penalization vs. the covariance matrix with penalization. The sum of the diagonal of a kind of ratio of these two gives you the effective d.f. $\endgroup$ – Frank Harrell Jul 20 '14 at 19:44
  • $\begingroup$ @FrankHarrell: So if I understand you correctly - it is ok to compute a bunch of models in glmnet (each with a different lambda parameter) and compute the AIC for each model, and then choose the lambda corresponding to the model with the lowest AIC? This is basically another way for choosing the lambda parameter, other than using Cross Validation. Am I right? $\endgroup$ – Corel Jan 16 '18 at 20:52
  • 1
    $\begingroup$ I was writing in the context of the rms package where a couple of the fitting functions when used with effective.df compute the effective number of parameters so you can get an effective AIC. This will approximate what you get from cross-validation with CV'ing. See this $\endgroup$ – Frank Harrell Jan 16 '18 at 22:24









Your Answer

