Thursday, February 15, 2007

Laplace Prior and Sparsity

Sparse parameter estimation can be achieved in the Bayesian framework by using the Laplace (double exponential) prior distribution. It is known that the MAP estimates using the Laplace prior are the same as those produced by applying the lasso algorithm that minimizes the usual sum of squared errors, with a bound on the sum of the absolute values of the coefficients.

The Bayesian Logistic Regression (BBR) and Bayesian Mutinomial Regression (BMR) software packages have implemented this idea and showed good performance in text classification.

1 comment:

Ibrahim said...

Hi Dell, just passed by this link via a google search..

so are you familar with any work or software that deals with estimating sparse vectors with Binomial sparsifying priors? All I found in the literature was laplacian and gamma priors although my problem requires a non-monotone function.. (exponential functions won't work)