Tuesday, August 05, 2008

Loss Functions and Prior Distributions

Andrew Gelman has an article that explains the relationship between loss functions and prior distributions. In a word, using L2 or L1 norm as regularisers for regresion is essentially equivalent to choosing Gaussian or Laplace priors for the parameters, i.e., L2 or L1 corresponds to the MAP of Gaussian or Laplace.

1 comment:

Anonymous said...

That post was by Aleks Jakulin, by the way, not Andrew. They did work together on a paper, which Andrew blogged about.

Your comments remind me of my own blog post on logistic regression by any other name.