Loss Functions and Prior Distributions

Andrew Gelman has an article that explains the relationship between loss functions and prior distributions. In a word, using L2 or L1 norm as regularisers for regresion is essentially equivalent to choosing Gaussian or Laplace priors for the parameters, i.e., L2 or L1 corresponds to the MAP of Gaussian or Laplace.

Using Python for Machine Learning

Python plus a C/C++ library seems a good combination for agile development in machine learning. Please refer to John Langford's post on Programming Languages for Machine Learning Implementations. There is a list of open source machin learning tools that support Python scripting. Orange is probably the most comprehensive one among them, e.g., it also contains data mining algorithms such as association rules.

Python defaultdict

I recently learned the trick of using the defaultdict class for frequency counting and smoothing from Peter Norvig's influential technical article How to Write a Spelling Corrector.

As its name suggests, defaultdict is like a regular Python dict except that a default value (factory in fact) can be specified in advance. For example, the following piece of code uses the stadard dict to build a term-frequency hash table.

tf = {}
for t in words:
    tf[t] = tf.get(t, 0) + 1

It can be simpler and faster by making use of the defaultdict from the collections module.

import collections
tf = collections.defaultdict{int}
for t in words:
    tf[t] += 1

PS: I am glad to see that my colleague Rogger Mitton's work on spell-checking is cited by the above article.

The Strength of Weak Ties, in Online Advertising

The Strength of Weak Ties was studied by sociologists 35 years ago and re-discovered by physicists about 10 years ago in the research of small-world networks. This theory actually provides a good explanation of the phenomenon noticed by Anand Rajaraman recently: Affinity and Herding Determine the Effectiveness of Social Media Advertising.

To be a leader

Paper number does not really make a lot of sense. To be a leader, you should have a clear focus and have real impact. For example, the only reason why one receives an academic award is that he contributes a lot to a specific research direction but not that he has published hundreds of papers.

The above words are said to be from Bruce Croft (a leading computer scientist in the field of information retrieval), though I could not find his original text.