Posted on June 1, 2007 @ 02:04:24 PM by Paul Meagher
Let w_{j} represent a particular state of the world.
Let p(w_{j}) represent the prior probability of observing a particular state of the world.
Let x represent a measurement value.
Let p(xw_{j}) represent the likelihood of a measurement value given a particular state of the world.
We may now state Bayes formula as:
p(w_{j}x) = p(xw_{j})p(w_{j})/p(x)
In words:
posterior = likelihood x prior / evidence
One of the first extensions we might want to make to this formula is to include more than one measurement value in our x term. We can let x (i.e., bolded x) denote a vector of measurement values. Part of the beauty of Bayes formula is how easy it is to extend the formula to handle such vector quantities:
p(w_{j}x) = p(xw_{j})p(w_{j})/p(x)
When studying Bayes theory we often think we have learned it when we have mastered a particularly simple version of Bayes formula; namely, a version where:
 There are only 2 states of the world.
 There is only one measurement value.
 There are only discrete valued terms.
 The resulting action only involves deciding on what state of the world exists (i.e., an "epistemic action").
 The only type of "loss" we are concerned with is the probability of error.
I hope this little exercise in generalizing Bayes formula to use a vector of measurements rather than just one will start you on the track to becoming a more sophisticated user of Bayes formula by exploring its various extensions.
The extension to using a vector of measurements is particularly useful in building classifiers where x represents a set of featural/dimensional measurements and w_{j} represents a particular classification that is possible.
