Recreational Mathematics   
   home  |  library  |  contact
 Math Notes
 Math Programming [25]
 Regression [3]
 Data Mining [17]
 Notation [6]
 Linear Algebra [9]
 Stats & Prob [15]
 Math Cognition [5]
 Space & Physics [6]
 Formulas [5]
 Fun & Games [2]
 Haskell [1]
 Bayes Theory [1]
 Site News [0]
 Math Projects [5]
 Polynomials [1]
 Calculus [9]
 Number Theory [3]
 Optimization [2]
 Financial [1]

 Math Links
 Andrew Gelman
 Chance Wiki
 Daniel Lemire
 KD Knuggets
 Social Stats
 MySQL Performance
 Matthew Hurst
 Hal Daume III
 Math Notes >> Permanent Link

Bayes formula and it's extensions [Regression
Posted on June 1, 2007 @ 02:04:24 PM by Paul Meagher

Let wj represent a particular state of the world.

Let p(wj) represent the prior probability of observing a particular state of the world.

Let x represent a measurement value.

Let p(x|wj) represent the likelihood of a measurement value given a particular state of the world. We may now state Bayes formula as:

p(wj|x) = p(x|wj)p(wj)/p(x)

In words:

posterior = likelihood x prior / evidence

One of the first extensions we might want to make to this formula is to include more than one measurement value in our x term. We can let x (i.e., bolded x) denote a vector of measurement values. Part of the beauty of Bayes formula is how easy it is to extend the formula to handle such vector quantities:

p(wj|x) = p(x|wj)p(wj)/p(x)

When studying Bayes theory we often think we have learned it when we have mastered a particularly simple version of Bayes formula; namely, a version where:

  • There are only 2 states of the world.
  • There is only one measurement value.
  • There are only discrete valued terms.
  • The resulting action only involves deciding on what state of the world exists (i.e., an "epistemic action").
  • The only type of "loss" we are concerned with is the probability of error.

I hope this little exercise in generalizing Bayes formula to use a vector of measurements rather than just one will start you on the track to becoming a more sophisticated user of Bayes formula by exploring its various extensions.

The extension to using a vector of measurements is particularly useful in building classifiers where x represents a set of featural/dimensional measurements and wj represents a particular classification that is possible.


No comments entered ...


php/Math Project
© 2011. All rights reserved.