Baye's Theorem

Bayesian logic

Named for Thomas Bayes, an English clergyman and mathematician, Bayesian logic is a branch of logic applied to decision making and inferential statistics that deals with probability inference: using the knowledge of prior events to predict future events. Bayes first proposed his theorem in his 1763 work (published two years after his death in 1761), An Essay Towards Solving a Problem in the Doctrine of Chances . Bayes theorem provided, for the first time, a mathematical method that could be used to calculate, given occurrences in prior trials, the likelihood of a target occurrence in future trials. According to Bayesian logic, the only way to quantify a situation with an uncertain outcome is through determining its probability.

Bayes Theorem is a means of quantifying uncertainty. Based on probability theory, the theorem defines a rule for refining an hypothesis by factoring in additional evidence and background information, and leads to a number representing the degree of probability that the hypothesis is true. To demonstrate an application of Bayes Theorem, suppose that we have a covered basket that contains three balls, each of which may be green or red. In a blind test, we reach in and pull out a red ball. We return the ball to the basket and try again, again pulling out a red ball. Once more, we return the ball to the basket and pull a ball out red again. We form a hypothesis that all the balls are all, in fact, red. Bayes Theorem can be used to calculate the probability (p) that all the balls are red (an event labeled as A) given (symbolized as /) that all the selections have been red (an event labeled as B):

p(A|B) = p{A + B}/p{B}

Of all the possible combinations (RRR, RRG, RGG, GGG), the chance that all the balls are red is 1/4; in 1/8 of all possible outcomes, all the balls are red AND all the selections are red. Bayes Theorem calculates the probability that all the balls in the basket are red, given that all the selections have been red as .5 (probabilities are expressed as numbers between 0. and 1., with 1. indicating 100% probability and 0. indicating zero probability).

The International Society for Bayesian Analysis (ISBA) was founded in 1992 with the purpose of promoting the application of Bayesian methods to problems in diverse industries and government, as well as throughout the Sciences. The modern incarnation of Bayesian logic has evolved beyond Bayes initial theorem, developed further by the 18th century French theorist Pierre-Simon de Laplace, and 20th and 21st century practitioners such as Edwin Jaynes, Larry Bretthorst, and Tom Loredo. Current and possible applications of Bayesian logic include an almost infinite range of research areas, including genetics, astrophysics, psychology, sociology, artificial intelligence ( AI ), data mining , and computer programming.

Development of Bayes Theorem

Terminology:

P(A): Probability of occurrence of event A (marginal)

P(B): Probability of occurrence of event B (marginal)

P(A,B): Probability of simultaneous occurrence of events A and B (joint)

P(A/B): Probability of occurrence of A given that B has occurred (conditional)

P(B/A): Probability of occurrence of B given that A has occurred (conditional)

Relationship of joint probability to conditional and marginal probabilities:

P(A,B) = P(A B)P(B) or P(A, B) = P(B A)P(A)

So . . .

P(A/B)P(B) = P(B/A)P(A)

Rearranging gives simplest statement of Bayes theorem: P(B/A)=P(A/B)P(B) / P(A)