ProbabilityDiscussion

October 7, 2010

Statistical Inference (Casella and Berger)

http://en.wikipedia.org/wiki/Set_%28mathematics%29

http://en.wikipedia.org/wiki/Probability_interpretations

Set Theory:

Union - combination of two sets

Complement - everything that's not in A

Empty - no elements

Definitions:

experiment - any activity generating observable results

outcome - result of experiment (IMPORTANT TO KEEP STRAIGHT! don't confuse events and outcomes)

trial - single performance of experiment

sample space - set of all possible outcomes

countable/uncountable: - countable = one-to-one correspondence (e.g. 1/n) - uncountable = no one-to-one correspondence can be made -- infinite loop: you can do an infinite loop, but still count it -- flipping a coin: countable; temperature: uncountable

event - any subset of the sample space

Example:

experiment - roll a dice outcome - 1, or 2, or 3, or 4, or 5, or 6 trial - one roll of the dice COUNTABLE sample space - {1,2,3,4,5,6} event - may be {1}, or {1,2,3}, etc...

Operators:

Union, empty set, complement, intersection

Commutative:

Associative:

Distributive:

DeMorgan's Law:

Call a set abnormal if it can be put into itself (otherwise it's normal)

Example: the set of all squares is not itself square, so it is not a member of the set of squares The complimentary set, containing all non-squares, is itself not a square, so is normal

Consider the set of all normal sets Is it normal or abnormal? If it were normal, it would be contained in itself, and would therefore be abnormal If it were abnormal, it would not be contained in itself, and would therefore be normal

You can resolve this using more rigorous set theory...

More Definitions:

Disjoint ("set" term) / mutually exclusive ("probability" term) - if the intersection of two sets is the null set, they are mutually exclusive

Partition - take a group of sets; if the union of these sets is the sample sapce, and they are mutually exclusive, this is a partition

---

Distinction between probability theory that has a physical meaning (and is therefore "contaminated" by intuition) and a more abstract probability theory that doesn't have a corresponding physical meaning

Axiomatic probability theory (Komolgorov)

A probability is a function that follows 3 axioms:

Sample space $ S $

$ \sigma $-algebra $ \mathfrak{B} $ (means the set is fully consistent)

Function P -> probability over the domain $ \mathfrak{B} $

1. $ P(A) \geq 0 $ for all $ A \in \mathfrak{B} $

2. $ P(S) = 1 $

3. If $ A \in \mathfrak{B} $ and $ B \in \mathfrak{B} $ are disjoint, then $ P(A \bigcup B) = P(A) + P(B) $

In other words, $ P( \bigcup_{i=1}^{\infty} = \sum_{l=1}^{\infty} P(A_{i}) $

This is a mathematician's viewpoint: a clean definition, as long as we follow these rules, the function is a probability.

What is the probability of the null set?

Create a partition: $ S = {S, \emptyset} $

The probability of the sample space is $ P(S) = 1 $

So $ P(\emptyset) = 1 - P(S) = 0 $

$ P(A) = 1 $

$ P(A^c) = 1-P(A) $

If $ A \subset B $ then $ P(A) \leq P(B) $

The size of the set is directly related to the probability...

Another way to do this is using measure theory (another route, besides rigorous set theory, that leads to probability theory)

[wikipedia:Measure theory]

[wikipedia:Sigma-algebra]

Bonferroni's inequality: $ P(A \bigcap B) \geq P(A) + P(B) - 1 $