Probability & Random Variables

Measure-Theoretic Foundations

Probability assigns a real number $P(E) \in [0, 1]$ to events in a sample space $\Omega$ satisfying Kolmogorov's axioms: $P(\Omega) = 1$ and countable additivity for mutually disjoint events $P(\bigcup E_i) = \sum P(E_i)$.

1. Conditional Probability & Multiplication Rule

P (A | B) = \frac{P (A \cap B)}{P (B)} (P (B) > 0)

Independent Events: $A$ and $B$ are independent iff $P (A \cap B) = P (A) \cdot P (B) ⟺ P (A | B) = P (A)$ .

2. Law of Total Probability & Bayes' Theorem

Bayes' Inverse Probability Theorem

Let $E_1, E_2, \dots, E_n$ form a partition of the sample space $\Omega$ ($E_i \cap E_j = \emptyset$ and $\bigcup E_i = \Omega$) with $P(E_i) > 0$. For any event $A$ with $P(A) > 0$: $$P(E_k | A) = \frac{P(E_k) \cdot P(A | E_k)}{\sum_{i=1}^n P(E_i) \cdot P(A | E_i)}$$

3. Random Variables, Expectation & Variance

For a discrete random variable $X$ with probability mass function $P (X = x_{i}) = p_{i}$ :

Expectation (Mean): $E [X] = μ = \sum x_{i} p_{i}$
Variance: $Var (X) = σ^{2} = E [(X - μ)^{2}] = E [X^{2}] - (E [X])^{2}$

Properties of Expectation and Variance

$E [a X + b] = a E [X] + b$
$Var (a X + b) = a^{2} Var (X)$
For independent random variables $X$ and $Y$ : $Var (X \pm Y) = Var (X) + Var (Y)$ .

4. Binomial Distribution $B (n, p)$

For $n$ independent Bernoulli trials with success probability $p$ and failure probability $q = 1 - p$ :

P (X = k) = (\binom{n}{k}) p^{k} q^{n - k}, k = 0, 1, 2, \dots, n

Mean: $E [X] = n p$
Variance: $Var (X) = n p q$
Standard Deviation: $σ = \sqrt{n p q}$

Probability & Random Variables ​

1. Conditional Probability & Multiplication Rule ​

2. Law of Total Probability & Bayes' Theorem ​

3. Random Variables, Expectation & Variance ​

Properties of Expectation and Variance ​

4. Binomial Distribution B(n,p) ​