September 20, 2016

Random Variable

  • A random variable is often denoted \(X\)
  • We don't know its value a priori, but we can characterize its behavior via its distribution
    • Describes relative frequency with which values occur
    • AUC = 1 = 100%


Further Characterizations

Its mean \(\mu\) and standard deviation \(\sigma\).

Sample Mean

  • We need to distinguish the true mean \(\mu\)
  • From the sample mean based on a sample \(X_1, \ldots, X_n\): \(\overline{X}_n = \frac{1}{n}\sum_{i=1}^{n}X_i\)
  • Moral of the Story: \(\overline{X}_n\) estimates \(\mu\)

Weak Law of Large Numbers

Convergence in Probability

For a given \(\epsilon\) and \(n\)

  • Let \(X\) have \(\mu=0\)
  • Based on a sample \(X_1, \ldots, X_n\) of size \(n\), compute \(\overline{X}_n\)
  • Do this a bunch of times to see how \(\overline{X}_n\) varies from sample to sample

Convergence in Probability

Recall \(\mu=0\). For \(\epsilon=0.1\)

Two Results

Let \(X\) be a RV with mean \(\mu\) and \(\sigma<\infty\). Then

  1. The standard deviation of \(\overline{X}_n = \sqrt{\frac{\sigma^2}{n}}\)
  2. Chebyshev's Inequality:
  • Example: Let \(k=2\), \(\mathbb{P}\left(\left|X-\mu\right| \geq 2\sigma\right) \leq \frac{1}{4}\)
  • For ANY RV, the prob that it lies outside 2 standard deviations of its mean is bounded by 25%.

Chebyshev's Inequality

Normal RV with \(\mu=0\) and \(\sigma=1\)

Chebyshev's Inequality

Gamma RV with \(\mu=\) 1.25 and \(\sigma=\) 0.559

Proof of WLLN