Proof of Chebyshev’s Inequality

Chebyshev’s Inequality is an important tool in probability theory. And it is a theoretical basis to prove the weak law of large numbers.

The theorem is named after Pafnuty Chebyshev, who is one of the greatest mathematician of Russia.

Пафну́тий Льво́вич Чебышёв

Пафну́тий Льво́вич Чебышёв

It is described as follows:

For a random variable \displaystyle X, has mathematical expectation \displaystyle \mu=E(X), and variance \displaystyle \sigma^2=V(X),

\displaystyle P\left(\mid X-\mu \mid \geqslant k\sigma \right) \leqslant \frac{1}{k^2}

The fabulous thing is that, Chebyshev’s Inequality works only by knowing the mathematical expectation and variance, whatever the distribution is(no matter the distribution is discrete or continuous).

Here is the proof of Chebyshev’s Inequality.

1. In the case of a discrete random variable \displaystyle X, the probability density function is \displaystyle f(x),

\displaystyle \sigma^2 = \sum\limits_{i=1} (x_i-\mu)^2f(x_i) \hspace{8mm} \textcircled 1

For those \displaystyle x_j in the domain of \displaystyle \mid X-\mu \mid \geqslant k\sigma, \displaystyle (x_j-\mu)^2 \geqslant k^2\sigma^2. So,

\displaystyle \textcircled 1 \geqslant \sum\limits_{j, \mid X-\mu \mid \geqslant k\sigma} (x_j-\mu)^2f(x_j) \\\\ \geqslant \sum\limits_{j, \mid X-\mu \mid \geqslant k\sigma} k^2\sigma^2f(x_j) \\\\ = k^2\sigma^2\sum\limits_{j, \mid X-\mu \mid \geqslant k\sigma} f(x_j) \\\\ = k^2\sigma^2 P \left(\mid X-\mu \mid \geqslant k\sigma \right)

Then we can get the inequality \displaystyle P\left(\mid X-\mu \mid \geqslant k\sigma \right) \leqslant \frac{1}{k^2}.

2. In the case of a continuous random variable \displaystyle X,

\displaystyle \sigma^2 = \int_{-\infty}^{\infty} (x-\mu)^2f(x)dx \\\\ = \int_{\mid X-\mu \mid \geqslant k\sigma} (x-\mu)^2f(x)dx + \int_{\mid X-\mu \mid < k\sigma} (x-\mu)^2f(x)dx \\\\ \geqslant \int_{\mid X-\mu \mid \geqslant k\sigma} (x-\mu)^2f(x)dx \hspace{8mm} \textcircled 2

Just like discrete distribution discussed, for those \displaystyle x in the domain of \displaystyle \mid X-\mu \mid \geqslant k\sigma, \displaystyle (x-\mu)^2 \geqslant k^2\sigma^2. So,

\displaystyle \textcircled 2 \geqslant \int_{\mid X-\mu \mid \geqslant k\sigma} k^2\sigma^2f(x)dx \\\\ = k^2\sigma^2 \int_{\mid X-\mu \mid \geqslant k\sigma} f(x)dx \\\\ = k^2\sigma^2 P \left(\mid X-\mu \mid \geqslant k\sigma \right)

And we can get the same inequality \displaystyle P\left(\mid X-\mu \mid \geqslant k\sigma \right) \leqslant \frac{1}{k^2}.

2 thoughts on “Proof of Chebyshev’s Inequality”

  1. Is there something approximate (or analogous or similar) to this result for distributions with no known mean or variance, say the Cauchy distribution? It is very handy indeed, but for distribution’s like the Cauchy, you can forget the law of large numbers and various other laws.

    Reply

    • Hi, Arbias, thank you for your nice comment. Now I see the Cauchy Distribution does not adapt to usual distribution which has measurable mean and variance. And I read your article of “Cauchys idea of probability”. It is very intelligible and interesting.

      Reply

Leave a comment