Chebyshev's Inequality

Chebyshev's Inequality 

There is a well-known inequality due to the Russian mathematician Chebyshev which will play an important role in our subsequent work. In addition , it will give us a means of understanding precisely how the variance measures variability about the,expected value of a random variable.
         
   If we know the probability distribution of a random variable X (either the pdf in the continuous case or the point probabilities in the discrete case ), we may then compute E(X) and V(X), if these exist. However, the converse is not true. That is, from a knowledge of E(X) and V(X) we cannot reconstruct the probability distribution of X and hence cannot compute quantities such as P[ |X - E(X)| ≤  C].
Nonetheless, it turns out that although we cannot evaluate such probabilities [from a knowledge of E(X) and V(X)), we can give a very useful upper (or lower) bound to such probabilities. This result is contained in what is known as Chebyshev's inequality. Chebyshev's inequality. Let X be a random variable with E(X) = µ and let c be any real number. Then, if E(X - c)²

is finite and E is any positive number, we have

P[|X-c| ≥ ∈] ≤ 1/∈² E(X - c)²

The following forms, equivalent to  are immediate:
(a) By considering the complementary event we obtain

≥ 1-1/∈² E(X - c)²

(b) Choosing c = µ we obtain

P[|X-μ| ≥ ∈] ≤  Var X/∈²

(c) Choosing c = µand ∈ = kσ, where σ² = Var X > 0, we obtain

P[|X-c| < kσ]≤ k^-2

This last is particularly indicative of how the variance measures the "degree of concentration" of probability near E(X) = µ.

Proof 

(since the others follow as indicated. We shall deal only with the continuous case. In the discrete case the argument is very similar with integrals replaced by sums. However, some care must be taken with endpoints of intervals.):
 Consider

P[|X-μ| ≥ ∈] = ∫_{x:[|X-μ| ≥ ∈]} f(x)dx

(The limit on the integral says that we are integrating between -∞ and c-∈ and between c+∈ and +∞)
        Now |x - c| ≥ ∈ is equivalent to (X - c)²/∈² ≥ 1. Hence the above integral is
where       R = {x:|x-c|≥ ∈}

This integral is, in turn,

\[\le \int_{-\infty }^{+\infty }{\frac{{{\left( x-c \right)}^{2}}}{{{\in }^{2}}}}f\left( x \right)dx\]

=1/∈² (X - c)²
as was to be shown.

Notes:
(a) It is important to realize that the above result is remarkable precisely because so little is assumed about the probabilistic behavior of the random variable X.
(b) As we might suspect, additional information about the distribution of the random variable X will enable us to improve on the inequality derived. For example, if C = ! we have, from Chebyshev's inequality,

P[|X-μ| ≥ (3/2) σ]≤4/9=0.44

Suppose that we also know that X is uniformly distributed over (1 - 1/√3, 1 + 1/√3). Hence E(X) = 1, V(X) = 1/9 and thus

P[|X-μ| ≥ (3/2) σ] = P[|X-1| ≥ (1/2) ] .= 1 - P[|X-1| < (1/2) ]
                                    =1 - P[1/2√3/2 = 0.134

Observe that although the statement obtained from Chebyshev's inequality is consistent with this result, the latter is a more precise statement. However, in many problems no assumption concerning the specific distribution of the random variable is justified, and in such cases Chebyshev's inequality can give us important information about the behavior of the random variable.
As we note  if V(X) is small, most of the probability distribution of X is "concentrated" near E(X). This may be expressed more precisely in the following theorem.

Theorem 1. Suppose that V(X) = 0. Then P[X = µ] (Informally, X = µ,with "probability l.")  where µ = ∈(X).

Proof·
we find that
P[|X-μ| ≥ ∈] = 0                  for any ∈ > 0.
Hence
P[|X-μ| < ∈] = 1                 for any ∈ > 0.


Since ∈ may be chosen arbitrarily small, the theorem is established.

Notes:
(a) This theorem shows that zero variance does imply that all the probability is concentrated at a single point, namely at E(X).

 (b) If E(X) = 0, then V(X) = E(X²), and hence in this case, E(X²) = 0 implies the same conclusion

(c) It is in the above sense that we say that a random variable X is degenerate: It assumes only one value with probability 1.