Conditional Expectations

Conditional Expectations 

In the following chapters we shall have occasion to find the expected value of random variables in conditional distributions, or the expected value of one random variable given the value of another.

Definition 1 

Conditional expectation Let (X, Y) be a two-dimensional random variable and g( . , . ), a function of two variables. The conditional expectation of g(X, Y) given X = x, denoted by δ[g(X, Y)I X = x], is defined to be
                    δ[g(X, Y)|X = x] = ∫_{lim -∞→∞}g(x, Y)f_y|x (y|x)dy               (1)
if (X, Y) are jointly continuous, and

                 δ[g(X, Y)| X = x] =   ∑g(x, y_j)f_y|x(y_j|x)                              (2)

if (X, Y) are jointly discrete, where the summation is over all. possible value of y
In particular, if g(x, y)=y, we have defined δ[Y|X=x]=δ[Y|x]. δ[Y|x] and δ[g(X, Y)|x] are functions of x. Note that this definition can be generalized to more than two dimensions. For example, let (X_l' ... , X_k , Y_l , ... , Y_m) be a (k + m)-dimensional continuous random variable with density
f _{Xl, ... , Xk , YI, ... , Ym}(X_l' . " , X_k' Y_l' ., ., Y_m); then

                        δ[g(X_l ... , X_k , Y_l ... , Y_m)|X_l, ... , X_k] .
                       = ∫_{lim -∞→∞} ............∫_{lim -∞→∞}g(x₁ ... , X_k' Y_l' ... , Y_m) *f _{(Yl, ... , Ym|X1..........Xk) dyidym}

EXAMPLE 1

In the experiment of tossing two tetrahedra with X, the number on the first, and Y, the larger of the two numbers, we found that 

                                     1/2          for y=2
f_ν|x (y|x)         =            1/4         for y=3
                                     1/4         for y=4

Hence δ[Y| X = 2]=∑yf_y|x(y|X=2)=2*1/2 + 3*1/4 + 4*1/4 = 11/4

EXAMPLE 2 

For f_{x , r}(x, y) = (x + y)I_{(0, 1)}(x)I_{(0, 1)}(y), we found that

f_y|x (y|x)(x+y)/(x+1/2)I_{(0, 1)}(y)                                                                    for 0 < x < I  

Hence

δ[ Y|X = x] = ∫_lim 0→1  y*(x+y)/(x+1/2)d Y = 1/(x+1/2) (x/2 + 1/3)          for 0 < x < 1

As we stated above, δ[g( Y) | x] is, in general, a function of x, Let us denote it by hex); that is, hex) = δ[g(Y)|x]. Now we can evaluate the expectation of h(x), a function of X, and will have δ[h(X)] = δ[δ[g(Y)| X]]. This gives us

δ[δ[g(Y)|X]] = δ[h(X)] = ∫_{lim -∞→∞} h(x)f_x(x) dx
                                      = ∫_{lim -∞→∞} δ[g(Y)|x]fx(x) dx 
                                      =∫_{lim -∞→∞} [∫_{lim -∞→∞}g(Y)f_ν|x (y|x)dY]f_x(X) dx 
                                      = ∫_{lim -∞→∞}∫_{lim -∞→∞} g(Y)f_ν|x (y|x)f_xxdydx
                                      =∫_{lim -∞→∞}∫_{lim -∞→∞}g(Y)f_{x , v}.(x , y) dy dx 
                                      = δ[g(Y)]
Thus we have proved for jointly continuous random variables X and Y (the proof for X and Y jointly discrete is similar) the following simple yet very useful theorem.

Theorem 1

Let (X, Y) be a two-dimensional random variable; then

 δ[g( Y)] = δ[ δ[g( Y) | X]],                                         (3)

and in particular

δ[ Y] = δ[δ[ Y| X]].                                                      (4)         

 Definition 2 

Regression curve δ[ Y| X = x] is called the regression curve of Y on x. It is also denoted by
μ_y|x=x =μ_y|x

Definition 3 

Conditional variance The variance of Y given X = x is defined by var [Y|X = x] = δ[ Y²| X = x] - (δ[ Y|X = X])².

Theorem 2 

var [Y] = δ[var [Y|X]] + var [δ[ Y|X]].

PROOF

                    δ[var [Y|X]] = δ[ δ[Y²| X]] - δ[(δ[ Y|X])2]
                                         = δ[ Y²] - (δ[ Y])² - δ[(δ[ Y₁ X])²] + (δ[ Y])²
                                         = var [ Y] - δ[(δ[ Y|X])²] + (δ[δ[ Y|X]])²
                                         = var [Y] - var [δ[ Y|X]].

Let us note in words what the two theorems say. Equation (4) states that the mean of Y is the mean or expectation of the conditional mean of Y, and Theorem 2 states that the variance of Y is the mean or expectation of the conditional variance of Y, plus the variance of the conditional mean of Y.
We will conclude this subsection with one further theorem. The proof can be routinely obtained from Definition 1 and is left as an exercise. Also, the theorem can be generalized to more than two dimensions.

Theorem 3 

Let (X, Y) be a two-dimensional random variable and g₁ () and g₂() functions of one variable. Then                    (i) δ'[g₁( y) + g₂( Y) | X = x] = δ[g₁ (Y) | X = x] +δ[g₂( Y) | X = x].
                (ii) δ[g₁( y)g₂(X) | X = x] = g₂(X)δ[g₁( Y) | X = x].