The Conditional Expectation Function (CEF)

The CEF for a dependent variable \(Y_i \), given a vector of \(k \times 1\) covariates \(X_i\) (with elements \(x_{ki}\)) is the expectation of the population average (or an infinitely large sample).

$$E[Y_i|X_i=x]$$

The CEF is only a function of $X_i$. The potential outcomes framework presented an important case of the CEF where $D_i\in 0,1$. If $X_i$ is random then the CEF is also random, but a particular value of $X_i$ can give a concrete answer to the CEF.

Assume that $Y_i$ with a conditional probability density function represented by $f_y(t|X_i=x)$ for $Y_i=t$. Then

$$E[Y_i|X_i=x]=\int t{f}_y(t|X_i=x)dt$$

In the discrete case
$$E[Y_i|X_i=x]=\sum _{t}t\cdot P(Y_i=t|X_i=x)$$

Law of Iterated Expectations

$$E[Y_i]=EE[Y_i|X_i]$$

A proof of LIE for the continuous case

$$\begin{array}{rl}EE[Y_i|X_i]& =\int E[Y_i|X_i=u]g_x(u)du\\ & =\int [\int t{f}_y(t|X_i=x)dt]g_x(u)du\\ & =\int \int t{f}_y(t|X_i=x)g_x(u)dtdu\\ & =\int t\int [f_y(t|X_i=x)g_x(u)]dt\\ & =\int t[\int f_{xy}(u,t)du]dt\\ & =\int t{g}_y(t)dt=E[y_i]\end{array}$$ In the discrete case, I start by noting that

$$P(Y_i=y_t|X_i=x_j)=\frac{P(Y_i=y_t,X_i=x_j)}{P(X_i=x_j)}$$

$$\begin{array}{rl}EE[Y_i|X_i]& =\sum _{j=1}^{d_x}E[Y_i|X_i=x_j]\cdot P(X_i=x_j)\\ & =\sum _{j=1}^{d_x}[\sum _{k=1}^{d_y}{y}_t\cdot P(Y_i=y_k|X_i=x_j)]\cdot P(X_i=x_j)\\ & =\sum _{k=1}^{d_y}{y}_t\sum _{j=1}^{d_x}P(Y_i=y_t,X=X_j)\\ & =\sum _{k=1}^{d_y}{y}_{t}P(Y_i=y_t)=E[Y_i]\end{array}$$ **The CEF Decomposition Property**

$$Y_i=E[y_i|X_i]+{\epsilon }_i$$ Under the assumptions that

• (1) $\epsilon$ is mean independent of $X_i$, $E[{\epsilon }_i|X_i]=0$
• (2) $\epsilon$ is uncorrelated with any function of $X_i$

The CEF Prediction Property

Let $m(X_i)$ be any function of $X_i$, the CEF solves

$$E[Y_i|X_i]=\arg \underset{m(X_i)}{min}E[(Y_i-m(X_i){)}^2]$$

Proof

$$\begin{array}{rl}(Y_i-m(X_i){)}^2& =((Y_i-E[Y_i|X_i])+(E[Y_i|X_i]-m(X_i)){)}^2\\ & =(Y_i-E[Y_i|X_i]{)}^2+2[(Y_i-E[Y_i|X_i])\times (E[Y_i|X_i]-m(X_i))]+(E[Y_i|X_i]-m(X_i){)}^2\\ & ={\epsilon }_i^2+2h(x_i){\epsilon }_i+(E[Y|X_i]-m(X_i){)}^2\end{array}$$ Taking the expectation we arrive at

$$\begin{array}{rl}E[(Y_i-m(X_i){)}^2]& =E[{\epsilon }_i^2]+2E[h(x_i)]E[{\epsilon }_i]+E[(E[Y_i|X_i]-m(X_i){)}^2]\\ & =E[{\epsilon }_i^2]+0+E[(E[Y_i|X_i]-m(X_i){)}^2]\\ & =E[{\epsilon }_i^2]+E[E[Y_i|X_i{]}^2]-2E[E[Y_i|X_i]m(X_i)]+E[m(X_i{)}^2]\end{array}$$

Taking the first order conditions

$$\begin{array}{r}\frac{\partial (Y_i-m(X_i){)}^2}{\partial m(X_i)}=-2E[Y_i|X_i]+2m(X_i)=0\\ \therefore m(X_i)=E[Y_i|X_i]\end{array}$$

The ANOVA Theorem

$$V(Y_i)=V(E[Y_i|X_i])+E[V(Y_i|X_i)]$$ Proof

By the CEF decomposition property we know that

$${\epsilon }_i=Y_i-E[Y_i|X_i]$$ Because ${\epsilon }_i$ and $E[Y_i|X_i]$ are by definition uncorrelated the variance of ${\epsilon }_i$ can be written as:

$$\begin{array}{rl}V({\epsilon }_i)& =E({\epsilon }_i^2)+E({\epsilon }_i{)}^2=E({\epsilon }_i^2)\\ & =E[E[{\epsilon }_i|X_i{]}^2]=E[V(Y_i|X_i)]\end{array}$$