Mdtryhht

I'm reading through my professor's lecture notes on the multiple linear regression model and at one point he writes the following:

$$E[(b-beta)e']=E[(X'X)^-1epsilonepsilon'M_[X]]. $$

In the above equation, $b$, $beta$, $e$, and $epsilon$ are all vectors, $X$ is a regressor matrix and $M$ is the residual maker matrix. In general, I have no idea why these expressions are equivalent, and I'm particularly confused at how the $e$ vector disappears and the $epsilon$ vector appears.

I am assuming $b$ is the OLS estimate of $beta$ and $e$ is the corresponding estimate of $epsilon$. Also I believe you have a typo above in your expression, as there should be $X'$ in front of $epsilon epsilon'$ and behind $(X'X)^-1$.

Start with the definition of $b$:
$$b=(X'X)^-1X'Y.$$
Replacing $Y$ with $Xbeta+epsilon$ in our expression above, we get
$$b=(X'X)^-1X'(Xbeta+epsilon)=beta+(X'X)^-1X'epsilon.$$
It follows that
$$b-beta = (X'X)^-1X'epsilon$$

Now turn to the defintion of $e$:
$$e=Y-hatY=Y-Xb=Y-X(X'X)^-1X'Y.$$

Notice $X(X'X)^-1X'$ is the projection matrix for $X$, which we will denote with $P_[X]$.
Replacing this in our expression for $e,$ we get
$$e=(I-P_[X])Y=M_[X]Y.$$
Replacing $Y$ in the expression above with $Xbeta+epsilon$, we get
$$e=M_[X](Xbeta+epsilon)=M_[X]epsilon,$$
since $M_[X]X$ is a matrix of zeros.

Post-multiplying $b-beta$ with $e'$, we get
$$(b-beta)e'=(X'X)^-1X'epsilon epsilon' M_[X],$$
since $e'=epsilon'M_[X].$

Assuming that the coefficient estimator $b$ is calculated by OLS estimation, you have:

$$beginequation beginaligned
b-beta
&= (X'X)^-1 X'y - beta \[6pt]
&= (X'X)^-1 X'(X beta + epsilon)- beta \[6pt]
&= (X'X)^-1 (X'X) beta + (X'X)^-1 X' epsilon - beta \[6pt]
&= beta + (X'X)^-1 X' epsilon - beta \[6pt]
&= (X'X)^-1 X' epsilon. \[6pt]
endaligned endequation$$

Presumably $e$ is the residual vector (different to the error vector $epsilon$) so we have $e = M_[X] Y = M_[X] epsilon$. Substituting this vector gives:

$$beginequation beginaligned
(b-beta) e'
&= (X'X)^-1 X' epsilon (M_[X] epsilon)' \[6pt]
&= (X'X)^-1 X' epsilon epsilon' M_[X]' \[6pt]
&= (X'X)^-1 X' epsilon epsilon' M_[X]. \[6pt]
endaligned endequation$$

(The last step follows from the fact that $M_[X]$ is a symmetric matrix.) So the expression given by your professor is missing the $X'$ term.