Chapter 4 Neymanian Repeated Sampling Inference in Completely Randomized Experiments

In his seminal paper, Neyman, (1923) not only proposed the notation of potential outcomes but also derived rigorous mathematical results for making inference for the average causal effect under a CRE. In contrast to Fisher’s idea of calculating the p-value under the sharp null hypothesis, Neyman, (1923) proposed an unbiased point estimator and a conservative confidence interval based on the sampling distribution of the point estimator. This chapter will introduce Neyman, (1923)’s fundamental results, which are very important for understanding later chapters in Part II of this book.

17 Finite population quantities

Consider a CRE with n units, where n_{1} of them receive the treatment and n_{0} of them receive the control. For unit i=1,\ldots,n, we have potential outcomes Y_{i}(1) and Y_{i}(0), and individual effect \tau_{i}=Y_{i}(1)-Y_{i}(0). The potential outcomes have finite population means

\bar{Y}(1)=n^{-1}\sum_{i=1}^{n}Y_{i}(1),\quad\bar{Y}(0)=n^{-1}\sum_{i=1}^{n}Y_{i}(0),

variances14

S^{2}(1)=(n-1)^{-1}\sum_{i=1}^{n}\{Y_{i}(1)-\bar{Y}(1)\}^{2},\quad S^{2}(0)=(n-1)^{-1}\sum_{i=1}^{n}\{Y_{i}(0)-\bar{Y}(0)\}^{2},

and covariance

S(1,0)=(n-1)^{-1}\sum_{i=1}^{n}\{Y_{i}(1)-\bar{Y}(1)\}\{Y_{i}(0)-\bar{Y}(0)\}.

The individual effects have mean

\tau=n^{-1}\sum_{i=1}^{n}\tau_{i}=\bar{Y}(1)-\bar{Y}(0).

and variance

S^{2}(\tau)=(n-1)^{-1}\sum_{i=1}^{n}(\tau_{i}-\tau)^{2}.

We have the following relationship between the variances and covariance.

Lemma 4.1

2S(1,0)=S^{2}(1)+S^{2}(0)-S^{2}(\tau).

Lemma 4.1 is a basic result and will be useful for later discussion. The proof of Lemma 4.1 follows from elementary algebra. I leave it as Problem 4.1.

These fixed quantities are functions of the Science Table \{Y_{i}(1),Y_{i}(0)\}_{i=1}^{n}. We are interested in estimating the average causal effect \tau based on the data (Z_{i},Y_{i})_{i=1}^{n} from a CRE.

18 Neyman, (1923)’s theorem

Based on the observed outcomes, we can calculate the sample means

\hat{\bar{Y}}(1)=n_{1}^{-1}\sum_{i=1}^{n}Z_{i}Y_{i},\quad\hat{\bar{Y}}(0)=n_{0}^{-1}\sum_{i=1}^{n}(1-Z_{i})Y_{i},

the sample variances

\hat{S}^{2}(1)=(n_{1}-1)^{-1}\sum_{i=1}^{n}Z_{i}\{Y_{i}-\hat{\bar{Y}}(1)\}^{2},\quad\hat{S}^{2}(0)=(n_{0}-1)^{-1}\sum_{i=1}^{n}(1-Z_{i})\{Y_{i}-\hat{\bar{Y}}(0)\}^{2}.

But there are no sample versions of S(1,0) and S^{2}(\tau) because the potential outcomes Y_{i}(1) and Y_{i}(0) are never jointly observed for each unit i. Neyman, (1923) proved the following theorem.

Theorem 4.1

Under a CRE,

  1. 1.

    the difference-in-means estimator \hat{\tau}=\hat{\bar{Y}}(1)-\hat{\bar{Y}}(0) is unbiased for \tau:

    E(\hat{\tau})=\tau;
  2. 2.

    \hat{\tau} has variance

    \displaystyle\textup{var}(\hat{\tau}) \displaystyle= \displaystyle\frac{S^{2}(1)}{n_{1}}+\frac{S^{2}(0)}{n_{0}}-\frac{S^{2}(\tau)}{n} (15)
    \displaystyle= \displaystyle\frac{n_{0}}{n_{1}n}S^{2}(1)+\frac{n_{1}}{n_{0}n}S^{2}(0)+\frac{2}{n}S(1,0); (16)
  3. 3.

    the variance estimator

    \hat{V}=\frac{\hat{S}^{2}(1)}{n_{1}}+\frac{\hat{S}^{2}(0)}{n_{0}}

    is conservative for estimating \textup{var}(\hat{\tau}) in the sense that

    E(\hat{V})-\textup{var}(\hat{\tau})=\frac{S^{2}(\tau)}{n}\geq 0

    with equality holding if and only if \tau_{i}=\tau for all units.

I will present the proof of Theorem 4.1 in Section 19. Before proving Theorem 4.1, it is important to clarify the meanings of E(\cdot) and \textup{var}(\cdot) in Theorem 4.1. The potential outcomes are all fixed numbers, and only the treatment indicators Z_{i}’s are random. Therefore, the expectations and variances are all over the randomness of the Z_{i}’s, which are random permutations of n_{1} 1’s and n_{0} 0’s. Figure 6 illustrates the randomness of \hat{\tau}, which is a discrete uniform distribution over \{\hat{\tau}^{1},\ldots,\hat{\tau}^{M}\} induced by M=\binom{n}{n_{1}} possible treatment allocations. Compare Figure 6 with Figure 3 to see the key differences between the FRT and Neyman, (1923)’s theorem:

  1. 1.

    The FRT works for any test statistic but Neyman, (1923)’s theorem is only about the difference in means. Although we could derive the properties of other estimators similar to Neyman, (1923)’s theorem, this mathematical exercise is often quite challenging for general estimators.

  2. 2.

    In Figure 3, the observed outcome vector \bm{Y} is fixed but in Figure 6, the observed outcome vector \bm{Y}(\bm{z}^{m}) changes as \bm{z}^{m} changes.

  3. 3.

    The T(\bm{z}^{m},\bm{Y})’s In Figure 3 are all computable based on the observed data, but the \hat{\tau}^{m}’s in Figure 6 are hypothetical values because not all potential outcomes are known.

插图
Figure 6: Illustration of Neyman, (1923)’s theorem, where \bm{Y}(\bm{z}^{m}) is the observed outcome vector under the treatment vector \bm{z}^{m}.

The point estimator \hat{\tau} is standard but it has a non-trivial variance under the potential outcomes framework with a CRE. The variance formula (15) differs from the classic variance formula for the difference in means15 because it not only depends on the finite population variances of the potential outcomes but also depends on the finite population variance of the individual effects, or, equivalently, the finite population covariance of the potential outcomes. Unfortunately, S^{2}(\tau) and S(1,0) are not identifiable from the data because Y_{i}(1) and Y_{i}(0) are never jointly observed.

The formula (15) is a little puzzling in that the more heterogeneous the individual effects are the smaller the variability of \hat{\tau} is. Section 21.1 will use numerical examples to verify (15). What is the intuition here? I give an explanation based on the equivalent form (16). Compare the case with positively correlated potential outcomes and the case with negatively correlated potential outcomes. Although the treatment group is a simple random sample from the finite population of n units, it is possible to observe relatively large treatment potential outcomes in a realized experiment. Assume this happens.

  1. 1.

    If S(1,0)>0, then those treated units also have relatively large control potential outcomes. Consequently, the observed outcomes under control are relatively small, which results in large \hat{\tau}.

  2. 2.

    If S(1,0)<0, then those treated units have relatively small control potential outcomes. Consequently, the observed outcomes under control are relatively large, which results in small \hat{\tau}.

The reverse can also happen if we observe relatively small treatment potential outcomes in a realized experiment. Overall, although the unbiasedness of \hat{\tau} does not depend on the correlation between the potential outcomes, it is more likely to observe more extreme \hat{\tau} under S(1,0)>0 than under S(1,0)<0. So the variance of \hat{\tau} is larger when the potential outcomes are positively correlated.

Li and Ding, (2017, Theorem 5 and Proposition 3) further proved the following asymptotic Normality of \hat{\tau} based on the finite population CLT.

Theorem 4.2

Let n\rightarrow\infty and n_{1}\rightarrow\infty. If n_{1}/n has a limiting value in (0,1), \{S^{2}(1),S^{2}(0),S(1,0)\} have limiting values, and

\max_{1\leq i\leq n}\{Y_{i}(1)-\bar{Y}(1)\}^{2}/n\rightarrow 0,\quad\max_{1\leq i\leq n}\{Y_{i}(0)-\bar{Y}(0)\}^{2}/n\rightarrow 0,

then

\frac{\hat{\tau}-\tau}{\sqrt{\textup{var}(\hat{\tau})}}\rightarrow\textsc{N}(0,1)

in distribution, and

\hat{S}^{2}(1)\rightarrow S^{2}(1),\quad\hat{S}^{2}(0)\rightarrow S^{2}(0)

in probability.

The proof of Theorem 4.2 is technical and beyond the scope of this book. It ensures that the sampling distribution of \hat{\tau} can be approximated by a Normal distribution with a large sample size and some regularity conditions. Moreover, it ensures that the sample variances of the outcomes are consistent for the population variances, which further ensures that the probability limit of Neyman, (1923)’s variance estimator is larger than the true variance of \hat{\tau}. This justifies a conservative large-sample confidence interval for \tau:

\hat{\tau}\pm z_{1-\alpha/2}\sqrt{\hat{V}},

where z_{1-\alpha/2} is the 1-\alpha/2 upper quantile of the standard Normal random variable. It is the same as the confidence interval for the standard two-sample problem asymptotically (see Chapter 30.D.1). This confidence interval covers \tau with probability at least as large as 1-\alpha when the sample size is large enough. By duality, the confidence interval implies a test for

H_{0\textsc{n}}:\tau=0,

which is called the weak null hypothesis.

Due to the fundamental problem of missing one potential outcome, we can at most obtain a conservative variance estimator. In statistics, the definition of the confidence interval allows for over coverage and thus conservativeness in variance estimation (see Chapter 30.B). Conservativeness is not a big problem if underreporting the treatment effect is not a big problem in practice. Sometimes, it can be problematic if the outcomes measure the side effects of a treatment. In medical experiments, underreporting the side effects of a new drug can have severe consequences on patients’ health.

19 Proofs

In this section, I will prove Theorem 4.1.

First, the unbiasedness of \hat{\tau} follows from the representation

\displaystyle\hat{\tau} \displaystyle= \displaystyle n_{1}^{-1}\sum_{i=1}^{n}Z_{i}Y_{i}-n_{0}^{-1}\sum_{i=1}^{n}(1-Z_{i})Y_{i}
\displaystyle= \displaystyle n_{1}^{-1}\sum_{i=1}^{n}Z_{i}Y_{i}(1)-n_{0}^{-1}\sum_{i=1}^{n}(1-Z_{i})Y_{i}(0)

and the linearity of the expectation:

\displaystyle E(\hat{\tau}) \displaystyle= \displaystyle E\left\{n_{1}^{-1}\sum_{i=1}^{n}Z_{i}Y_{i}(1)-n_{0}^{-1}\sum_{i=1}^{n}(1-Z_{i})Y_{i}(0)\right\}
\displaystyle= \displaystyle n_{1}^{-1}\sum_{i=1}^{n}E(Z_{i})Y_{i}(1)-n_{0}^{-1}\sum_{i=1}^{n}E(1-Z_{i})Y_{i}(0)
\displaystyle= \displaystyle n_{1}^{-1}\sum_{i=1}^{n}\frac{n_{1}}{n}Y_{i}(1)-n_{0}^{-1}\sum_{i=1}^{n}\frac{n_{0}}{n}Y_{i}(0)
\displaystyle= \displaystyle n^{-1}\sum_{i=1}^{n}Y_{i}(1)-n^{-1}\sum_{i=1}^{n}Y_{i}(0)
\displaystyle= \displaystyle\tau.

Second, we can further write \hat{\tau} as

\displaystyle\hat{\tau} \displaystyle= \displaystyle\sum_{i=1}^{n}Z_{i}\left\{\frac{Y_{i}(1)}{n_{1}}+\frac{Y_{i}(0)}{n_{0}}\right\}-n_{0}^{-1}\sum_{i=1}^{n}Y_{i}(0).

The variance of \hat{\tau} follows from Lemma 32.2 of simple random sampling:

\displaystyle\textup{var}(\hat{\tau}) \displaystyle= \displaystyle\frac{n_{1}n_{0}}{n(n-1)}\sum_{i=1}^{n}\left\{\frac{Y_{i}(1)}{n_{1}}+\frac{Y_{i}(0)}{n_{0}}-\frac{\bar{Y}(1)}{n_{1}}-\frac{\bar{Y}(0)}{n_{0}}\right\}^{2}
\displaystyle= \displaystyle\frac{n_{1}n_{0}}{n(n-1)}\left[\frac{1}{n_{1}^{2}}\sum_{i=1}^{n}\{Y_{i}(1)-\bar{Y}(1)\}^{2}+\frac{1}{n_{0}^{2}}\sum_{i=1}^{n}\{Y_{i}(0)-\bar{Y}(0)\}^{2}\right.
\displaystyle\left.\qquad\qquad+\frac{2}{n_{1}n_{0}}\sum_{i=1}^{n}\{Y_{i}(1)-\bar{Y}(1)\}\{Y_{i}(0)-\bar{Y}(0)\}\right]
\displaystyle= \displaystyle\frac{n_{0}}{n_{1}n}S^{2}(1)+\frac{n_{1}}{n_{0}n}S^{2}(0)+\frac{2}{n}S(1,0).

From Lemma 4.1, we can also write the variance as

\displaystyle\textup{var}(\hat{\tau}) \displaystyle= \displaystyle\frac{n_{0}}{n_{1}n}S^{2}(1)+\frac{n_{1}}{n_{0}n}S^{2}(0)+\frac{1}{n}\{S^{2}(1)+S^{2}(0)-S^{2}(\tau)\}
\displaystyle= \displaystyle\frac{S^{2}(1)}{n_{1}}+\frac{S^{2}(0)}{n_{0}}-\frac{S^{2}(\tau)}{n}.

Third, because the treatment group is a simple random sample of size n_{1} from the n units, Lemma 32.3 ensures that the sample variance of Y_{i}(1)’s is unbiased for its population variance:

E\{\hat{S}^{2}(1)\}=S^{2}(1).

Similarly, E\{\hat{S}^{2}(0)\}=S^{2}(0). Therefore, \hat{V} is unbiased for the first two terms in (15).

20 Regression analysis of the CRE

Practitioners often use regression-based inference for the average causal effect \tau. A standard approach is to run the ordinary least squares (OLS) of the outcomes on the treatment indicators with an intercept

(\hat{\alpha},\hat{\beta})=\arg\min_{(a,b)}\sum_{i=1}^{n}(Y_{i}-a-bZ_{i})^{2},

and use the coefficient of the treatment \hat{\beta} as the estimator for the average causal effect. We can show the coefficient \hat{\beta} equals the difference in means:

\hat{\beta}=\hat{\tau}. (17)

However, the usual variance estimator from the OLS (see (161) in Chapter 31), e.g., the output from the lm function of R, equals

\displaystyle\hat{V}_{\textsc{ols}} \displaystyle= \displaystyle\frac{n\left(n_{1}-1\right)}{(n-2)n_{1}n_{0}}\hat{S}^{2}(1)+\frac{n\left(n_{0}-1\right)}{(n-2)n_{1}n_{0}}\hat{S}^{2}(0) (18)
\displaystyle\approx \displaystyle\frac{\hat{S}^{2}(1)}{n_{0}}+\frac{\hat{S}^{2}(0)}{n_{1}},

where the approximation holds with large n_{1} and n_{0}. It differs from \hat{V} even with large n_{1} and n_{0}.

Fortunately, the Eicker–Huber–White (EHW) robust variance estimator (see (160) in Chapter 31) is close to \hat{V}:

\displaystyle\hat{V}_{\textsc{ehw}} \displaystyle= \displaystyle\frac{\hat{S}^{2}(1)}{n_{1}}\frac{n_{1}-1}{n_{1}}+\frac{\hat{S}^{2}(0)}{n_{0}}\frac{n_{0}-1}{n_{0}} (19)
\displaystyle\approx \displaystyle\frac{\hat{S}^{2}(1)}{n_{1}}+\frac{\hat{S}^{2}(0)}{n_{0}}

where the approximation holds with large n_{1} and n_{0}. It is almost identical to \hat{V}. Moreover, the so-called HC2 variant of the EHW robust variance estimator is identical to \hat{V}. The hccm function in the car package returns the EHW robust variance estimator as well as its HC2 variant.

Problem 4.3 provides more technical details for (17)–(19).

21 Examples

21.1 Simulation

I first choose the sample size as n=100 with 60 treated and 40 control units, and generate the potential outcomes with constant individual causal effects.

n = 100
n1 = 60
n0 = 40
y0 = rexp(n)
y0 = sort(y0, decreasing = TRUE)
y1 = y0 + 1

With the Science Table fixed, I repeatedly generate CREs and apply Theorem 4.1 to obtain the point estimator, the conservative variance estimator, and the confidence interval based on the Normal approximation. The (1,1)th panel of Figure 7 shows the scatter plot of the potential outcomes, and the (1,2)th panel of Figure 7 shows the histogram of \hat{\tau}-\tau.

I then change the potential outcomes by sorting the control potential outcomes in reverse order

y0 = sort(y0, decreasing = FALSE)

and repeat the above simulation. The (2,1)th panel of Figure 7 shows the scatter plot of the potential outcomes, and the (2,2)th panel of Figure 7 shows the histogram of \hat{\tau}-\tau.

I finally permute the control potential outcomes randomly

y0 = sample(y0)

and repeat the above simulation. The (3,1)th panel of Figure 7 shows the scatter plot of the potential outcomes, and the (2,2)th panel of Figure 7 shows the histogram of \hat{\tau}-\tau.

Importantly, in the above three sets of simulations, the correlations between potential outcomes are different but the marginal distributions are the same. The following table compares the true variances, the average estimated variances, and the coverage rates of the 95\% confidence intervals.

constant negative independent
var 0.036 0.007 0.020
estimated var 0.036 0.036 0.036
coverege rate 0.947 1.000 0.989

The true variance depends on the correlation between the potential outcomes, with positively correlated potential outcomes corresponding to a larger sampling variance. This verifies (16). The estimated variances are almost identical because the formula of \hat{V} depends only on the marginal distributions of the potential outcomes. Due to the discrepancy between the true and estimated variances, the coverage rates differ across the three sets of simulations. Only with constant causal effects, the estimated variance is identical to the true variance, verifying point 3 of Theorem 4.1.

Figure 7 also shows the Normal density curves based on the CLT for \hat{\tau}. They are very close to the histogram over simulations, verifying Theorem 4.2.

插图
Figure 7: Left: joint distributions of the potential outcomes with the same marginal distributions. Right: sampling distribution of \hat{\tau}-\tau over 10^{4} simulations with different joint distributions of the potential outcomes.
21.2 Heavy-tailed outcome and failure of Normal approximations

The CLT of \hat{\tau} in Theorem 4.2 holds under some regularity conditions. Those conditions will be violated with heavy-tailed potential outcomes. We can modify the above simulation studies to illustrate this point. Assume the individual causal effects are constant but the control potential outcomes are contaminated by a Cauchy component with probability 0.1,0.3, or 0.5. The following code generates the potential outcomes with the probability of contamination being 0.1.

eps = rbinom(n, 1, 0.1)
y0 = (1 - eps)*rexp(n) + eps*rcauchy(n)
y1 = y0 + 1

Figures 8 and 9 show two realizations of the histograms of \hat{\tau}-\tau with the corresponding Normal approximations. With heavy-tailed potential outcomes, the Normal approximations are quite poor. Moreover, unlike Figure 7, the histograms are quite sensitive to the random seed of the simulation.

插图
Figure 8: Sampling distribution of \hat{\tau}-\tau with contaminated potential outcomes (with different contamination probabilities): realization one
插图
Figure 9: Sampling distribution of \hat{\tau}-\tau with contaminated potential outcomes (with different contamination probabilities): realization two
21.3 Application

I analyzed the lalonde data in Chapter 13 to illustrate the idea of the FRT. Now I use the data to illustrate the theory in this chapter.

> library(Matching)
> data(lalonde)
> z = lalonde$treat
> y = lalonde$re78

We can easily calculate the point estimator and standard error based on the formulas in Theorem 4.1:

> n1= sum(z)
> n0= length(z) - n1
> tauhat = mean(y[z==1]) - mean(y[z==0])
> vhat = var(y[z==1])/n1 + var(y[z==0])/n0
> sehat = sqrt(vhat)
> tauhat
[1] 1794.343
> sehat
[1] 670.9967

Practitioners often use OLS to estimate the average causal effect which also gives a standard error.

> olsfit = lm(y ~ z)
> summary(olsfit)$coef[2, 1: 2]
Estimate Std. Error
1794.3431 632.8536

However, the above standard error seems too small compared to the one based on Theorem 4.1. This can be solved by using the EHW robust standard error.

> library(car)
> sqrt(hccm(olsfit)[2, 2])
[1] 672.6823
> sqrt(hccm(olsfit, type = "hc0")[2, 2])
[1] 669.3155
> sqrt(hccm(olsfit, type = "hc2")[2, 2])
[1] 670.9967

22 Homework Problems

4.1 Proof of Lemma 4.1

Prove Lemma 4.1.

4.2 Alternative proof of Theorem 4.1

Under a CRE, calculate

\textup{var}\{\hat{\bar{Y}}(1)\},\quad\textup{var}\{\hat{\bar{Y}}(0)\},\quad\textup{cov}\{\hat{\bar{Y}}(1),\hat{\bar{Y}}(0)\}

and use these formulas to calculate \textup{var}(\hat{\tau}).

Remark: Use the results in Chapter 32.

4.3 Neymanian inference and OLS

Prove (17)–(19). Moreover, prove that the HC2 variant of the EHW robust variance estimator recovers \hat{V} exactly.

Remark: Chapter 31 reviews some important technical results about OLS.

4.4 Treatment effect heterogeneity

Show that S^{2}(\tau)=0 implies that S^{2}(1)=S^{2}(0). Given a counterexample with S^{2}(1)=S^{2}(0) but S^{2}(\tau)\neq 0.

Show that S^{2}(1)<S^{2}(0) implies that

S(Y(0),\tau)=(n-1)^{-1}\sum_{i=1}^{n}\{Y_{i}(0)-\bar{Y}(0)\}(\tau_{i}-\tau)<0.

Give a counterexample with S^{2}(1)>S^{2}(0) but S(Y(0),\tau)<0.

Remark: The first result states that no treatment effect heterogeneity implies equal variances in the treated and control potential outcomes. But the converse is not true. The second result states that if the treated potential outcome has a smaller variance than the control potential outcome, then the individual treatment effect is negatively correlated with the control potential outcome. But the converse is not true. Gerber and Green, (2012, page 293) and Ding et al., (2019, Appendix B.3) gave related discussions.

4.5 A better bound of the variance formula

Neyman, (1923)’s conservative variance estimator essentially uses the following upper bound on the true variance:

\textup{var}(\widehat{\tau})=\frac{S^{2}(1)}{n_{1}}+\frac{S^{2}(0)}{n_{0}}-\frac{S^{2}(\tau)}{n}\leq\frac{S^{2}(1)}{n_{1}}+\frac{S^{2}(0)}{n_{0}},

which uses the trivial fact that S^{2}(\tau)\geq 0. Show the following upper bound

\displaystyle\textup{var}(\widehat{\tau})\leq\frac{1}{n}\left\{\sqrt{\frac{n_{0}}{n_{1}}}S(1)+\sqrt{\frac{n_{1}}{n_{0}}}S(0)\right\}^{2}. (20)

When does the equality in (20) hold?

The upper bound (20) motivates another conservative variance estimator

\hat{V}^{\prime}=\frac{1}{n}\left\{\sqrt{\frac{n_{0}}{n_{1}}}\hat{S}(1)+\sqrt{\frac{n_{1}}{n_{0}}}\hat{S}(0)\right\}^{2}.

Section 21.1 used \hat{V} in the simulation. Repeat the simulation with an additional comparison with the variance estimator \hat{V}^{\prime} and the associated confidence interval.

Remark: You may find Section 30.A.4 useful for the proof. The upper bound (20) can be further improved. Aronow et al., (2014) derived the sharp upper bound for \textup{var}(\widehat{\tau}) using the Frechet–Hoeffding inequality. Those improvements are rarely used in practice mainly for two reasons. First, they are more complicated than \hat{V} which can be conveniently implemented by OLS. Second, the confidence interval based on \hat{V} also works under other formulations, for example, under a true linear model of the outcome on the treatment, but those improvements do not. Although they are theoretically interesting, those improvements have little practical impact.

4.6 Vector version of Neyman, (1923)

The classic result of Neyman, (1923) is about a scalar outcome. It is common to have multiple outcomes in practice. Therefore, we now extend the potential outcomes to vectors. We consider the average causal effect on a vector outcome \bm{V}\in\mathbb{R}^{K}:

\tau_{\bm{V}}=\frac{1}{n}\sum_{i=1}^{n}\left\{\bm{V}_{i}(1)-\bm{V}_{i}(0)\right\},

where \bm{V}_{i}(1) and \bm{V}_{i}(0) are the potential outcomes of \bm{V} for unit i. The Neyman-type estimator for \tau_{\bm{V}} is the difference between the sample mean vectors of the observed outcomes under treatment and control:

\widehat{\tau}_{\bm{V}}=\bar{\bm{V}}_{1}-\bar{\bm{V}}_{0}=\frac{1}{n_{1}}\sum_{i=1}^{n}Z_{i}\bm{V}_{i}-\frac{1}{n_{0}}\sum_{i=1}^{n}(1-Z_{i})\bm{V}_{i}.

Consider a CRE. Show that \widehat{\tau}_{\bm{V}} is unbiased for \tau_{\bm{V}}. Find the covariance matrix of \widehat{\tau}_{\bm{V}}. Find a (possibly conservative) estimator for the variance.

4.7 Inference in the BRE

In this book, I use the following definition for the BRE.

Definition 4.1 (BRE)

The treatment indicators Z_{i}’s are IID Bernoulli(\pi) with n_{1}=\sum_{i=1}^{n}Z_{i} receiving the treatment and n_{0}=\sum_{i=1}^{n}(1-Z_{i}) receiving the control, respectively.

First, we can use the FRT to analyze the BRE. How do we test H_{0\textsc{f}} in the BRE? Can we use the same FRT procedure as in the CRE if the actual experiment is the BRE? If yes, give a justification; if no, explain why.

Second, we can obtain point estimators for \tau and find the associated variance estimators, as Neyman, (1923) did for the CRE.

  1. 1.

    Is \hat{\tau} unbiased for \tau? Is it consistent?

  2. 2.

    Find an unbiased estimator for \tau.

  3. 3.

    Compare the variance of the above unbiased estimator and the asymptotic variance of \hat{\tau}.

Remark: Under the BRE, the estimator \hat{\tau} does not have finite variance but the variance of its asymptotic distribution is finite.

4.8 Recommended reading

Ding, (2016) compared the Fisherian approach and Neymanian approach to analyzing the CRE.

脚注

14. Here the divisor n-1 makes the main theorems in this chapter more elegant. Changing the divisor to n complicates the formulas but does not change the results fundamentally. With large n, the difference is minor.

15. In the classic two-sample problem, the outcomes under the treatment are IID draws from a distribution with mean \mu_{1} and variance \sigma_{1}^{2}, and the outcomes under the control are IID draws from a distribution with mean \mu_{0} and variance \sigma_{0}^{2}. Under this assumption, we have \textup{var}(\hat{\tau})=\frac{\sigma_{1}^{2}}{n_{1}}+\frac{\sigma_{0}^{2}}{n_{0}}. Here, \textup{var}(\cdot) is over the randomness of the outcomes. This variance formula does not involve a third term that depends on the variance of the individual causal effects.

本书:A First Course in Causal Inference,作者 Peng Ding。原文:arXiv:2305.18793v2,按 CC BY 4.0 许可发布。本站(yuanens.com)把 arXiv 网页版改成分章的纯 HTML,把公式渲染成图片,方便在 Kindle 等电子书阅读器上看,内容没有改动。转换难免有错,以原文为准。本站与作者和出版社无关。