Chapter 9 Bridging Finite and Super Population Causal Inference

We have focused on the finite population perspective in randomized experiments. It treats all the potential outcomes as fixed numbers. Even if the potential outcomes are random variables, we can condition on them under the finite population perspective. The advantages of this perspective are

  1. 1.

    it focuses on the design of the experiments;

  2. 2.

    it requires minimal assumptions on the data-generating process of the outcomes.

However, it is often criticized for having only internal validity but not necessarily external validity, with informal definitions below.

Definition 9.1 (internal validity)

The statistical analysis is valid for the study samples at hand.

Definition 9.2 (external validity)

The statistical analysis is valid for a broader population beyond the study samples.

Obviously, all experimenters want not only the internal validity but also the external validity of their experiments. Since all statistical properties are conditional on the potential outcomes for the units we have, the results are only about the observed units under the finite population perspective. Then a natural question arises: do the finite population results generalize to a bigger population?

This is a fair critique of the finite population framework conditional on the potential outcomes. However, this can be a philosophical question. What we observed is a finite population, so any experimental design and analysis directly give us information about this finite population. Randomization only ensures internal validity given the potential outcomes of these units. The external validity of the results depends on the sampling process of the units. If the finite population is a representative sample of a larger population we are interested in, then of course the experimental results also have external validity. Otherwise, the results based on randomization inference may not generalize. To rigorously discuss this issue, we need to have a framework with two levels of randomness, one due to the sampling process of the units and the other due to the random assignment of the treatment. See Miratrix et al., (2018) and Abadie et al., (2020) for formal discussions of this idea.25

For some statisticians, this is just a technical problem. We can change the statistical framework, assuming that the units are sampled from a superpopulation. Then all the statements are about the population of interest. This is a convenient framework, although it does not really solve the problem mentioned above. Below, I will introduce this framework for two purposes:

  1. 1.

    it gives a different perspective on randomized experiments;

  2. 2.

    it serves as a bridge between Parts II and III of this book.

The latter purpose is more important since the superpopulation framework allows us to derive more fruitful results for observational studies in which the treatment is not randomly assigned.

49 CRE

Assume

\{Z_{i},Y_{i}(1),Y_{i}(0),X_{i}\}_{i=1}^{n}\stackrel{{\scriptstyle\textup{IID}}}{{\sim}}\{Z,Y(1),Y(0),X\}

from a superpopulation. So we can drop the subscript i for quantities of this population. With a little abuse of notation, we define the population average causal effect as

\tau=E\{Y(1)-Y(0)\}=E\{Y(1)\}-E\{Y(0)\}.

Under the superpopulation framework, we can formulate the CRE as below.

Definition 9.3 (CRE under the superpopulation framework)

We have

Z\ind \{Y(1),Y(0),X\}.

Under Definition 9.3, the average causal effect can be written as

\displaystyle\tau \displaystyle= \displaystyle E\{Y(1)\mid Z=1\}-E\{Y(0)\mid Z=0\} (42)
\displaystyle= \displaystyle E(Y\mid Z=1)-E(Y\mid Z=0),

which equals the difference in expectations of the outcomes. The first line in (42) is the key step that leverages the value of randomization. Since \tau can be expressed as a function of the distributions of the observables, it is nonparametrically identifiable26. The formula (42) immediately suggests a moment estimator \hat{\tau}, which is the difference in means of the outcomes defined before. Conditioning on \bm{Z}, this is then a standard two-sample problem comparing the means of two independent samples. We have

E(\hat{\tau}\mid\bm{Z})=\tau

and

\textup{var}(\hat{\tau}\mid\bm{Z})=\frac{\textup{var}\{Y(1)\}}{n_{1}}+\frac{\textup{var}\{Y(0)\}}{n_{0}}.

Under IID sampling, the sample variances are unbiased for the population variances, so Neyman, (1923)’s variance estimator is unbiased for \textup{var}(\hat{\tau}\mid\bm{Z}). The conservativeness problem goes away under this superpopulation framework.

We can also discuss covariate adjustment. Based on the OLS decompositions (see Chapter 31)

\displaystyle Y(1) \displaystyle= \displaystyle\gamma_{1}+\beta_{1}^{\mkern-1.5mu\mathsf{T}}X+\varepsilon(1), (43)
\displaystyle Y(0) \displaystyle= \displaystyle\gamma_{0}+\beta_{0}^{\mkern-1.5mu\mathsf{T}}X+\varepsilon(0), (44)

we have

\displaystyle\tau \displaystyle= \displaystyle E\{Y(1)-Y(0)\}
\displaystyle= \displaystyle\gamma_{1}-\gamma_{0}+(\beta_{1}-\beta_{0})^{\mkern-1.5mu\mathsf{T}}E(X),

since the residuals \varepsilon(1) and \varepsilon(0) have mean zero due to the inclusion of the intercepts. We can use the OLS with the treated and control data to estimate the coefficients in (43) and (44), respectively. The sample versions of the coefficients are \hat{\gamma}_{1},\hat{\beta}_{1},\hat{\gamma}_{0},\hat{\beta}_{0}, so a covariate-adjusted estimator for \tau is

\hat{\tau}_{\text{adj}}=\hat{\gamma}_{1}-\hat{\gamma}_{0}+(\hat{\beta}_{1}-\hat{\beta}_{0})^{\mkern-1.5mu\mathsf{T}}\bar{X}.

If we center covariates with \bar{X}=0, the above estimator reduces to Lin, (2013)’s estimator

\hat{\tau}_{\textsc{L}}=\hat{\gamma}_{1}-\hat{\gamma}_{0},

which equals the coefficient of Z in the pooled regression with treatment-covariates interactions; see Proposition 6.2.

Unfortunately, the EHW variance estimator does not work for \hat{\tau}_{\textsc{L}} because of the additional uncertainty in the sample mean of covariates \bar{X} under the super population framework. Berk et al., (2013), Negi and Wooldridge, (2021) and Zhao and Ding, 2021a () proposed a correction of the EHW variance estimator by adding an extra term

(\hat{\beta}_{1}-\hat{\beta}_{0})^{\mkern-1.5mu\mathsf{T}}S_{X}^{2}(\hat{\beta}_{1}-\hat{\beta}_{0})/n (45)

where \hat{\beta}_{1}-\hat{\beta}_{0} equals the coefficient of the interaction Z_{i}X_{i} in obtaining Lin, (2013)’s estimator and S_{X}^{2} is the finite-population covariance matrix of the covariates. A conceptually simpler yet computationally intensive approach is to use the bootstrap to estimate the variance; see Chapter 30.F and Problem 9.2.

50 Simulation under the CRE: the super population perspective

The following linestimator function can compute Lin, (2013)’s estimator, the EHW standard error, and the corrected standard error based on (45) for the super population.

library("car")
linestimator = function(Z, Y, X){
## standardize X
X = scale(X)
n = dim(X)[1]
p = dim(X)[2]
## fully interacted OLS
linreg = lm(Y ~ Z*X)
est = coef(linreg)[2]
vehw = hccm(linreg)[2, 2]
## super population correction
inter = coef(linreg)[(p+3):(2*p+2)]
vsuper = vehw + sum(inter*(cov(X)%*%inter))/n
c(est, sqrt(vehw), sqrt(vsuper))
}

I then use simulation to compare the EHW standard error and the corrected standard error. I choose sample size n=500 and use 2000 Monte Carlo repetitions. The potential outcomes are nonlinear in covariates and the error terms are not Normal. The true average causal effect equals 0.

res = replicate(2000, {
n = 500
X = matrix(rnorm(n*2), n, 2)
Y1 = X[, 1] + X[, 1]^2 + runif(n, -0.5, 0.5)
Y0 = X[, 2] + X[, 2]^2 + runif(n, -1, 1)
Z = rbinom(n, 1, 0.6)
Y = Z*Y1 + (1-Z)*Y0
linestimator(Z, Y, X)
})

The following results confirm the theory. First, Lin, (2013)’s estimator is nearly unbiased. Second, the average EHW standard error is smaller than the empirical standard derivation, resulting in undercoverage of the 95% confidence intervals. Third, the average corrected standard error for super population is nearly identical to the empirical standard derivation, resulting in more accurate coverage of the 95% confidence intervals.

> ## bias
> mean(res[1, ])
[1] -0.0001247585
> ## empirical standard deviation
> sd(res[1, ])
[1] 0.1507773
> ## estimated EHW standard error
> mean(res[2, ])
[1] 0.1388657
> ## coverage based on EHW standard error
> mean((res[1, ]-1.96*res[2, ])*(res[1, ]+1.96*res[2, ])<=0)
[1] 0.927
> ## estimated super population standard error
> mean(res[3, ])
[1] 0.1531519
> ## coverage based on population standard error
> mean((res[1, ]-1.96*res[3, ])*(res[1, ]+1.96*res[3, ])<=0)
[1] 0.9525

51 Extension to the SRE

We can extend the discussion in Section 49 to the SRE since it is equivalent to independent CREs within strata. The notation below will be slightly different from that in Chapter 5.

Assume that

\{Z_{i},Y_{i}(1),Y_{i}(0),X_{i}\}\stackrel{{\scriptstyle\textup{IID}}}{{\sim}}\{Z,Y(1),Y(0),X\}.

With a discrete covariate X_{i}\in\{1,\ldots,K\}, we can formulate the SRE as below.

Definition 9.4 (SRE under the super population framework)

We have

Z\ind \{Y(1),Y(0)\}\mid X.

Under Definition 9.4, the conditional average causal effect can be rewritten as

\displaystyle\tau_{[k]} \displaystyle= \displaystyle E\{Y(1)-Y(0)\mid X=k\}
\displaystyle= \displaystyle E(Y\mid Z=1,X=k)-E(Y\mid Z=0,X=k),

so the average causal effect can be rewritten as

\displaystyle\tau \displaystyle= \displaystyle E\{Y(1)-Y(0)\}
\displaystyle= \displaystyle\sum_{k=1}^{K}\textup{pr}(X=k)E\{Y(1)-Y(0)\mid X=k\}
\displaystyle= \displaystyle\sum_{k=1}^{K}\textup{pr}(X=k)\tau_{[k]}.

The discussion in Section 49 holds within all strata, so we can derive the superpopulation analog for the SRE. When there are more than two treatment and control units within each stratum, we can use \hat{V}_{\textsc{S}} as the variance estimator for \textup{var}(\hat{\tau}_{\textsc{S}}).

We will see the exact form of Definition 9.4 in Part III later.

52 Homework Problems

9.1 OLS decomposition of the observed outcome under the CRE

Based on (43) and (44), show that the OLS decomposition of the observed outcome on the treatment, covariates, and their interaction is

Y=\alpha_{0}+\alpha_{Z}Z+\alpha_{X}^{\mkern-1.5mu\mathsf{T}}X+\alpha_{ZX}^{\mkern-1.5mu\mathsf{T}}XZ+\varepsilon

where

\alpha_{0}=\gamma_{0},\quad\alpha_{Z}=\gamma_{1}-\gamma_{0},\quad\alpha_{X}=\beta_{0},\quad\alpha_{ZX}=\beta_{1}-\beta_{0}

and

\varepsilon=Z\varepsilon(1)+(1-Z)\varepsilon(0).

That is,

(\alpha_{0},\alpha_{Z},\alpha_{X},\alpha_{ZX})=\arg\min_{a_{0},a_{Z},a_{X},a_{ZX}}E(Y-a_{0}-a_{Z}Z-a_{X}^{\mkern-1.5mu\mathsf{T}}X-a_{ZX}^{\mkern-1.5mu\mathsf{T}}XZ)^{2}.
9.2 Variance estimation of Lin, (2013)’s estimator under the super population framework

Under the super population framework, simulate \{X_{i},Z_{i},Y_{i}(1),Y_{i}(0)\}_{i=1}^{n} with \beta_{1}\neq\beta_{0} in (43) and (44). Calculate Lin, (2013)’s estimator in each simulated dataset. Compare the true variance and the following estimated variances:

  1. 1.

    the EHW robust variance estimator;

  2. 2.

    the EHW robust variance estimator with the correction term defined in (45);

  3. 3.

    the bootstrap variance estimator, with covariates centered at the beginning but not re-centered for each bootstrap sample;

  4. 4.

    the bootstrap variance estimator, with covariates re-centered for each bootstrap sample.

9.3 Recommended reading

Ding et al., 2017a () provide a unified discussion of the finite population and superpopulation inferences for the average causal effect.

脚注

25. Pearl and Bareinboim, (2014) discussed the transportability problem from the perspective of causal diagrams.

26. In causal inference, we say that a parameter is nonparametrically identifiable if it can be determined by the distribution of the observed variables without imposing further parametric assumptions. See Definition 10.1 later for a more formal discussion.

本书:A First Course in Causal Inference,作者 Peng Ding。原文:arXiv:2305.18793v2,按 CC BY 4.0 许可发布。本站(yuanens.com)把 arXiv 网页版改成分章的纯 HTML,把公式渲染成图片,方便在 Kindle 等电子书阅读器上看,内容没有改动。转换难免有错,以原文为准。本站与作者和出版社无关。