Chapter 32 Some Useful Lemmas for Simple Random Sampling From a Finite Population

Appendix 32.A Lemmas

Simple random sampling is a basic topic in standard survey sampling textbooks (Cochran,, 1953, e.g.,). Below I review some results for simple random sampling that are useful for design-based inference in the CRE in Chapters 3 and 4. I define simple random sampling based on the distribution of the inclusion indicators.

Definition 32.1 (simple random sampling)

A simple random sample of size n_{1} consists of a subset from a finite population of n units indexed by i=1,\ldots,n. Let \bm{Z}=(Z_{1},\ldots,Z_{n}) be the inclusion indicators of the n units with Z_{i}=1 if unit i is sampled and Z_{i}=0 otherwise. The vector \bm{Z} can take \binom{n}{n_{1}} possible permutations of a vector of n_{1} 1’s and n_{0} 0’s, and each has equal probability.

By Definition 32.1, simple random sampling is a special form of sampling without replacement because it does not allow for repeatedly sampling the same unit. Lemma 32.1 below summarizes the first two moments of the inclusion indicators.

Lemma 32.1

Under simple random sampling, we have

E(Z_{i})=\frac{n_{1}}{n},\quad\textup{var}(Z_{i})=\frac{n_{1}n_{0}}{n^{2}},\quad\textup{cov}(Z_{i},Z_{j})=-\frac{n_{1}n_{0}}{n^{2}(n-1)}.

In more compact forms, we have

E(\bm{Z})=\frac{n_{1}}{n}\bm{1}_{n},\quad\textup{cov}(\bm{Z})=\frac{n_{1}n_{0}}{n(n-1)}\bm{P}_{n},

where \bm{1}_{n} is a n-dimensional vector of 1’s, and \bm{P}_{n}=\bm{I}_{n}-n^{-1}\bm{1}_{n}\bm{1}_{n}^{\mkern-1.5mu\mathsf{T}} is the n\times n projection matrix orthogonal to \bm{1}_{n}.

Let \{c_{1},\ldots,c_{n}\} be a finite population with mean \bar{c}=\sum_{i=1}^{n}c_{i}/n and variance

S_{c}^{2}=(n-1)^{-1}\sum_{i=1}^{n}(c_{i}-\bar{c})^{2};

let \{d_{1},\ldots,d_{n}\} be another finite population with mean \bar{d}=\sum_{i=1}^{n}d_{i}/n and variance

S_{d}^{2}=(n-1)^{-1}\sum_{i=1}^{n}(d_{i}-\bar{d})^{2};

their covariance is

S_{cd}=(n-1)^{-1}\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d}).

Under simple random sampling, the sample means are

\hat{\bar{c}}=n_{1}^{-1}\sum_{i=1}^{n}Z_{i}c_{i},\quad\hat{\bar{d}}=n_{1}^{-1}\sum_{i=1}^{n}Z_{i}d_{i};

sample variances are

\hat{S}_{c}^{2}=(n_{1}-1)^{-1}\sum_{i=1}^{n}Z_{i}(c_{i}-\hat{\bar{c}})^{2},\quad\hat{S}_{d}^{2}=(n_{1}-1)^{-1}\sum_{i=1}^{n}Z_{i}(d_{i}-\hat{\bar{d}})^{2};

the sample covariance is

\hat{S}_{cd}=(n_{1}-1)^{-1}\sum_{i=1}^{n}Z_{i}(c_{i}-\hat{\bar{c}})(d_{i}-\hat{\bar{d}}).

Lemma 32.2 below gives the moments of the sample means \hat{\bar{c}} and \hat{\bar{d}}.

Lemma 32.2

Under simple random sampling, the sample means are unbiased for the population means:

E(\hat{\bar{c}})=\bar{c},\quad E(\hat{\bar{d}})=\bar{d}.

Their variances and covariance are

\displaystyle\textup{var}\left(\hat{\bar{c}}\right)=\frac{n_{0}}{nn_{1}}S_{c}^{2},\quad\textup{var}\left(\hat{\bar{d}}\right)=\frac{n_{0}}{nn_{1}}S_{d}^{2},\quad\textup{cov}\left(\hat{\bar{c}},\hat{\bar{d}}\right)=\frac{n_{0}}{nn_{1}}S_{cd}.

In the variance formulas in Lemma 32.2, the coefficient n_{0}/(nn_{1})=1/n_{1}\times(1-n_{1}/n) in Lemma 32.2 is different from 1/n_{1} under IID sampling. The additional term 1-n_{1}/n=n_{0}/n is called the finite population correction factor.

Lemma 32.3 below gives the unbiasedness of the sample variances and covariance for estimating the population analogs.

Lemma 32.3

Under simple random sampling, the sample variances and covariance are unbiased for their population versions:

E(\hat{S}_{c}^{2})=S_{c}^{2},\quad E(\hat{S}_{d}^{2})=S_{d}^{2},\quad E(\hat{S}_{cd})=S_{cd}.

An important practical question is to make inference about \bar{c} based on the simple random sample. This requires a more precise characterization of the distribution of its unbiased estimator \hat{\bar{c}}. The finite-sample exact distribution of \hat{\bar{c}} depends on the whole finite population \{c_{1},\ldots,c_{n}\}, which is unknown. The following finite population CLT characterizes the asymptotic distribution of \hat{\bar{c}} based on its first two moments.

Lemma 32.4 (finite population CLT)

Under Under simple random sampling, as n\rightarrow\infty, if

\frac{\max_{1\leq i\leq n}(c_{i}-\bar{c})^{2}}{\min(n_{1},n_{0})S_{c}^{2}}\rightarrow 0,

then

\frac{\hat{\bar{c}}-\bar{c}}{\sqrt{\frac{n_{0}}{nn_{1}}S_{c}^{2}}}\rightarrow\textsc{N}(0,1)

in distribution, and \hat{S}_{c}^{2}/S_{c}^{2}\rightarrow 1 in probability.

Lemma 32.4 justifies the Wald-type 1-\alpha confidence interval for \bar{c}:

\hat{\bar{c}}\pm z_{1-\alpha/2}\sqrt{\frac{n_{0}}{nn_{1}}\hat{S}_{c}^{2}}

where z_{1-\alpha/2} is the 1-\alpha/2 upper quantile of the standard Normal random variable.

Appendix 32.B Proofs

Proof of Lemma 32.1: By symmetry, the Z_{i}’s have the same mean, so

n_{1}=\sum_{i=1}^{n}Z_{i}=E\left(\sum_{i=1}^{n}Z_{i}\right)=nE(Z_{i})\Longrightarrow E(Z_{i})=n_{1}/n.

Because Z_{i} is a Bernoulli random variable, its variance is

\textup{var}(Z_{i})=\frac{n_{1}}{n}\left(1-\frac{n_{1}}{n}\right)=\frac{n_{1}n_{0}}{n^{2}}.

By symmetry again, the Z_{i}’s have the same variance and the pairs (Z_{i},Z_{j})’s have the same covariance, so

0=\textup{var}\left(\sum_{i=1}^{n}Z_{i}\right)=n\textup{var}(Z_{i})+n(n-1)\textup{cov}(Z_{i},Z_{j})

which implies that

\textup{cov}(Z_{i},Z_{j})=-\frac{n_{1}n_{0}}{n^{2}(n-1)}\quad(i\neq j).

\square

Proof of Lemma 32.2: The unbiasedness of the sample mean follows from linearity. For example,

E(\hat{\bar{c}})=E\left(\frac{1}{n_{1}}\sum_{i=1}^{n}Z_{i}c_{i}\right)=\frac{1}{n_{1}}\sum_{i=1}^{n}E(Z_{i})c_{i}=\bar{c}.

The covariance of the sample means is

\displaystyle\textup{cov}(\hat{\bar{c}},\hat{\bar{d}})
\displaystyle= \displaystyle\textup{cov}\left\{\frac{1}{n_{1}}\sum_{i=1}^{n}Z_{i}(c_{i}-\bar{c}),\frac{1}{n_{1}}\sum_{i=1}^{n}Z_{i}(d_{i}-\bar{d})\right\}
\displaystyle= \displaystyle\frac{1}{n_{1}^{2}}\left[\sum_{i=1}^{n}\textup{var}(Z_{i})(c_{i}-\bar{c})(d_{i}-\bar{d})+\sum_{i\neq j}\textup{cov}(Z_{i},Z_{j})(c_{i}-\bar{c})(d_{j}-\bar{d})\right]
\displaystyle= \displaystyle\frac{1}{n_{1}^{2}}\left[\frac{n_{1}n_{0}}{n^{2}}\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d})-\frac{n_{1}n_{0}}{n^{2}(n-1)}\sum_{i\neq j}(c_{i}-\bar{c})(d_{j}-\bar{d})\right].

Because

0=\sum_{i=1}^{n}(c_{i}-\bar{c})\sum_{i=1}^{n}(d_{i}-\bar{d})=\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d})+\sum_{i\neq j}(c_{i}-\bar{c})(d_{j}-\bar{d}),

the covariance of the sample means reduces to

\displaystyle\textup{cov}(\hat{\bar{c}},\hat{\bar{d}})
\displaystyle= \displaystyle\frac{1}{n_{1}^{2}}\left[\frac{n_{1}n_{0}}{n^{2}}\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d})+\frac{n_{1}n_{0}}{n^{2}(n-1)}\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d})\right]
\displaystyle= \displaystyle\frac{n_{0}}{nn_{1}}S_{cd}.

The variance formulas are just special cases with \hat{\bar{c}}=\hat{\bar{d}}. \square

Proof of Lemma 32.3: We prove only the sample covariance term because the formulas for sample variances are special cases. We have the following decomposition:

\displaystyle(n_{1}-1)\hat{S}_{cd} \displaystyle= \displaystyle\sum_{i=1}^{n}Z_{i}(c_{i}-\hat{\bar{c}})(d_{i}-\hat{\bar{d}})
\displaystyle= \displaystyle\sum_{i=1}^{n}Z_{i}\{(c_{i}-\bar{c})-(\hat{\bar{c}}-\bar{c})\}\{(d_{i}-\bar{d})-(\hat{\bar{d}}-\bar{d})\}
\displaystyle= \displaystyle\sum_{i=1}^{n}Z_{i}(c_{i}-\bar{c})(d_{i}-\bar{d})-n_{1}(\hat{\bar{c}}-\bar{c})(\hat{\bar{d}}-\bar{d}).

Taking expectations on both sides, we have

\displaystyle E\{(n_{1}-1)\hat{S}_{cd}\} \displaystyle= \displaystyle\sum_{i=1}^{n}E(Z_{i})(c_{i}-\bar{c})(d_{i}-\bar{d})-n_{1}E\{(\hat{\bar{c}}-\bar{c})(\hat{\bar{d}}-\bar{d})\}
\displaystyle= \displaystyle\frac{n_{1}}{n}\sum_{i=1}^{n}(c_{i}-\bar{c})(d_{i}-\bar{d})-n_{1}\frac{n_{0}}{nn_{1}}S_{cd}
\displaystyle= \displaystyle S_{cd}\left\{\frac{n_{1}(n-1)}{n}-\frac{n_{0}}{n}\right\}
\displaystyle= \displaystyle(n_{1}-1)S_{cd},

and the conclusion follows by dividing both sides by n_{1}-1. \square

Proof of Lemma 32.4: Hájek, (1960) gave a proof of the CLT for simple random sampling, and Lehmann, (1975) gave a more accessible version of the proof. Li and Ding, (2017) modified the CLT as presented in Lemma 32.4, and proved the consistency of the sample variance. Due to the technical complexities, I omit the proof. \square

Appendix 32.C Comments on the literature

Survey sampling and experimental design have been deeply connected ever since Neyman, (1934); Neyman, (1935)’s seminal work. Li and Ding, (2017) and Mukerjee et al., (2018) made many theoretical ties between these two areas.

Appendix 32.D Homework Problems

AF.1 Sampling without replacement and the Hypergeometric distribution

Consider a special case of Lemma 32.2 with c_{i}’s being binary. Assume the total number of 1’s equals T so the total number of 0’s equals n-T. Let t=\sum_{i=1}^{n}Z_{i}c_{i} denote the total number of 1’s in the sample of size n_{1}.

Find the distribution, mean, and variance of t.

Remark: t follows a Hypergeometric distribution.

AF.2 Vector form of Lemma 32.2

Assume the c_{i}’s are vectors and define

S_{c}^{2}=(n-1)^{-1}\sum_{i=1}^{n}(c_{i}-\bar{c})(c_{i}-\bar{c})^{\mkern-1.5mu\mathsf{T}},\quad\hat{S}_{c}^{2}=(n_{1}-1)^{-1}\sum_{i=1}^{n}Z_{i}(c_{i}-\hat{\bar{c}})(c_{i}-\hat{\bar{c}})^{\mkern-1.5mu\mathsf{T}}.

Show that

E(\hat{\bar{c}})=\bar{c},\quad\textup{cov}(\hat{\bar{c}})=\frac{n_{0}}{nn_{1}}S_{c}^{2},\quad E(\hat{S}_{c}^{2})=S_{c}^{2}.

References

本书:A First Course in Causal Inference,作者 Peng Ding。原文:arXiv:2305.18793v2,按 CC BY 4.0 许可发布。本站(yuanens.com)把 arXiv 网页版改成分章的纯 HTML,把公式渲染成图片,方便在 Kindle 等电子书阅读器上看,内容没有改动。转换难免有错,以原文为准。本站与作者和出版社无关。