Four Regressions

The goal of the bate package is to present some functions to compute quantiles of the empirical distribution of the bias-adjusted treatment effect (BATE) in a linear econometric model with omitted variables. To analyze such models, a researcher should consider four regression models: (a) a short regression model where the outcome variable is regressed on the treatment variable, with or without additional controls; (b) an intermediate regression model where additional control variables are added to the short regression; (c) a hypothetical long regression model, where an index of the omitted variable(s) is added to the intermediate regressions; and (d) an auxiliary regression where the treatment variable is regressed on all observed (and included) control variables.

As an example, suppose a researcher has estimated the following model, \[y = \alpha + \beta_1 x + \gamma_1 w_1 + \gamma_2 w_2 + \varepsilon,\] and is interested in understanding the impact of some omitted variables on the results. In this case:

• outcome variable: \(y\)
• treatment variable: \(x\)
• short regression: \(y\) regressed on \(x\);
• intermediate regression: \(y\) regressed on \(x, w_1, w_2\);
• auxiliary regression: \(x\) regressed on \(w_1, w_2\);
• hypothetical long regression: \(y\) regressed on \(x, w_1,w_2\) and the omitted variables;

In this example, the estimated treatment effect is \(\hat{\beta}_1\). In the presence of omitted variables, \(\hat{\beta}_1\) is a biased estimate of the true treatment effect, \(\beta_1\). The functions in this package will allow researchers to create quantiles of the empirical distribution of the BATE, i.e. the treatment effect once we have adjusted for the effect of omitted variable bias. We will denote the BATE as \(\beta^*\).

The researcher will need to supply the data set (as a data frame), the name of the outcome variable, the name of the treatment variable, and the names of the additional regressors in the intermediate regression. The functions in this package will then compute the quantiles of the empirical distribution of BATE, \(\beta^*\).

Two Important Parameters

Two parameters capture the effect of the omitted variables in this set up.

The first parameter is \(\delta\). This captures the relative strength of the unobservables, compared to the observable controls, in explaining variation in the treatement variable. In the functions below this is denoted as the parameter delta. This parameter is a real number and can take any value on the real line, i.e. it is unbounded. Hence, in any specific analysis, the researcher will have to choose a lower and an upper bound for delta. For instance, if in any empirical analysis, the researcher believes, based on knowledge of the specific problem being investigated, that the unobservables are less important than the observed controls in explaining the variation in the treatment variable, then she could choose delta to lie between 0 and 1. On the other hand, if she believes that the unobservables are more important than the observed controls in explaining the variation in the treatment variable, then she should choose delta to lie between 1 and 2 or 1 and 5.

The second parameter is \(R_{max}\). This captures the relative strength of the unobservables, compared to the observable controls, in explaining variation in the outcome variable. In the functions below, this is captured by the parameter Rmax. The parameter Rmax is the R-squared in the hypothetical long regression. Hence, it lies between the R-squared in the intermediate regression (\(\tilde{R}\)) and 1. Since the lower bound of Rmax is given by \(\tilde{R}\), in any specific analysis, the researcher will only have to choose an upper bound for Rmax.

In a specific empirical analysis, a researcher will use domain knowledge about the specific issue under investigation to determine a plausible range for delta (e.g. \(0.01 \leq \delta \leq 0.99\)). This will be given by the interval on the real line lying between deltalow and deltahigh (the researcher will choose deltalow and deltahigh). Using the example in this paragraph, deltalow=0.01 and deltahigh=0.99.

In a similar manner, a researcher will use domain knowledge about the specific issue under investigation to determine Rmax. Here, it will be important to keep in mind that Rmax is the R-squared in the hypothetical long regression. Now, it is unlikely that including all omitted variables and thereby estimating the hypothetical long regression will give an R-squared of 1. This is because, even after all the regressors have been included, some variation of the outcome might be plausibly explained by a stochastic element. Hence, Rmax will most likely be different from, and less than, 1. This will be denoted by Rhigh (e.g. Rmax=0.61).

The Algorithm

How is the omitted variable bias and the BATE computed? The key result that is used to compute the BATE is this: the omitted variable bias is the real root of the following cubic equation \[a \nu^3 + b \nu^2 + c \nu + d =0,\] where

• \(a = (\delta -1)(\tau_X \sigma_X^2 - \tau_X^2) \neq 0,\)
• \(b = \tau_X \left( \mathring{\beta} - \tilde{\beta}\right) \sigma^2_X \left( \delta - 2 \right),\)
• \(c = \delta \left( R_{max} - \tilde{R} \right) \sigma^2_Y \left( \sigma^2_X - \tau_X \right) - \left( \tilde{R} - \mathring{R} \right) \sigma^2_Y \tau_X - \sigma^2_X \tau_X \left( \mathring{\beta} - \tilde{\beta}\right)^2,\) and
• \(d = \delta \left( R_{max} - \tilde{R} \right) \sigma^2_Y \left( \mathring{\beta} - \tilde{\beta}\right) \sigma^2_X,\)

where, in turn,

• \(\sigma_Y^2\) is the variance of the outcome variable,
• \(\sigma_X^2\) is the variance of the treatment variable,
• \(\mathring{\beta}\) is the treatment effect in the short regression,
• \(\mathring{R}\) is the R-squared in the short regression,
• \(\tilde{\beta}\) is the treatment effect in the intermediate regression,
• \(\tilde{R}\) is the R-squared in the intermediate regression, and
• \(\tau_X^2\) is the variance of the residual in the auxiliary regression.

Hence, we see that the coefficients of the cubic equation are functions of the variances of the outcome and treatment variables (\(\sigma_Y^2, \sigma_X^2\)), parameters of the short regression (\(\mathring{\beta}, \mathring{R}\)), intermediate regression (\(\tilde{\beta}, \tilde{R}\)) and auxiliary regression (\(\tau_X^2\)), and the values of delta and Rmax. The important result is that the omitted variable bias is the real root of the above cubic equation (Proposition 2, Oster, 2019; Proposition 1 and 2 in Basu, 2022).

In a specific empirical analysis, the variances of the outcome and treatment variables, and the parameters of the short, intermediate and auxiliary regressions are known. Hence, the coefficients of the cubic equation become functions of delta and Rmax, the two key parameters that the researcher chooses, using domain knowledge.

Once the researcher has chosen deltalow, deltahigh and Rhigh, this defines a bounded box on the (delta, Rmax) plane defined by the Cartesian product of the interval [deltalow, deltahigh] and of the interval [Rlow, Rhigh]. The main functions in this package computes the root of the cubic equation on a sufficiently granular grid (the degree of granularity will be chosen by the user) covering the bounded box.

To compute the root of the cubic equation, the algorithm first evaluates the discriminant of the cubic equation on each point of the grid and partitions the box into two regions: (a) unique real root (URR) and NURR (no unique real root). There are three cases to consider.

• Case 1: If all points of the bounded box are in URR, then the algorithm chooses the unique real root of the cubic at each point as the estimate of the omitted variable bias.
• Case 2: If some non-empty part of the box is in NURR, then the algorithm first computes roots on the URR region, and then, starting from the boundary points of URR/NURR, covers points on the NURR in small steps. At each step, the algorithm chooses the real root at a grid point in the NURR that is closest in absolute value to the real root at a previously selected grid point. Continuity of the roots of a polynomial with respect to its coefficients guarantees that the algorithm selects the correct real root at each point.
• Case 3: If the bounded box is completely contained in NURR, then the algorithm extends the size of the box in small steps in the delta direction to generate a nonempty intersection with a URR region. Once that is found, the algorithm implements the steps outlined in step 2.

The bias is then used to compute the BATE, \(\beta^*\), which is defined as the estimated treatment effect in the intermediate regression minus the bias, i.e. \[\beta^* = \tilde{\beta}-\nu,\] where \(\beta^*\) is the bias-adjusted treatment effect, \(\tilde{\beta}\) is the treatment effect estimated from the intermediate regression and \(\nu\) is the real root of the relevant cubic equation.

The functions in this package will compute \(\beta^*\) at each point of the grid that covers the bounded box chosen by the researcher. Hence, this will generate a large vector of values of \(\beta^*\) and we can use this to compute the empirical distribution of \(\beta^*\), the BATE. Asymptotic theory shows that the BATE converges in probability to the true treatment effect, i.e. \[\beta^* \overset{p}{\to} \beta.\] Hence, the interval defined by the 2.5-th and 97.5-th quantiles of the empirical distribution of the BATE will contain the true treatment effect with 95 percent probability.

The Main Functions

This package provides three functions to compute quantiles of the empirical distribution of the omitted bias and BATE, ovbias(), ovbias_lm() and ovbias_par(). These functions implement the same algorithm to compute the empirical distribution of the bias and BATE and only differ in how the user provides the parameters of the short, intermediate and auxiliary regressions. But before we discuss the main functions, let us look at two other functions that are useful.

An useful function to collect relevant parameters from the short, intermediate and auxiliary regressions is:

• collect_par(): collects parameters from the short, intermediate and auxiliary regressions; (user provides name of the data set, name of outcome variable, name of treatment variable, names of control variables in the short regression, if relevant, and names of additional variables in the intermediate regression); the output of this function is a data frame.

Users can use the output from collect_par() to construct an area plot of the bounded box using:

• urrplot(): creates a colored area plot of the bounded box chosen by the user demarcating the area where the cubic equation has unique real root (URR) from the area where the cubic equation has three real roots (NURR); the output is a plot object.

Example: Impact of Maternal Behavior on Child IQ

Install the package from CRAN and then load it.

library(bate)

data("NLSY_IQ")

names(NLSY_IQ)
#>  [1] "iq_std"             "BF_months"          "mom_drink_preg_all"
#>  [4] "lbw_preterm"        "age"                "female"            
#>  [7] "black"              "motherAge"          "motherEDU"         
#> [10] "mom_married"        "income"             "sex"               
#> [13] "race"

NLSY_IQ$age <- factor(NLSY_IQ$age)
NLSY_IQ$race <- factor(NLSY_IQ$race)

library(vtable)
#> Loading required package: kableExtra
vtable::st(NLSY_IQ)

parameters <- bate::collect_par(data=NLSY_IQ,
            outcome="iq_std",
            treatment="BF_months",
            control=c("age","sex","income","motherAge","motherEDU","mom_married","race"),
            other_regressors = c("sex","age"))

(parameters)
#>                beta0         R0  betatilde   Rtilde    sigmay   sigmax     taux
#> BF_months 0.04447926 0.04465201 0.01740748 0.255621 0.9900242 4.629618 18.99883

# Upper bound of Rmax
Rhigh <- 0.61
# Lower bound of delta
deltalow <- 0.01
# Upper bound of delta
deltahigh <- 0.99
# step size to construct grid
e <- 0.01

bate::urrplot(parameters = parameters, deltalow = deltalow, 
              deltahigh = deltahigh, Rlow = parameters$Rtilde,
              Rhigh = 0.61, e=0.01)

OVB <- bate::ovbias(
  parameters = parameters,
  deltalow=deltalow, 
  deltahigh=deltahigh,
  Rhigh=Rhigh, 
  e=e)

(OVB$bias_Distribution)
#>  2.5%    5%   50%   95% 97.5% 
#> 0.000 0.000 0.009 0.034 0.039

(OVB$bstar_Distribution)
#>   2.5%     5%    50%    95%  97.5% 
#> -0.021 -0.017  0.009  0.017  0.017

cplotbias(OVB$Data)

dplotbate(OVB$Data)

bate::osterbds(parameters = parameters, Rmax=0.61)
#>       Discriminant          Interval1          Interval2 
#> "20.3210250174802"  "[0.0174,0.3752]" "[-0.0337,0.0174]"

bate::osterdelstar(parameters = parameters, Rmax=0.61)
#>           deltastar       discontinuity               slope 
#> "0.366194618840608"             "FALSE"          "Negative"

bate::delfplot(parameters = parameters)

OVB.par <- ovbias_par(
  data=NLSY_IQ,
  outcome="iq_std",
  treatment="BF_months",
  control=c("age","sex","income","motherAge","motherEDU","mom_married","race"),
  other_regressors = c("sex","age"),
  deltalow=deltalow, 
  deltahigh=deltahigh,
  Rhigh=Rhigh, 
  e=e)

(OVB.par$bias_Distribution)
#>  2.5%    5%   50%   95% 97.5% 
#> 0.000 0.000 0.009 0.034 0.039

(OVB.par$bstar_Distribution)
#>   2.5%     5%    50%    95%  97.5% 
#> -0.021 -0.017  0.009  0.017  0.017

reg_col1 <- lm(
  iq_std ~ BF_months + factor(age) + sex,
  data = NLSY_IQ
)

reg_col2 <- lm(
  iq_std ~ BF_months + factor(age) + sex +
    income + motherAge + motherEDU + mom_married +
    factor(race),
  data = NLSY_IQ
)

reg_aux <- lm(
  BF_months ~ factor(age) + sex +
    income + motherAge + motherEDU + mom_married +
    factor(race),
  data = NLSY_IQ
)

OVB.lm <- ovbias_lm(
  lm_shrt = reg_col1,
  lm_int = reg_col2,
  lm_aux = reg_aux,
  deltalow=deltalow, 
  deltahigh=deltahigh,
  Rhigh=Rhigh, 
  e=e)

(OVB.lm$bias_Distribution)
#>  2.5%    5%   50%   95% 97.5% 
#> 0.000 0.000 0.009 0.034 0.039

(OVB.lm$bstar_Distribution)
#>   2.5%     5%    50%    95%  97.5% 
#> -0.021 -0.017  0.009  0.017  0.017

References

• Basu, D. (2022). “Bounds for Bias-Adjusted Treatment Effect in Linear Econometric Models.” <arXiv:2203.12431>

• Oster, E. (2019). “Unobservable Selection and Coefficient Stability: Theory and Evidence.” Journal of Business & Economic Statistics, 37:2, 187-204, https://doi.org/10.1080/07350015.2016.1227711