Trimming and overlap

With covariates, method(dr) and method(ipw) weight comparison units by a propensity score (the estimated probability of belonging to the treated cohort). A unit with a score near 1 gets an enormous weight, and a handful of such units can dominate the estimate and inflate its variance at the same time. That is why the problem is easy to miss in a table of point estimates.

That is an overlap problem: the data contain treated units with nothing comparable to compare them against. csdid guards against it in two ways, a warning and a trimming rule, both visible in the output.

The data

Let’s build the sample once and vary only the trimming threshold (everything else about the specification is held fixed below).

import delimited using ///
    "https://raw.githubusercontent.com/pedrohcgs/JEL-DiD/50f4f18/data/county_mortality_data.csv", ///
    clear varnames(1) bindquote(strict) stringcols(_all)
destring deaths population_20_64 year yaca county_code stfips unemp_rate poverty_rate, ///
    replace force
generate double mrate = 100000 * deaths / population_20_64
drop if missing(mrate) | population_20_64 <= 0
generate int gvar = yaca
replace gvar = 0 if missing(gvar) | gvar > 2019
bysort county_code: generate byte nyears = _N
keep if nyears == 11
save "jel_overlap.dta", replace

The overlap warning

Every dr and ipw cell is checked, cell by cell rather than once for the sample as a whole. If the fitted propensity scores get too close to 1, csdid warns for that cohort–period cell and keeps going. You learn which particular comparison is fragile, and it is not averaged silently into the cohort total:

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) analytical
estat event

A warning identifies a fitted propensity problem; its absence does not prove population overlap. Inspect whether comparison observations cover the treated units’ covariate values. A richer specification can make cohort membership easier to predict and expose comparisons with little support.

Trimming

pscoretrim() drops comparison observations whose fitted score is at or above the threshold; it does not cap their scores. Treated observations are retained. The default threshold is .995:

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) analytical
display "trim: " e(pscoretrim)
estat simple

Tighten it to see how sensitive the estimate is to the most extreme weights (a cheap robustness check):

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(0.95) analytical
display "trim: " e(pscoretrim)
estat simple

Turn trimming off with pscoretrim(1). The separate overlap checks still apply, so this does not force a failed cell to be estimated:

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(1) analytical
estat simple

A value of zero or below is refused (the run stops), since it would trim away every observation in the sample:

use "jel_overlap.dta", clear
capture noisily csdid mrate unemp_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(0) analytical
display "return code: " _rc

Reading the sensitivity

If the estimate moves a lot between pscoretrim(1) and a tighter bound, then a small number of extreme-weight units are driving the result. We would report that. It tells a reader that the estimate rests on units with very few counterparts in the comparison group, and that a different trimming rule would have given a different answer.

Trimming removes comparison observations and can change the result substantially. It does not automatically identify an ATT for a new common-support population, because treated observations are retained. A binding threshold calls for investigating the design, not just reporting a smoother estimate. State the threshold and the comparisons affected.

Avoiding the problem

It is easier to keep overlap at the design stage than to repair it afterwards, so we think about it when choosing the specification.

Fewer covariates. Every covariate makes cohort membership easier to predict (that is what a propensity score is for), so include what parallel trends plausibly needs, which is usually fewer covariates than the dataset happens to contain.
Outcome regression. method(reg) avoids inverse propensity weights, but with poor overlap it extrapolates the outcome model beyond the comparison data. A finite regression estimate does not resolve absent support.
A larger comparison pool. notyet can expand the available comparison pool by using eligible later-treated cohorts. More observations do not guarantee better overlap; inspect support under the chosen specification. See Comparison groups.

method(dr), the default, is doubly robust: it is consistent if either the outcome model or the propensity model is correct. That is why we made it the default. Double robustness still needs overlap and does not protect against extreme weights or missing support. Read the warnings before trusting any individual cell. See Covariates and estimators.