Trimming and overlap

With covariates, method(dr) and method(ipw) weight comparison units by a propensity score (the estimated probability of belonging to the treated cohort). A unit with a score near 1 gets an enormous weight, and a handful of such units can dominate the estimate and inflate its variance at the same time. That is why the problem is easy to miss in a table of point estimates.

That is an overlap problem: the data contain treated units with nothing comparable to compare them against. csdid guards against it in two ways, a warning and a trimming rule, both visible in the output.

The data

Let’s build the sample once and vary only the trimming threshold (everything else about the specification is held fixed below).

import delimited using ///
    "https://raw.githubusercontent.com/pedrohcgs/JEL-DiD/50f4f18/data/county_mortality_data.csv", ///
    clear varnames(1) bindquote(strict) stringcols(_all)
destring deaths population_20_64 year yaca county_code stfips unemp_rate poverty_rate, ///
    replace force
generate double mrate = 100000 * deaths / population_20_64
drop if missing(mrate) | population_20_64 <= 0
generate int gvar = yaca
replace gvar = 0 if missing(gvar) | gvar > 2016
bysort county_code: generate byte nyears = _N
keep if nyears == 11
save "jel_overlap.dta", replace

The overlap warning

Every dr and ipw cell is checked, cell by cell rather than once for the sample as a whole. If the fitted propensity scores get too close to 1, csdid warns for that cohort–period cell and keeps going. You learn which particular comparison is fragile, and it is not averaged silently into the cohort total:

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) analytical
estat event

No warning here means overlap held in every cell for these covariates. This is good news! However, add enough covariates and it will not hold. The more you condition on, the easier it becomes to predict cohort membership perfectly.

Trimming

pscoretrim() caps how extreme a propensity score may get, and the default is a deliberately mild .995:

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) analytical
display "trim: " e(pscoretrim)
estat simple

Tighten it to see how sensitive the estimate is to the most extreme weights (a cheap robustness check):

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(0.95) analytical
display "trim: " e(pscoretrim)
estat simple

Turn it off with pscoretrim(1), which permits any score (including scores that ought to worry you):

use "jel_overlap.dta", clear
csdid mrate unemp_rate poverty_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(1) analytical
estat simple

A value of zero or below is refused (the run stops), since it would trim away every observation in the sample:

use "jel_overlap.dta", clear
capture noisily csdid mrate unemp_rate, ivar(county_code) time(year) gvar(gvar) ///
    method(dr) pscoretrim(0) analytical
display "return code: " _rc

Reading the sensitivity

If the estimate moves a lot between pscoretrim(1) and a tighter bound, then a small number of extreme-weight units are driving the result. We would report that. It tells a reader that the estimate rests on units with very few counterparts in the comparison group, and that a different trimming rule would have given a different answer.

Trimming changes the estimand slightly. It reweights toward the region of common support, the region where comparison units actually exist. That is usually preferable to an estimate dominated by units with no real comparison. It is still a choice, so state the value you used.

Avoiding the problem

It is easier to keep overlap at the design stage than to repair it afterwards, so we think about it when choosing the specification.

Fewer covariates. Every covariate makes cohort membership easier to predict (that is what a propensity score is for), so include what parallel trends plausibly needs, which is usually fewer covariates than the dataset happens to contain.
Outcome regression. We prefer method(reg) when overlap is genuinely poor. It does not weight by a propensity score, so it cannot blow up on extreme scores, and it relies instead on the outcome model being right.
A larger comparison pool. notyet makes extreme scores less likely (it is the default, as it happens), simply because there are more units available to compare against. See Comparison groups.

method(dr), the default, is doubly robust: it is consistent if either the outcome model or the propensity model is correct. That is why we made it the default, and it is the reason overlap matters less here than it does for plain ipw. Double robustness still needs overlap, though, and we would read the warnings before trusting any individual cell. See Covariates and estimators.

capture erase "jel_overlap.dta"