t-stat discrepancies for boottests with and without fixed effects

[s3alfisc]

Hi David,

There is a discrepancy in t-stats between wildboottestjlr and fwildclusterboot when fixed effects are used.

As I did not find the bug in R, I tried to figure out if it might be on the Julia side, and I believe this is the case. Note that this difference is in higher post-comma digits. p-values are not affected - likely because the differences are so small, but potentially also because the fedfadj = false argument might not work as intended?

I am working with the most recent dev version of WildBootTests.jl and Julia 1.7.0.

Here's a reproducible example:

using WildBootTests, CSV, DataFrames, GLM, Distributions, Random

d = download("https://raw.github.com/vincentarelbundock/Rdatasets/master/csv/sandwich/PetersenCL.csv");
df = CSV.read(d, DataFrame);
clustid = df.firm;
df.dummy = Distributions.sample([0, 1], 5000)

f1 = @formula(y ~ 1+ x + year + dummy);  # state OLS model
f1 = apply_schema(f1, schema(f1, df));  # link model to data
lm(f1, df) 
resp1, predexog1 = modelcols(f1, df);  # extract response & (exogenous) predictor variables
unique(df.year)
R = [0 1 0 0]; r = [1];
test1 = wildboottest(R, r; resp=resp1, predexog=predexog1, clustid=clustid, rng = MersenneTwister(9615128512))

f2 = @formula(y ~ -1 + year + x );  # state OLS model, fe provided as fe
f2 = apply_schema(f2, schema(f2, df));  # link model to data
lm(f2, df) 
resp2, predexog2 = modelcols(f2, df);  # extract response & (exogenous) predictor variables
R = [0 1]; r = [1];
test2 = wildboottest(R, r; resp=resp2, predexog=predexog2, clustid=clustid, feid = df.dummy, fedfadj = false,  rng = MersenneTwister(9615128512))

test1
test2

teststat(test1) # 0.6906846f0
teststat(test2) # 0.6908233f0

In fwildclusterboot and boottest, these test stats will be identical.

As a sidepoint: When you feed in e.g. R = [1] for a one-covariate, no intercept model, wildboottest() throws an error (I think fwildclusterboot fails as well).

f3 = @formula(y ~  -1 + x );  # state OLS model, fe provided as fe
f3 = apply_schema(f3, schema(f3, df));  # link model to data
lm(f3, df) 
resp3, predexog3 = modelcols(f3, df);  # extract response & (exogenous) predictor variables
R = [1]; r = [1];
test3 = wildboottest(R, r; resp=resp2, predexog=predexog2, clustid=clustid, feid = df.dummy, fedfadj = false,  rng = MersenneTwister(9615128512))
# LoadError: MethodError: no method matching _wildboottest

[droodman]

Hi Alexander. The default for the feadfadj option is true, so doesn't it make sense that putting in fedfadj=false would slightly change the result? I got a near-perfect match when dropping the fedfadj option. The disagreement was just in the last digit. Going to Float64 resulted in the same, at higher precision:

julia> teststat(wildboottest(Float64, [0 1 0 0], [1]; resp=resp1, predexog=predexog1, clustid, rng = MersenneTwister(9615128512)))
0.6924735748472406

julia> teststat(wildboottest(Float64, [0 1], [1]; resp=resp2, predexog=predexog2, clustid, feid = df.dummy, rng = MersenneTwister(9615128512)))
0.6924735748472409

julia> teststat(wildboottest(Float64, [0 1], [1]; resp=resp2, predexog=predexog2, clustid, feid = df.dummy, fedfadj=false, rng = MersenneTwister(9615128512)))
0.6926121665778084

The crash when R=[1] arises because [1] is a vector and the function requires a matrix, as documented.... but I agree it would be nice if it handled this more gracefully, so I'll try to make it do that.

[droodman]

Done, and committed.

[s3alfisc]

Great, looks like it was a false alarm!