Error of crossfit folds splits with DynamicDML

[juandavidgutier]

Hi,

I am estimating the effect of high levels of particulate matter (PM2.5) on excess deaths from panel data for 25 municipalities with daily resolution. It means my treatment is a binary variable where T=1, when the level of PM2.5 is high, and T=0, when the level of PM2.5 is low. The outcome is also a binary variable, where Y=0 for non-excess deaths, and Y=1 for excess deaths.

I am using the class DynamicDML to fit my model, but I get this error message: "AttributeError: Provided crossfit folds contain training splits that don't contain all treatments". But, 50% of the data corresponds to observations with T=1, I think it is enough to obtain balanced crossfit folds.

Here is my code with econml version 0.15 and dowhy version 0.10.1
dataset_pm_deaths.csv

`
import dowhy
import econml
from dowhy import CausalModel
import pandas as pd
import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LassoCV
import scipy.stats as stats
from itertools import product
from econml.utilities import WeightedModelWrapper
from sklearn.model_selection import train_test_split
from econml.panel.dml import DynamicDML

data_all = pd.read_csv("D:/dataset_pm_deaths.csv")
data = data_all[data_all['Year'] >= 2009]

median_pm25 = data['PM25'].median()
data['PM25'] = (data['PM25'] >= median_pm25).astype(int)

data.BC = stats.zscore(data.BC, nan_policy='omit')
data.DMS = stats.zscore(data.DMS, nan_policy='omit')
data.PM = stats.zscore(data.PM, nan_policy='omit')
data.OC = stats.zscore(data.OC, nan_policy='omit')
data.SO2 = stats.zscore(data.SO2, nan_policy='omit')
data.SO4 = stats.zscore(data.SO4, nan_policy='omit')

data0 = data[['excess', 'PM25', 'cod_munici',
'BC', 'DMS', 'PM', 'OC', 'SO2', 'SO4', 'Temperature', 'lead1_PM25']]
data0 = data0.dropna()
Y = data0.excess.to_numpy()
T = data0.PM25.to_numpy()
percentage_high_PM25 = np.mean(T == 1) * 100
W = data0[['BC', 'DMS', 'PM', 'OC', 'SO2', 'SO4', 'Temperature']].to_numpy().reshape(-1, 7)
X = data0[['Temperature', 'lead1_PM25']].to_numpy().reshape(-1, 2)
groups = data0.cod_munici.to_numpy()

estimate0 = DynamicDML(discrete_treatment=True,
featurizer=PolynomialFeatures(degree=3),
linear_first_stages=False, cv=3, random_state=123)
estimate0.fit(Y=Y, T=T, X=X, W=W, inference='auto', groups=groups) # HERE IS THE ERROR
`

[TimCosemans]

Have you tried passing a StratifiedKFold-object or creating your own cv-splitter? That could help you out in the meantime

[juandavidgutier]

Hi @TimCosemans

Thanks for your suggestions!