Loading Data in Chunks

[BPC-001]

Hello - I was wondering if anyone has experimented with loading data in chunks. The use case it that we'd like to run a backtest against 20 years worth of tickframe data, and therefore, were considering loading it in chunks.

Thanks in advance for any suggestions!

[samueltg92]

Hello!
I don't know if it helps, but I use this:

60% of the dataset

df60 = df[0:270000]

40% of the dataset

df40 = df[270000:449192]
df60.dropna()
df40.dropna()