Consider using SXSSWorkbook for reading xlsx files

[koperagen]

xssfworkbook keeps whole file in memory, which for even relatively small dataframes or 100k rows can lead to a very significant memory consumption. sxssfworkbook implements streaming writing and flushes rows on disk as you write them. For comparison, max heap writing dataFrameOf('a'..'z').fill(1_000_000) { it }:
xssfworkbook 15 gb
sxssfworkbook 1.6 gb

The only concern is some potential incompatibilities mentioned in javadoc here:
https://poi.apache.org/apidocs/dev/org/apache/poi/xssf/streaming/SXSSFWorkbook.html

Other than that, from user perspective df.writeExcel(file) will benefit from much smaller memory consumption without lose of functionality:
https://poi.apache.org/components/spreadsheet/
https://stackoverflow.com/questions/33047512/hssfworkbook-vs-xssfworkbook-vs-sxssfworkbook-apache-poi

Check some other JVM DF libraries, do they use it?

[koperagen]

https://github.com/dhatim/fastexcel

Its streaming API may mitigate this problem but it introduces several limitations:

Its sliding window mechanism prevents you from accessing cells above the current writing position.
It writes stuff to a temporary file.
It comes with an overhead on the file size because shared strings are disabled by default. Enabling shared strings is likely to consume much more heap if you deal with string values.