Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to the what)
@matthewmturner is working on adding datafusion to the h2oai database benchmarks. Some of the csv data uses scientific notation which the current schema inference does not support. apache/datafusion#1488
Describe the solution you'd like
A clear and concise description of what you want to happen.
Extend floating point schema inference to recognize numbers in valid scientific notation. This only requires updating the DECIMAL_RE regex that is used during schema inference as lexical_core already supports parsing numbers in this form.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Implementing alternative csv schema inference in datafusion.