Skip to content

3.0.0-palantir.59
Compare
Choose a tag to compare
@lwwmanning lwwmanning released this 10 Jan 11:35
· 4212 commits to master since this release
3.0.0-palantir.59
d50bdf7
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Cherry picks (#631) of the following commits from upstream:

[SPARK-27291][SQL] PartitioningAwareFileIndex: Filter out empty files on listing files
[SPARK-27627][SQL] Make option "pathGlobFilter" as a general option for all file sources
[SPARK-27990][SQL][ML] Provide a way to recursively load data from datasource
[SPARK-25348][SQL] Data source for binary files
[SPARK-27473][SQL] Support filter push down for status fields in binary file data source
[SPARK-27534][SQL] Do not load content column in binary data source if it is not selected
[SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
[SPARK-28030][SQL] convert filePath to URI in binary file data source

Assets 2
Loading