You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just got bit by trying to open a path that was a directory (I meant to pass a BAM); failure mode was not the most clear:
Exception in thread "Driver" java.io.IOException: Could not read footer: java.lang.RuntimeException: hdfs://demeter-nn1.demeter.hpc.mssm.edu:8020/user/willir31/data/set3/normal/set3.normal.fq/part-01487 is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [67, 68, 67, 10]
at parquet.hadoop.ParquetFileReader.readAllFootersInParallel(ParquetFileReader.java:238)
at parquet.hadoop.ParquetFileReader.readAllFootersInParallelUsingSummaryFiles(ParquetFileReader.java:179)
at parquet.hadoop.ParquetInputFormat.getFooters(ParquetInputFormat.java:389)
at parquet.hadoop.ParquetInputFormat.getFooters(ParquetInputFormat.java:361)
at parquet.hadoop.ParquetInputFormat.getSplits(ParquetInputFormat.java:245)
From digging around, #190 seems to imply that there is a decent amount of convention behind assuming ADAM parquet files will have an .adam extension; any interest in:
requiring presumed-ADAM files to have a .adam extension (throwing an Exception if the extension is not recognized)
printing a warning
Arun suggested: catching such an Exception and attempting to throw a new Exception with a better inferred message, e.g. inspecting whether the failing path is actually a directory and making this explicit in the re-thrown exception?
The text was updated successfully, but these errors were encountered:
Porting this over from guacamole #266:
I just got bit by trying to open a path that was a directory (I meant to pass a BAM); failure mode was not the most clear:
From digging around, #190 seems to imply that there is a decent amount of convention behind assuming ADAM parquet files will have an
.adam
extension; any interest in:.adam
extension (throwing an Exception if the extension is not recognized)The text was updated successfully, but these errors were encountered: