Describe the bug, including details regarding any error messages, version, and platform.
Parquet version: From 1.14.0 to 1.15.2
There is a data issue after this improvement. The result is empty after this patch. My query is:
CREATE TEMPORARY VIEW parquetTable
USING org.apache.spark.sql.parquet
OPTIONS (
path "viewfs://cluster/path/to/file/00000-62-eff62af9-bea8-4eb6-b9c9-c3ee29fd795e-0-00001.parquet"
);
SELECT *
FROM parquetTable
WHERE JNL_DEDUP_ID = 'PG.PG_CHARGE_ID.25030465176'
AND ACCT_ID = 4920012120302 AND SEQ_NUM = 734343586;
We build bloom filter on the column of JNL_DEDUP_ID using Parquet 1.13.1(Read data using Parquet 1.15.2), and it's data type is string type.
Component(s)
Core