[SPARK-41096][SQL] Support reading parquet FIXED_LEN_BYTE_ARRAY type

[kazuyukitanimura]

What changes were proposed in this pull request?

Parquet supports FIXED_LEN_BYTE_ARRAY (FLBA) data type. However, Spark Parquet reader currently cannot handle FLBA.
This PR proposes to read FLBA column as BinaryType data in Spark.

Why are the changes needed?

Iceberg Parquet reader, for example, can handle FLBA. This PR reduces the implementation gap between Spark and Iceberg Parquet reader.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit test added

[kazuyukitanimura]

cc @huaxingao @sunchao @viirya

[huaxingao]

@kazuyukitanimura Thanks for working on this!

I took a look at how Iceberg handles FLBA. For iceberg type Types.FixedType, the underneath Parquet type is fixed_len_byte_array. In Scan.readSchema, Iceberg map Types.FixedType to Spark BinaryType. Since FLBA is a valid parquet data type, it makes sense to me for Spark support this type and map it to BinaryType.

[sunchao]

LGTM. Even though Spark itself won't write binary using FLBA, it's good for compatibility against Parquet files written by 3rd party tools. This is similar to the other cases like SPARK-34816.

[sunchao]

Committed to master, thanks @kazuyukitanimura !

[kazuyukitanimura]

Thank you @huaxingao @sunchao @LuciferYang

[viirya]

Looks good to me.

[viirya]

Just found a previous PR #35902. The change is the same, but there are some avro test stuff that we can consider to add as a followup too.

[kazuyukitanimura]

Thanks @viirya I also realized PR #35902 along with #20826 and #1737 after I submit this PR.
I believe all attempts were not merged due to missing Parquet test as pointed here #20826 (comment)
This PR has a dedicated Parquet test and merged.

The avro compatibility tests are nice to have. Wondering if the previous authors are still interested to work on. @ghost @aws-awinstan @nicolaslrveiga