[AMORO-2407] Fix access data file from dashboard of non-partitioned table

[liaoyt]

Why are the changes needed?

Close #2407

Brief change log

• convert string value "null" to null

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? no

[zhoujinsong]

LGTM.

Thanks a lot for your contribution!

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (cbada32) 53.51% compared to head (3156493) 53.44%.

Files	Patch %	Lines
...rver/dashboard/MixedAndIcebergTableDescriptor.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #2408      +/-   ##
============================================
- Coverage     53.51%   53.44%   -0.07%     
+ Complexity     4358     4356       -2     
============================================
  Files           517      517              
  Lines         29841    29841              
  Branches       2905     2905              
============================================
- Hits          15969    15950      -19     
- Misses        12585    12604      +19     
  Partials       1287     1287              

Flag	Coverage Δ
core	53.93% <0.00%> (-0.08%)	⬇️
trino	50.87% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[zhoujinsong]

After I review the code of MixedAndIcebergTableDescriptor#collectFileInfo, I found a possibly better way to fix the issue.
We can change the conditions here(https://github.com/NetEase/amoro/blob/master/ams/server/src/main/java/com/netease/arctic/server/dashboard/MixedAndIcebergTableDescriptor.java#L502) like this:

if (table.spec().isPartitioned() && partition != null && specId != null) {
  GenericRecord partitionData = ArcticDataFiles.data(specs.get(specId), partition);
  manifestReader.inPartitions(specs.get(specId), partitionData);
}

Then we won't confuse non-partitioned tables with partitioned values that are null.
HDYT? @liaoyt

[wangtaohz]

I'm a little worried if Paimon has the same issue, cc @shidayang.

[liaoyt]

After I review the code of MixedAndIcebergTableDescriptor#collectFileInfo, I found a possibly better way to fix the issue. We can change the conditions here(https://github.com/NetEase/amoro/blob/master/ams/server/src/main/java/com/netease/arctic/server/dashboard/MixedAndIcebergTableDescriptor.java#L502) like this:

if (table.spec().isPartitioned() && partition != null && specId != null) {
  GenericRecord partitionData = ArcticDataFiles.data(specs.get(specId), partition);
  manifestReader.inPartitions(specs.get(specId), partitionData);
}

Then we won't confuse non-partitioned tables with partitioned values that are null. HDYT? @liaoyt

it looks more reasonable

[wangtaohz]

LGTM

[shidayang]

I'm a little worried if Paimon has the same issue, cc @shidayang.

Paimon doesn't have this issue. However, Paimon's partition filtering method needs optimization.

[shidayang]

+1