fix: stats is optional in add action

[jkylling]

Description

The stats field in add actions is optional, https://github.com/delta-io/delta/blob/master/PROTOCOL.md#add-file-and-remove-file

We currently silently skip reading add actions from snapshots without stats.

Related Issue(s)

Documentation

[codecov]

Codecov Report

Attention: Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 72.48%. Comparing base (1392ac3) to head (4d1a156).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/core/src/protocol/mod.rs	85.71%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2841      +/-   ##
==========================================
- Coverage   72.50%   72.48%   -0.02%     
==========================================
  Files         132      132              
  Lines       40049    40057       +8     
  Branches    40049    40057       +8     
==========================================
- Hits        29038    29036       -2     
  Misses       9159     9159              
- Partials     1852     1862      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rtyler]

@jkylling I am trying to add a test case to validate this scenario and cannot reproduce the issue since my Add actions with no stats are coming through. Lots of our tests have Add actions with no stats too 😕

[jkylling]

@jkylling I am trying to add a test case to validate this scenario and cannot reproduce the issue since my Add actions with no stats are coming through. Lots of our tests have Add actions with no stats too 😕

This code path seems to only be triggered when reading checkpoints without a stats column? It might be delta-rs always includes a stats column, even if all its values are null. I encountered this on a table written by Databricks Spark.

[jkylling]

@rtyler if we remove stats from the checkpoint schema the checkpoint_writer::simple_checkpoint_simple_checkpoint_test fails on main because some files are missing. Please see 6da8b6b

[ion-elgreco]

@jkylling can you add a test for this

[jkylling]

@rtyler @ion-elgreco I've added a test with a checkpoint without a stats column. Please have a look.