Before expiring snapshots is there need to provide history snapshot file statistics

[MichaelDeSteven]

Feature Request / Improvement

• context: when I want to keep the size of table small by expiring snapshots and delete orphan files, I find that there is no any way to estimate the files that can be deleted after the snapshot expiration before it occurs
• suggest: Perhaps we can expose the total size of the files to be deleted after snapshot expiration through SQL or procedure.

Query engine

Spark

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[manuzhang]

It might be valuable to add a dry_run option to expire_snapshots like remove_orphan_files.

[MichaelDeSteven]

It might be valuable to add a dry_run option to expire_snapshots like remove_orphan_files.

good idea! When dry_run is true, don't actually remove files but return info.

[ajreid21]

The ExpireSnaphots procedure does 2 things:

1. Removes old snapshots from table's metadata.
2. Deletes expired files (files no longer reachable after removing snapshots from table).

Would the expectation be that if you set dry_run to true that only #2 is simulated? IMO, I think that would be a different parameter such as cleanup_expired_files which would default to true and if false, the procedure wouldn't delete any files (and those files would now be considered orphan files that would/could be cleaned up by the orphan files procedure). But this probably wouldn't provide what you wanted as I don't think it would return simulated results for the deletes (the delete results would be 0).

I think if we added a dry_run parameter that if set to true, it wouldn't remove old snapshots, nor remove files, just return info on what would happen (wonder if we should also add # of expired snapshots to the result set 🤔 ).

I can try and add either or both of these, just wondering what people think.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'