Nessie S3 remote signing endpoint not refreshed

[lwfitzgerald]

Apache Iceberg version

main (development)

Please describe the bug 🐞

When accessing Iceberg tables provided by an instance of the Nessie catalog, if remote S3 signing is enabled on Nessie, signing requests use a stale endpoint after 3 hours, causing S3 requests to fail.

Nessie supports S3 remote signing by setting a custom S3 signer endpoint via s3.signer.endpoint in the table configs it returns. As part of the s3.signer.endpoint it includes a HMAC signature with an expiry of 3 hours.

pyiceberg does not appear to refresh the s3.signer.endpoint (from the table config) after the first interaction with a table, meaning that after 3 hours signing requests to Nessie start failing.

This issue also exists in the Iceberg implementation for Spark - apache/iceberg#12602.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[Fokko]

Thanks for raising this @lwfitzgerald. When we refresh the token, we could also recreate a new FileIO, which does not look entirely unreasonable. Let's hear what comes out of apache/iceberg#12602