Skip to content

Core: Send Idempotency-Key on mutation requests when advertised#14740

Merged
huaxingao merged 4 commits intoapache:mainfrom
huaxingao:idempotency_3
Dec 5, 2025
Merged

Core: Send Idempotency-Key on mutation requests when advertised#14740
huaxingao merged 4 commits intoapache:mainfrom
huaxingao:idempotency_3

Conversation

@huaxingao
Copy link
Contributor

This is the 3rd PR for Idempotency Key.

  • Discover idempotency-key-lifetime in ConfigResponse; if present, use RESTUtil.idempotencyHeaders, else Map::of.
  • Attach headers to all POST/DELETE calls (tables/namespaces/views) and plumb into RESTTableOperations/RESTViewOperations.
  • Tests: in TestRESTCatalog, verify header present when advertised and absent otherwise (via server spy).
  • No server changes; E2E replay behavior comes in next PR.

First PR: #14649
SecondPR: #14700

Follow-ups (separate PRs)

  • E2E tests with test server idempotency layer.

@github-actions github-actions bot added the core label Dec 3, 2025
tableClient,
paths.table(ident),
Map::of,
mutationHeaders,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we missing the same change for L806?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed L806 to use mutationHeaders

tableClient,
paths.table(finalIdentifier),
Map::of,
mutationHeaders,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not adjusting new RESTMetricsReporter(restClient, metricsEndpoint, Map::of); is probably fine for now as well, since we didn't add the param to the Spec and because typically we don't care too much whether metrics actually arrive or not, since it's best-effort and we don't want to interfere with the read/write path and add additional latency

@nastra
Copy link
Contributor

nastra commented Dec 3, 2025

@huaxingao I think for RESTTableOps and RESTViewOps we accidentally send the new headers also for GET requests

@huaxingao
Copy link
Contributor Author

I think for RESTTableOps and RESTViewOps we accidentally send the new headers also for GET requests

@nastra I split read vs mutation headers in both RESTTableOperations and new RESTViewOperations so GETs won’t carry Idempotency-Key anymore. I updated constructors and all call sites accordingly. Could you take one more look to see if it's OK?

Set<Endpoint> supportedEndpoints) {
return new RESTTableOperations(restClient, path, headers, fileIO, current, supportedEndpoints);
return new RESTTableOperations(
restClient, path, Map::of, headers, fileIO, current, supportedEndpoints);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we might want to add the non-mutating headers as params to newTableOps and newViewOps as well since those methods basically just mimic the parameters of the underlying constructor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion! Done!

Mockito.spy(
new RESTCatalogAdapter(backendCatalog) {
@Override
public <T extends RESTResponse> T handleRequest(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of handling this globally for all tests and introducing idempHeaderExpectation and advertiseIdempInConfig it's probably better to use a custom RESTCatalogAdapter for the 2 new tests that you're adding. One example where this is done is in verifyNamespaceExistsFallbackToGETRequest and we could use the same approach in the new tests as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Fixed!

@nastra nastra changed the title Core: REST client – send Idempotency-Key on mutation requests when advertised Core: Send Idempotency-Key on mutation requests when advertised Dec 5, 2025
@huaxingao huaxingao merged commit 7bac865 into apache:main Dec 5, 2025
44 checks passed
@huaxingao
Copy link
Contributor Author

Thanks @nastra a lot for your review!

@huaxingao huaxingao deleted the idempotency_3 branch December 5, 2025 17:24
thomaschow pushed a commit to thomaschow/iceberg that referenced this pull request Jan 19, 2026
…he#14740)

* Core: REST client – send Idempotency-Key on mutation requests when advertised

* rebase

* address comments

* address comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants