feat: add finalized metadata field to HTTP API responses
#6645
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nflaig
changed the title
finalized metadata field to HTTP API responses
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## unstable #6645 +/- ##
============================================
+ Coverage 61.69% 61.77% +0.08%
============================================
Files 556 556
Lines 58820 58946 +126
Branches 1887 1889 +2
============================================
+ Hits 36287 36413 +126
Misses 22492 22492
Partials 41 41 |
Performance Report✔️ no performance regression detected Full benchmark results
|
ensi321
reviewed
Apr 6, 2024
twoeths
reviewed
Apr 8, 2024
twoeths
reviewed
Apr 8, 2024
twoeths
reviewed
Apr 8, 2024
ensi321
reviewed
Apr 9, 2024
| @@ -256,6 +256,8 @@ export function getBeaconBlockApi({ | |||
|
|
|||
| // If one block in the response contains an optimistic block, mark the entire response as optimistic | |||
| let executionOptimistic = false; | |||
| // If one block in the response is non finalized, mark the entire response as unfinalized | |||
There was a problem hiding this comment.
This behaviour is not clearly defined in the beacon api spec. Might be a good idea to update Finalized description in the spec for the case the endpoint is returning multiple blocks
There was a problem hiding this comment.
This is the only place where it is relevant, I don't even know why this API has top level metadata fields for finalized and execution_optimisitic...
Those should be part of header response and per block
lodestar/packages/beacon-node/src/api/impl/beacon/blocks/utils.ts
Lines 16 to 21 in 7fadd30
There was a problem hiding this comment.
We could consider proposing this in beacon api spec as current behavior does not make much sense, but would likely require to bump the api anyways
ensi321
reviewed
Apr 9, 2024
| [HttpStatusCode.OK]: { | ||
| data: deneb.BlobSidecars; | ||
| executionOptimistic: ExecutionOptimistic; | ||
| finalized: Finalized; |
There was a problem hiding this comment.
Hmmm I don't think we need executionOptimistic or finalized for this endpoint: https://ethereum.github.io/beacon-APIs/#/Beacon/getBlobSidecars
There was a problem hiding this comment.
Yeah it does not, but we already had execution_optimistic field here and I think it should?
I have opened an issue for this a few days ago on the spec
There was a problem hiding this comment.
As this API is not used by other clients but only users that want to retrieve chain data I would prefer to keep the metadata, unless someone has a good argument why it shouldn't be there
There was a problem hiding this comment.
yeah I think it's better to have it as it's still compliant to the spec
ensi321
approved these changes
Apr 11, 2024
nflaig
added a commit
that referenced
this pull request
Apr 11, 2024
* feat: add finalized metadata field to HTTP API responses * Clean up getState type casts * Fix finalized when getting block root * Add comment for block from hot db * Fix case where slot equals finalized block slot * Calculate epoch from unmodded slot
|
🎉 This PR is included in v1.18.0 🎉 |
nflaig
added a commit
that referenced
this pull request
Apr 27, 2024
wemeetagain
added a commit
that referenced
this pull request
Jun 10, 2024
* Add config route definitions * Add debug route definitions * Add events route description * Add lightclient route definitions * Flatten function params * Type safety for optional params in write / parse req * Method args are optional if only optional props * Fix genesisValidatorsRoot type issue * Revert requiring all params in write / parse req * Update http client errors * Add lodestar route definitions * Add node route definitions * Add proof route definitions * Add builder route definitions * Add validator route definitions * Application method response can be void * Generic options can be passed to application methods * Default endpoint request type has body property * Improve types of transform methods * Export server types from index (to be removed) * Update config api impl * Update lightclient api impl * Update events api impl * Update lodestar api impl * Update proof api impl * Update node api impl * Update debug api impl * Update state api impl * Update pool api impl * Update blocks api impl * Partially update validator api impl * Update beacon routes export * Align submitPoolBlsToExecutionChange method args * Filters are always a object * Update errors messages * Add beacon client methods * Add missing routeId label to stream time metric * Fix json casing in codecs * Apply remaining changes from #6227 * Produce block apis only have version meta * Add block values meta to all produce block apis * Apply changes from #6337 * Handle unsafe version in WithMeta and WithVersion * Restore server api error * Update fastify route types * Update server routes / handlers * Remove unnecessary type cast * Restore per route clients * Fix beacon route types * Remove option to patch fetch from http client * Update eventstream client, remove fetch override Fallback does not work like this, see #6180 for proper solution * Use StringType for validator status until #6059 * Remove empty fetch.ts file * Add a few todos * Update builder client and server routes * Update beacon exports * Update api index exports * Update builder index imports * Improve type safety of schema definitions * Add headers to fastify schema * Fix schema definition type * Add missing schemas to route definitions * Fix response codec type * Remove response codec type casts * Fix casing in json only codec * Reuse EmptyResponseCodec * Update base rest api server * Update keymanager routes, client and server * Reuse data types in keymanager impl * Do not await setting headers, not a promise * Improve type safety of empty codecs * Only require to implement supported req methods * Handle requests that only support one format * Handle responses that only support one format * Add json / ssz only req codecs * Update only support errors * Fix assertion * Set correct accept header if only supports one format * Fix eslint / prettier issues * More formatting fixes * Fix fallback request retries in case of http errors * Formatting of res.error * Add add retry functionality to http client (from #6387) * Update rewards routes and server (#6178 and #6260) * Allow to omit body in ssz req if not defined * Always set metadata headers in response * Cache wire format in api response * Only call raw body for json meta * Update api package tests (wip) * Test json and ssz format in generic server tests * Add a bunch of todos * Fix a few broken route definitions * Fix partial config test * Another todo note * Stringify body of json requests * Override default response json method * Validate external consensus version headers in request * Add error handling todo * Skip body schema validation for ssz request bodies * Clean up generic server tests * Pass node, proof, events generic tests * Use enum for media types * Fix a bunch of route definitions * Add justified to blockid type * Properly handle booleans, remove block values codec * Create Uint8Array test data without allocating Buffer * Let fastify handle Buffer conversion * Convert Buffer to Uint8Array in content type parser * Fix build issues * Fix fork type in builder routes * Add some notes * Properly parse request headers * Fix incorrect type assumptions in transform * Generic server tests are passing (except lightclient) * Correctly handle APIs with empty responses * Update getHeader return type to reflect no bid responses * Do not append '?' to URL if query string is empty * Let server handler set status code for parsing errors * Remove unused import * Rename function, request specific * Completely drop ssz support from getSpec * Spec tests are passing against latest releases * Drop unused fastify route config * Drop ssz request from builder routes, not yet supported * Remove import * Apply change from #6695 * Update execution optimistic meta * Apply changes from #6645 * Add workaround to fix epoch committees type issue * Add todo to fix inefficient state conversion * Convert committee to normal array * Apply changes from #6655 * Align args of validators endpoints * Convert indices to str in rewards apis * Update api spec version of README badges * Revert table formatting changes * Make this accessible for class-basd API implementations * Throw err if metadata is accessed for failed response * Add assertOk to api response * Tweak api error message * Update operationIds match spec value * Add missing version to blob sidecars metadata * Test headers and ssz bodies against spec * Minor reordering of code in spec parsing * submitBlindedBlock throws err if fork is not execution * responseOk might be undefined * Remove statusOk from route definition * Remove stale comment * Less build errors in beacon-node * getBlobSidecars return version from server impl * Update validator produce block impl * More expressive pool method args * Application methods might be undefined in mock implementations * Adress open TODOs in server handler * Api response methods are synchronous now * Fix all remaining build issues * Use more performant from/toHex in server api impls * Clean up some TODOs * Fix ApiError type * Errors related to parsing return a 400 status code * Simplify method binding * Forward api context to application methods * There is no easy way to make generic opts typesafe * Better separation of server / client code * Fix comment about missing builder bid * Remove todo, not worth the change / extra indentation * Rename route definitions functions * Return 400 if data passed to keymanager is invalid * Properly handle response metadata headers * Fix lint issues * Add header jsdoc * Move metadata related code into separate file * Remove ssz from POST requests without body * Only set content-type header if body exists * Fix headers extra * POST requests without body are handled similar to GET requests * Fix http client options tests * Improve validation and type safety of JSON metadata * Add type guard for request without body * Differentiate based on body instead of GET vs POST * More renaming * Simplify RequestCode type * Review routes, improve validation * Remaining local diff * Fix accept header handling if only support one wire format * Update 406 error to more closely match spec example * Enforce version header via custom check instead of schema * Use ssz as default request wire format * Log failure to receive head event to verbose * Do not set default value for context * Update getClient return type to better align with method name * Consistent pattern to get route definitions * Dedupe api client type for builder and keymanager * Fix fallback logic if server returns http error * Update head event error logging * Retry 415 errors with JSON and cache SSZ not supported * Use fetch spy to assert call times * Update comment * Update getLightClientUpdatesByRange endpoint meta * Do not forward ssz bytes of blinded block to publishBlock * Fix lightclient e2e tests * Version header in publishBlock api is optional * Reduce type duplication * Add option to override request init in route definition * Add JsonOnlyResp codec * Validate boolean str value from headers * Document default wire formats * Simplify merging of inits in http client * Remove type hacks from fetchBeaconHealth * Reduce call stack in http client * Add .ssz() equivalent method for json to api response * More http client tests * Ensure topics query is provided to eventstream api * Validate request content type in handler Fastify does not cover all edge cases * Review routes, fix param docs, no empty comments * Fix typo * Add note about builder spec not supporting ssz * Consistently move keymanager jsdoc to routes * Sanitize user provided init values before merging * Remove unused ssz only codec * Allow passing wire formats as string literals * chore: review proof routes (#6843) Review proof routes * chore: review lightclient routes (#6842) Review lightclient routes * chore: review node routes (#6844) Review node routes * feat: add cli flags to configure http wire format (#6840) * Review PR, mostly cosmetic changes * Fix event stream error handling --------- Co-authored-by: Cayman <caymannava@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
finalizedflag to chain data requests ethereum/beacon-APIs#254Description
Add
finalizedmetadata field to HTTP API responsesCloses #5693