[Access] Add util command to backfill tx error messages db #6525
Conversation
…BlockID, backend and ingestion engine, updated tests
… of github.com:The-K-R-O-K/flow-go into UlyanaAndrukhiv/6413-backfill-tx-error-messages
…sages' of github.com:The-K-R-O-K/flow-go into UlyanaAndrukhiv/6497-refactor-executionNodesForBlockID
… of github.com:The-K-R-O-K/flow-go into UlyanaAndrukhiv/6413-backfill-tx-error-messages
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6525 +/- ##
==========================================
- Coverage 41.15% 41.09% -0.07%
==========================================
Files 2052 1934 -118
Lines 182212 169729 -12483
==========================================
- Hits 74989 69744 -5245
+ Misses 100944 94232 -6712
+ Partials 6279 5753 -526
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
… of github.com:The-K-R-O-K/flow-go into UlyanaAndrukhiv/6413-backfill-tx-error-messages
… of github.com:The-K-R-O-K/flow-go into UlyanaAndrukhiv/6413-backfill-tx-error-messages
…ality, updated tests
| } else if startHeight > rootHeight { | ||
| data.startHeight = startHeight | ||
| } |
There was a problem hiding this comment.
let's have it return an error if startHeight < rootHeight rather than silently updating the value
| if startHeightIn, ok := input["start-height"]; ok { | ||
| if startHeight, err := parseN(startHeightIn); err != nil { | ||
| return admin.NewInvalidAdminReqErrorf("invalid 'start-height' field: %w", err) | ||
| } else if startHeight > lastSealedHeight { |
There was a problem hiding this comment.
nit: start a new conditional when the previous one returns. it makes the code a bit easier to read
| } else if startHeight > lastSealedHeight { | |
| } | |
| if startHeight > lastSealedHeight { |
| if endHeightIn, ok := input["end-height"]; ok { | ||
| if endHeight, err := parseN(endHeightIn); err != nil { | ||
| return admin.NewInvalidAdminReqErrorf("invalid 'end-height' field: %w", err) | ||
| } else if endHeight < lastSealedHeight { |
There was a problem hiding this comment.
same here. return an error if endHeight > lastSealedHeight
|
|
||
| if data.endHeight < data.startHeight { | ||
| return admin.NewInvalidAdminReqErrorf( | ||
| "'start-height' %d should not be smaller than 'end-height' %d", |
There was a problem hiding this comment.
| "'start-height' %d should not be smaller than 'end-height' %d", | |
| "'start-height' %d must not be smaller than 'end-height' %d", |
| for _, en := range requestedENIdentifiers { | ||
| id, exists := allIdentities.ByNodeID(en) | ||
| if !exists { | ||
| return nil, admin.NewInvalidAdminReqParameterError("execution-node-ids", "could not found execution nodes by provided ids", executionNodeIdsIn) |
There was a problem hiding this comment.
can you add the specific nodeID that's missing to the error. that would make it easier to debug
| } | ||
|
|
||
| blockID := header.ID() | ||
| err = b.txErrorMessagesCore.HandleTransactionResultErrorMessagesByENs(ctx, blockID, data.executionNodeIds.ToSkeleton()) |
There was a problem hiding this comment.
rather than calling data.executionNodeIds.ToSkeleton() each height and rebuilding the list, either cache it before the loop, or store the IdentitySkeletonList into data.
| @@ -303,13 +303,10 @@ func (b *backendEvents) getBlockEventsFromExecutionNode( | |||
| // choose the last block ID to find the list of execution nodes | |||
| lastBlockID := blockIDs[len(blockIDs)-1] | |||
|
|
|||
| execNodes, err := rpc.ExecutionNodesForBlockID(ctx, | |||
| execNodes, err := b.execNodeIdentitiesProvider.ExecutionNodesForBlockID( | |||
There was a problem hiding this comment.
nit
execNodes, err := b.execNodeIdentitiesProvider.ExecutionNodesForBlockID(ctx, lastBlockID)
Closes: #6413
Context
This pull request introduces a utility command that allows ANs to backfill missing transaction error messages from start to end height and a set of ENs to their local databases. This ensures that historical ANs have the necessary data to serve requests moving forward.
The command accepts a list of EN IDs (
execution-node-ids) , if not provided, it defaults to query any available EN. It optionally takes a start and end block height (start-height,end-height); if not provided, it defaults to theroot blockandlatest sealed block.Before execution, the command checks if AN indexing is enabled. For each failed transaction within the block range, it retrieves and stores the error messages locally. It queries ENs in order and stops after the first successful response.
The command uses
GetTransactionErrorMessagesByBlockIDfor optimized batch retrieval.Unit tests are included to ensure functionality.