Optimize balance-related queries with a cache #2383

rafal-ch · 2024-10-22T15:02:46Z

Linked Issues/PRs

Closes #1965

Description

The client uses the off-chain database metadata to tell if it can use the optimized queries or fall-back to the legacy ones.
When client starts the metadata is either:
1. Kept intact in-case genesis exists (meaning - no optimized queries are available)
  - this stems from the fact that the pre-existing DB contains some coins already and we'll not be re-indexing them in order to create the lookup indexes
  - in this mode of operation the "old" way of getting balances is used as confirmed in logs:
    - worker_service: 607: Balances cache available: false
    - query::balance: 100: Querying balances without balances cache owner=53a9c6a...
2. Initialized with V2 if genesis is missing
  - this means that while re-syncing the DB the balance indexes are going to be created and could be used to satisfy the requests
    - worker_service: 607: Balances cache available: true
    - query::balance: 151: Querying balances using balances cache owner=53a9c6a...

Technical considerations:

Metadata is extended with V2 which now contains indexation_availability. This is a set that holds the available indexations (currently only Balances)
New databases are added: CoinBalances and MessageBalances that keep the coin and message balances respectively
- CoinBalances stores the balances per owner and asset_id
- MessageBalances stores balances per owner, separately for retryable and non-retryable messages
In case balances indexation is available (see "Description" above):
- these databases are updated in the GrapQL API worker service when processing one of these events: MessageImported, MessageConsumed, CoinCreated, CoinConsumed
- balance() and balances() queries use these databases to satisfy the query

Checklist

New behavior is reflected in tests

Before requesting review

I have reviewed the code myself

acerone85

Overall LGTM.

A side question, right now indexation is enabled by default on nodes that did not process genesis. Do we want to also make this configurable? E.g. do some nodes like the block producer not want to perform any indexation on balances, regardless of whether they start at genesis or not?

crates/fuel-core/src/graphql_api/storage/balances.rs

crates/fuel-core/src/database/metadata.rs

crates/fuel-core/src/graphql_api/indexation.rs

crates/fuel-core/src/service/adapters/graphql_api/off_chain.rs

xgreenx

PR looks good=) I think some parts can be simplified and we can connect all places related to the same code together via same functions.

Created a PR with suggestions=) #2465

rafal-ch · 2024-11-29T13:41:46Z

I think some parts can be simplified and we can connect all places related to the same code together via same functions.

Nice one, thanks. In particular I like how you got rid of the AssetBalanceWithRetrievalErrors, as it was not something I was especially proud of 👍

…2465) Suggestions and simplifications for the #2383.

rafal-ch · 2024-12-05T12:53:44Z

Overall LGTM.

A side question, right now indexation is enabled by default on nodes that did not process genesis. Do we want to also make this configurable? E.g. do some nodes like the block producer not want to perform any indexation on balances, regardless of whether they start at genesis or not?

@xgreenx
Actually, @acerone85 has a point here. Opting out from the balances cache might be potentially useful in case the index somehow becomes corrupted. But otoh, in such case also other parts of the DB could be corrupted and it'll be probably better to sync from scratch and rebuild the index.

…#2472) ## Description This PR partially reverts the changes introduced in the [balances indexation PR](#2383). In particular: * reverts the type of `amount` in the `Balance` GraphQL type back to `U64` * introduces a new field `amountU128` in the `Balance` GraphQL type * reverts the `FuelClient::balance()` function to return `u64`. There is no `balance_u128()` function added to the `FuelClient` for now. ![image](https://github.com/user-attachments/assets/621db3b9-54f8-46c9-8bdd-c29f65b01c7c) This is done to maintain the full backward compatibility on the GraphQL level, internally the total balance is still processed using `u128`. ### Before requesting review - [X] I have reviewed the code myself

xgreenx · 2024-12-12T04:49:37Z

We have an issue with adding a flag that allows you to not support off-chain database. We could consider to allows customization on what stuff we want from the off-chain database. But I think we don't need to worry about it, until someone will ask for this kind of feature=)

rafal-ch added 30 commits October 14, 2024 10:27

Add basic balances functionality

4a70f52

Add support for querying all balances for user

7d472fb

Adding balances_indexation_progress to DB metadata

4296fd4

Attempt at migrating the metadata to store the indexation progress

77d0f16

Merge remote-tracking branch 'upstream/master' into 1965_balances

525e6f9

Hack the replace_forced() and commit_changes_forced in

8eba5d7

Introduce ForcedCommitDatabase

c083f45

Update dependencies

6b76c37

DB metadata can track multiple indexation progresses

f246cd0

into_genesis() attemt

7472db7

Add some TODOs with ideas for the future

b4d2e0f

Use double_key! macro to define the balances key

d38fdb3

Add basic_storage_tests! for Balances

292acab

Merge remote-tracking branch 'upstream/master' into 1965_balances

651bcc8

Balances DB stores separate information for coins and messages

8342c8f

Fix the recursive call

9a9f120

Init indexation progresses with 0 upon metadata migration

6aa9325

Remove debug prints

b153db4

Store incoming balance in the new Balances DB

512a8a3

Read balance from the new Balances database

adf9e2a

Update coin balance, don't overwrite

344ca90

Use more detailed IndexationStatus, not just block height

f73dbed

Add processing of MessageImported

80da09b

Simplify processing of coins and message amounts

ad5216d

Extract increase_balance()

c784e44

Store coin and message balances separately

5762665

Clean up column naming

5595985

Add test for coin balances

38dd8d6

Support both coins and messages in the new balance system

530bcaf

Merge remote-tracking branch 'upstream/master' into 1965_balances

5604f30

rafal-ch added 6 commits November 21, 2024 14:16

Do not import tracing and remove some log messages

7a95506

Move the indexation initialization to combined databases

33f75af

Return balances via iterator, not collection

6b8f18e

Remove dbg print

2408e56

Make the balances() implementation more readable

67d17d5

Update comment to mention follow-up issue

bb2537e

rafal-ch removed the no changelog Skip the CI check of the changelog modification label Nov 22, 2024

rafal-ch added 2 commits November 22, 2024 14:58

Revert the stray change in balance.rs

53bbc7b

Add basic storage tests for CoinBalances and MessageBalances

cca1891

rafal-ch mentioned this pull request Nov 26, 2024

Create indexation cache for "coins to spend" queries #2455

Closed

1 task

Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

c0cb68d

rafal-ch mentioned this pull request Nov 26, 2024

Create uninitialized task for v1 gas price service #2442

Merged

9 tasks

acerone85 previously approved these changes Nov 26, 2024

View reviewed changes

Merge remote-tracking branch 'upstream/master' into 1965_balances_cache

370a196

rafal-ch mentioned this pull request Nov 28, 2024

Use indexation cache to satisfy "coins to spend" queries #2463

Open

2 tasks

xgreenx mentioned this pull request Nov 29, 2024

Small suggestions and simplification to the balances indexation PR #2465

Merged

xgreenx reviewed Nov 29, 2024

View reviewed changes

Small suggestions and simplification to the balances indexation PR (#…

be9b742

…2465) Suggestions and simplifications for the #2383.

rafal-ch dismissed acerone85’s stale review via be9b742 November 29, 2024 13:44

xgreenx approved these changes Nov 29, 2024

View reviewed changes

acerone85 approved these changes Nov 29, 2024

View reviewed changes

rafal-ch enabled auto-merge (squash) November 29, 2024 15:42

Merge branch 'master' into 1965_balances_cache

181811a

rafal-ch merged commit ffb7644 into master Nov 29, 2024
30 checks passed

rafal-ch deleted the 1965_balances_cache branch November 29, 2024 16:26

rafal-ch mentioned this pull request Dec 5, 2024

Revert balances amount to U64 and introduce new amountU128 getter #2472

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize balance-related queries with a cache #2383

Optimize balance-related queries with a cache #2383

rafal-ch commented Oct 22, 2024 •

edited

Loading

acerone85 left a comment

xgreenx left a comment

rafal-ch commented Nov 29, 2024

rafal-ch commented Dec 5, 2024

xgreenx commented Dec 12, 2024

Optimize balance-related queries with a cache #2383

Optimize balance-related queries with a cache #2383

Conversation

rafal-ch commented Oct 22, 2024 • edited Loading

Linked Issues/PRs

Description

Checklist

Before requesting review

acerone85 left a comment

Choose a reason for hiding this comment

xgreenx left a comment

Choose a reason for hiding this comment

rafal-ch commented Nov 29, 2024

rafal-ch commented Dec 5, 2024

xgreenx commented Dec 12, 2024

rafal-ch commented Oct 22, 2024 •

edited

Loading