Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Params, Clients] add generic ParamsQuerier #995

Open
wants to merge 9 commits into
base: issues/543/cache/memory
Choose a base branch
from

Conversation

bryanchriswhite
Copy link
Contributor

@bryanchriswhite bryanchriswhite commented Dec 11, 2024

Summary

Adds the the ParamsQuerier interface, cachedParamsQuerier implementation, config, and options functions.

ParamsQuerier is a generic implementation of a "params querier" (i.e. interface { GetParams(ctx) (P, error }, where P is the params type) that can be embedded in any module-specific query client implementation. It implements configurable params caching for historical heights (incl. current height).

---
title: Legend
---

classDiagram-v2

class GenericInterface__T__any {
    <<interface>>
    GenericMethod() T
}

class Implemenetation {
    ExportedField FieldType
    unexportedField FieldType
}

Implemenetation --|> GenericInterface__T__any: implements

class Embedder__T__any {
    <<interface>>
    GenericInterface[T]
}

Embedder__T__any ..|> GenericInterface__T__any: embeds

Loading
---
title: Client caching options
---

classDiagram-v2

class ParamsQuerier__P__sdk_Msg {
    <<interface>>
    GetParams(ctx context.Context) (params P, err error)
    GetParamsAtHeight(ctx context.Context, height int64) (params P, err error)
}

%% class paramsQuerier__P__sdk_Msg {
%%     <<interface>>
%%     GetParams(ctx context.Context) (P, error)
%% }

class cachedParamsQuerier__P__sdk_Msg__Q__paramsQuerier:::cacheImpl {
    queryClient Q__paramsQuerier[P]
    paramsCache HistoricalQueryCache[P]
    ...
}
ParamsQuerier__P__sdk_Msg --|> cachedParamsQuerier__P__sdk_Msg__Q__paramsQuerier
cachedParamsQuerier__P__sdk_Msg__Q__paramsQuerier --* InMemoryCache: EITHER
cachedParamsQuerier__P__sdk_Msg__Q__paramsQuerier ..* BadgerCache: OR
%% cachedParamsQuerier__P__sdk_Msg__Q__paramsQuerier --> paramsQuerier__P__sdk_Msg


class HistoricalQueryCache__T__any {
    <<interface>>
    GetLatestVersion(key string) (value T, err error)
    GetVersion(key string, version int64) (value T, err error)
    SetVersion(key string, value T, version int64) (err error)
}
%% HistoricalQueryCache__T__any --|> QueryCache__T__any

class InMemoryCache:::cacheImpl
InMemoryCache --|> HistoricalQueryCache__T__any

class BadgerCache:::cacheImpl {
    // TODO
}
BadgerCache --|> HistoricalQueryCache__T__any

%% TODO_IN_THIS_COMMIT: uncomment classDef in docs - github can't parse it.
%%classDef cacheImpl fill:#074,color:#fff;
Loading

Issue

Type of change

Select one or more from the following:

Testing

  • Documentation: make docusaurus_start; only needed if you make doc changes
  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

@bryanchriswhite bryanchriswhite added relayminer Changes related to the Relayminer off-chain Off-chain business logic on-chain On-chain business logic labels Dec 11, 2024
@bryanchriswhite bryanchriswhite self-assigned this Dec 11, 2024
@bryanchriswhite bryanchriswhite linked an issue Dec 11, 2024 that may be closed by this pull request
4 tasks
@bryanchriswhite bryanchriswhite force-pushed the issues/543/params/query-client branch from 48f5762 to 49183d7 Compare December 12, 2024 14:07
@bryanchriswhite bryanchriswhite force-pushed the issues/543/params/query-client branch from 49183d7 to ba61d14 Compare December 12, 2024 14:12
@bryanchriswhite bryanchriswhite marked this pull request as ready for review December 12, 2024 14:19
pkg/client/interface.go Outdated Show resolved Hide resolved
// P is a pointer type of the parameters, and Q is the interface type of the
// corresponding query client.
type cachedParamsQuerier[P cosmostypes.Msg, Q paramsQuerierIface[P]] struct {
clientConn gogogrpc.ClientConn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed at the cachedParamsQuerier level?

Copy link
Contributor Author

@bryanchriswhite bryanchriswhite Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed. The cachedParamsQuerier has to hold a reference to this because it's a dependency of all query clients, and it also has to know how to generically construct a given modules query client. See NewCachedParamsQuerier():

	if err = depinject.Inject(
		deps,
		&querier.clientConn,
	); err != nil {
		return nil, err
	}

	querier.queryClient = queryClientConstructor(querier.clientConn)

pkg/client/query/paramsquerier.go Outdated Show resolved Hide resolved
}

// Update the cache.
if err = bq.paramsCache.Set("params", params); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider the case where paramsCache is in historical mode, otherwise this call might fail.

Additionally, I think we should find a way to get the height/version of the retrieved params in order to set it using SetAsOfVersion.

Maybe add the height to the QueryParamsResponse?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed:

  1. cachedParamsQuerier's cache is ALWAYS in historical mode.
  2. This isn't possible anymore due to the decision we made to disable #Set() in historical mode.
  3. We're going to transition to event-driven cache warming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to transition to event-driven cache warming.

Can you add a TODO for that somewhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cachedParamsQuerier's cache is ALWAYS in historical mode.

#PUC

pkg/client/query/paramsquerier.go Outdated Show resolved Hide resolved
bryanchriswhite and others added 2 commits December 18, 2024 17:31
Co-authored-by: Redouane Lakrache <r3d0ne@gmail.com>
@bryanchriswhite bryanchriswhite force-pushed the issues/543/params/query-client branch from fbd5934 to 3abc4b5 Compare December 18, 2024 16:51
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing big. Appreciate all the TODOs, comments and explanations.

Will approve after we merge in the base!

@@ -380,3 +381,15 @@ type HistoricalQueryCache[T any] interface {
// SetAsOfVersion adds or updates a value at a specific version number.
SetAsOfVersion(key string, value T, version int64) error
}

// ParamsQuerier represents a generic querier for module parameters.
// This interface should be implemented by any module-specific querier
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// This interface should be implemented by any module-specific querier
// This interface SHOULD be implemented by any module-specific querier

Comment on lines +13 to +14
defaultMaxVersionAge = 100
defaultMaxKeys = 1000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
defaultMaxVersionAge = 100
defaultMaxKeys = 1000
// These are sensible defaults that can be customized below.
// The default oldest version (relative to the latest) that'll be considered
// a cache hit by the client.
defaultMaxVersionAge = 100
// The default maximum number of (key, value) pairs that'll be maintained
// in the cache irrespective of the key or value size.
defaultMaxKeys = 1000

return func(cfg *paramsQuerierConfig) {
cfg.logger = logger
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is clean!

}

// Update the cache.
if err = bq.paramsCache.Set("params", params); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to transition to event-driven cache warming.

Can you add a TODO for that somewhere?

}

// Update the cache.
if err = bq.paramsCache.Set("params", params); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cachedParamsQuerier's cache is ALWAYS in historical mode.

#PUC

err = cache.ErrCacheMiss.Wrapf("TODO: on-chain historical data not implemented")
logger.Error().Msgf("%s", err)

// Meanwhile, return current params as fallback. 😬
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rube_Goldberg's__Self-Operating_Napkin__(cropped)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
off-chain Off-chain business logic on-chain On-chain business logic relayminer Changes related to the Relayminer
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

[Off-Chain] ModuleParamsClient & Historical Params
3 participants