Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TECHDEBT] Change the PersistenceRWContext interface to pass paramName when getting a parameter #286

Closed
8 tasks
andrewnguyen22 opened this issue Oct 5, 2022 · 8 comments · Fixed by #422
Closed
8 tasks
Assignees
Labels
code health Nice to have code improvement core starter task Good for newcomers, but aimed at core team members though still open for everyone persistence Persistence specific changes

Comments

@andrewnguyen22
Copy link
Contributor

Objective

Deprecate

  • GetBlocksPerSession(height int64) (int, error)
    and
  • GetServiceNodesPerSessionAt(height int64) (int, error)

for

  • GetParameter(paramName string, height int64) (interface, error)

Origin Document

We genericized the param setters like:

SetParam(paramName string, value interface{}) error

We need a similar pattern for getters

Goals

  • Lower code footprint,
  • Remove placeholder constants
  • Maintain a strict pattern

Deliverable

  • Code complete implementation
  • Updated documentation

General issue deliverables

  • Update the appropriate CHANGELOG
  • Update any relevant READMEs (local and/or global)
  • Update any relevant global documentation & references
  • If applicable, update the source code tree explanation
  • If applicable, add or update a state, sequence or flowchart diagram using mermaid

Creator: @andrewnguyen22

@Olshansk
Copy link
Member

Olshansk commented Oct 5, 2022

Great starter task!

@Olshansk Olshansk added persistence Persistence specific changes core starter task Good for newcomers, but aimed at core team members though still open for everyone labels Oct 7, 2022
@jessicadaugherty jessicadaugherty moved this from Backlog to Up Next in V1 Dashboard Oct 19, 2022
@Jasonyou1995
Copy link
Contributor

Jasonyou1995 commented Oct 27, 2022

I might need some help to get started (some super basic questions):

  • Task questions
    • Do I need to implement the GetParameter function?
    • Should I comment out the two deprecated functions GetBlocksPerSession and GetServiceNodesPerSessionAt then combine them to GetParameter?
    • I saw the ./utility/context.go and ./persistence/module.go scripts are using this interface, should I update them all?
  • Documentation questions
    • What is the specific documentation I should update other than CHANGELOG?

Thank you!

@Olshansk
Copy link
Member

Olshansk commented Oct 27, 2022

Do I need to implement the GetParameter function?

Yes.

Should I comment out the two deprecated functions GetBlocksPerSession and GetServiceNodesPerSessionAt then combine them to GetParameter?

Yes.

I saw the ./utility/context.go and ./persistence/module.go scripts are using this interface, should I update them all?

Yes.

What is the specific documentation I should update other than CHANGELOG?

For this change, since it's small. The changelog is enough.

@h5law
Copy link
Contributor

h5law commented Dec 23, 2022

I would be interested in taking up this task if its still relevant. The only question I have is around determining the type from the paramName argument to the GetParameter function.

If the function is solely to replace the two ones listed why not use this function which is already defined? As both the functions are int32 types if I am correct.

func (p PostgresContext) GetIntParam(paramName string, height int64) (int, error) {
	v, _, err := getParamOrFlag[int](p, types.ParamsTableName, paramName, height)
	return v, err
}

However, if you are aiming to really generalise the 3 wrappers for the getParamOrFlag function you would need to know the type of the parameter. Or simply just return it as a string. Please let me know if I am wrong here as I am not super familiar with the codebase.

I think this involves editing the following files:

shared/modules/persistence_module.go <-- comment out old functions define new function in interface
persistence/gov.go <-- comment out old functions add new GetParameter function logic
utility/gov.go <-- Comment out GetBlocksPerSession replace with GetParameter
utility/test/gov_test.go <-- Comment out GetBlocksPerSession test replace with new GetParameter test

In addition to the CHANGELOG.md

Let me know the situation I am happy to get to work. The main question is about knowing the type the paramName string will return from the database.

One way this could work is by having the function signature like: GetParameter(paramName string, value any, height int64) (any, error) which is similar to what the SetParam function does. By explicitly saying the value we want to receive from his function it becomes a lot easier. Otherwise I am not too sure about how to infer the return type.

@h5law
Copy link
Contributor

h5law commented Dec 23, 2022

@Olshansk what do you think about this?

@Olshansk
Copy link
Member

I would be interested in taking up this task if its still relevant

Yup, still relevant!

If the function is solely to replace the two ones listed why not use this function which is already defined? As both the functions are int32 types if I am correct.

This is the first step of this task and what I would call the "first commit" as you start working on it locally. It is to simply make these two functions follow the same pattern as the rest of the codebase by using SetParam(paramName string, value interface{}) error from the PersistenceModule interface.

Screenshot 2022-12-28 at 2 27 35 PM

However, if you are aiming to really generalise the 3 wrappers for the getParamOrFlag function you would need to know the type of the parameter. Or simply just return it as a string. Please let me know if I am wrong here as I am not super familiar with the codebase.

Similar to how we have SetParam(paramName string, value interface{}) error in PersistenceModule, we were thinking of adding a GetParam(paramName string, height int64) (interface{}, error) to PersistenceReadContext. There might even be an opportunity to use GetParam[T int | string | []byte](paramName string, height int64) (T, error) so the caller doesn't have to type cast it explicitly after retrieving the data.

In terms of files that'll be affected, you're on the right track.

Disclaimer: I haven't tried implementing the suggestion above so there may be some hurdles I'm unaware of.


cc @deblasis for context. I know you're planning on making some other changes to the persistence interface and curious what you think about this change.

@h5law
Copy link
Contributor

h5law commented Dec 28, 2022

@Olshansk I have actually wrote the code for this PR and update the unit test in utility/test/gov_test.go (all working successfully) I ended up going for an automatic type detection method by checking the types in the genesis_state protobuf (but reading the json tag to avoid having the proto imports used) I will open a PR so you can check it out and I don't mind changing to your proposed method as it will probably be easier to have explicit types given. My method will also never allow for a []byte return type as none of the functions in the genesis state actually have this type (reading this from persistence/types/persistence_genesis.pb.go). I'll get to work on the PR and the changes now

@deblasis
Copy link
Contributor

@Olshansk

cc @deblasis for context. I know you're planning on making some other changes to the persistence interface and curious what you think about this change.

Hi @h5law !

I would be interested in taking up this task if its still relevant. The only question I have is around determining the type from the paramName argument to the GetParameter function.

In order to determine the type of the params we can leverage the struct tags

image

These are parsed in parseGovProto() which should return everything you need to move forward. Do not hesitate if you have any questions.

Oh as I was writing I saw your message and it seems you already found your way :) well done!

h5law added a commit that referenced this issue Jan 11, 2023
…e when getting a parameter - (Issue #286) (#422)

## Description

This PR is in response to issue #286. A general function to retrieve
parameters from their `paramName` string will help to address the goals
laid out in the issue. Namely a lower code footprint, the removal of
placeholder constants and it allows for a pattern to be used from now on
regarding retrieval of parameters - similar to the pattern for setting
parameters.

The function uses the `ParameterNameTypeMap` built by the `init()`
function in `persistence/gov.go` to find the correct return type from
the parameter name string. The map is built by:
- Initialising an empty `Params` struct as defined in
`persistence/types/persistence_genesis.pb.go`
 - Loop through the fields of this stuct:
    - Extracting the parameter name from the json tag of each field
- Match the json tag as it is in the format of `{paramName},omitempty`
and the parameter name can easily be found
- Setting ParameterNameTypeMap[paramName] = stringified type of field in
struct

Building the map with the `init()` function and storing it in memory
allows for quick access each time `GetParameter` is called removing the
need to parse the `Params` struct each time it is called.

The function's signature is `GetParameter(paramName string, height
int64) (any, error)`

The function's logic is defined in the file `persistence/gov.go` and
works as follows:
- Using the `paramName` argument find the parameter return type from the
`ParameterNameTypeMap`
- Using this stringified parameter type, call the proper getter function
`getParamOrFlag[int | string | []byte]` and the values from this call
are returned.

The old functions have been removed and replaced with the new generic
function. This includes the unit test in `utility/test/gov.go` where the
`GetBlocksPerSession` test now uses the `GetParameter` function -
passing successfully. The `GetSetIntParam` and `GetSetStringParam` unit
tests in `persistence/test/gov_test.go` have also been updated to use
`GetParameter` also passing successfully.

```
ok  	github.com/pokt-network/pocket/persistence/test	20.742s
ok  	github.com/pokt-network/pocket/utility/test	17.468s
```

## Issue

Fixes #286 

## Type of change

Please mark the relevant option(s):

- [x] New feature, functionality or library
- [ ] Bug fix
- [x] Code health or cleanup
- [ ] Major breaking change
- [ ] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- Deprecate old functions (`GetBlocksPerSession` and
`GetServiceNodesPerSessionAt`) in favour of using the more general
`GetParameter` in `shared/modules/persistence_module.go`
- Add `init()` function in `persistence/gov.go` to build
`ParameterNameTypeMap` of parameter names and their return types
- Define the logic for the `GetParameter`  in `persistence/gov.go`
- Replace old function with `GetParameter` in `utility/gov.go`
- Change unit test for `GetBlocksPerSession` in
`utility/test/gov_test.go` to use the new function
- Change unit tests in `persistence/test/gov_test.go` to use the
`GetParameter()` function instead of the `GetIntParam`, and
`GetStringParam` functions

## Testing

- [x] `make develop_test`
- [x]
[LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md)
w/ all of the steps outlined in the `README`


## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have tested my changes using the available tooling
- [x] I have updated the corresponding CHANGELOG

### If Applicable Checklist
- [ ] I have updated the corresponding README(s); local and/or global
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have added, or updated,
[mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding
README(s)
- [ ] I have added, or updated, documentation and
[mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*`
if I updated `shared/*`README(s)
@github-project-automation github-project-automation bot moved this from Up Next to Done in V1 Dashboard Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code health Nice to have code improvement core starter task Good for newcomers, but aimed at core team members though still open for everyone persistence Persistence specific changes
Projects
Status: Done
5 participants