nextKey is not returned when calling GetTxsEvent

[swmd]

Summary of Bug

It appears that the pagination does not work properly when calling QueryClient.queryUnverified().
There are two issues I found in the response:

1. nextKey is always an empty array.
2. offset and limit keys don't work correctly. When I passed offset=4, limit=7, it just returned the result from the beginning. The response had the transactions of indexes 0-6.

Version

v0.44

Steps to Reproduce

Here's the code snippet I used to get the transactions history.

import { Tendermint34Client } from "@cosmjs/tendermint-rpc"
import { QueryClient } from "@cosmjs/stargate"
import { GetTxsEventRequest } from "cosmjs-types/cosmos/tx/v1beta1/service"

const client = QueryClient.withExtensions(
    await Tendermint34Client.connect({rpc_endpoint}),
)

client.queryUnverified("/cosmos.tx.v1beta1.Service/GetTxsEvent", GetTxsEventRequest.encode({
    events: [
        "message.action='delegate'",
	"delegate.validator='{validator_address}'",
    ],
    pagination: {
	countTotal: true,
	key: new Uint8Array(),
	offset: Long.fromNumber(0, true),
	limit: Long.fromNumber(5, true),
	reverse: false,
    },
    orderBy: OrderBy.ORDER_BY_ASC,
}).finish())

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[alexanderbez]

So one thing to note about this gRPC query method: It's just a proxy to Tendermint which does the indexing, unlike native gRPC queries that go against the application state. Ref QueryTxsByEvents -> node.TxSearch.

What does this mean? This means there is now way, or at least no direct or accessible way to know NextKey. So that hopefully addresses your first point.

As for offset and page, that should be working but that does seem broken due to NewSearchTxsResult

[alexanderbez]

So to summarize I think there are two issues:

1. Tendermint pagination is 1-based, SDK is 0, so we need to account for that in QueryTxsByEvents.
2. Fix QueryTxsByEvents to apply the offset.

[swmd]

@alexanderbez Could you hopefully provide a working code example with pagination?

[alexanderbez]

@alexanderbez Could you hopefully provide a working code example with pagination?

Of state-based queries? If so, that happens via a generic auxiliary function FilteredPaginate.

[swmd]

@alexanderbez I couldn't find a useful doc or examples of FilteredPaginate.
My goal is to get the whole transactions history using any query. Can you help me to identify the best way to do it?

[alexanderbez]

FilteredPaginate is used for state kvstore based queries, as I mentioned, it doesn't apply to querying for txs as that is indexed by Tendermint. However, you can see an example of FilteredPaginate used here.

My goal is to get the whole transactions history using any query.

Use GetTxsEvent as you already are. Just keep iterating over the pages.

[swmd]

@alexanderbez Can you provide a working example of the GetTxsEvent query? It was my original question.
The pagination doesn't work properly especially when I pass the offset.

[alexanderbez]

Sure!

https://lcd-osmosis.blockapsis.com/cosmos/tx/v1beta1/txs?events=message.action=%27superfluid_delegate%27&pagination.limit=50

[DeimonDB]

Hey @swmd this is late but I hope it would help.

I faced the same issue. It seems that currently the pagination is incorrectly implemented.

Let's say in total there are 10 data.

• If we pass limit = 5 and offset = 0, it will return data [0..4]
• if we pass limit = 5 and offset = 5, it will return data[5..9]
• but if we pass limit = 5 and offset = 2:
  • it will return data[0..4] instead of returning data[2..6]
  • The pagination logic seems to try to figure out the "page" of the pagination by the given limit and offset.
  • I guess it determines page by this formula page = floor(offset / limit).
  • That's why when we give limit = 5 and offset = 2, it calculates page=0, and returns data for the first page

The pagination doesn't work properly especially when I pass the offset.

Therefore, when you use the offset, you need to make sure the offset is a multiple of the limit, i.e., if you give limit = 10, your possible offsets are 0, 10, 20, 30, etc

[alexanderbez]

@facundomedica or @julienrbrt would you have some bandwidth to look into pagination?

[blushi]

The thing is that node.TxSearch from tendermint that is used internally by GetTxsEvent only accepts some page and perPage parameters: https://github.com/tendermint/tendermint/blob/v0.35.6/rpc/client/interface.go#L93, which basically makes it impossible by design to accept an offset that wouldn't be a multiple of the limit...

Probably GetTxsEvent shouldn't accept sdk query.PageRequest (nor return a query.PageResponse) and just use page and limit/perPage params like it's the case for the corresponding CLI query command QueryTxsByEventsCmd?

[alexanderbez]

Probably GetTxsEvent shouldn't accept sdk query.PageRequest (nor return a query.PageResponse) and just use page and limit/perPage params like it's the case for the corresponding CLI query command QueryTxsByEventsCmd?

Yes, this seems reasonable to me. It should not accept and return those parameters 👍

[blushi]

Probably GetTxsEvent shouldn't accept sdk query.PageRequest (nor return a query.PageResponse) and just use page and limit/perPage params like it's the case for the corresponding CLI query command QueryTxsByEventsCmd?

Yes, this seems reasonable to me. It should not accept and return those parameters 👍

A few UX questions regarding this change:

• should we just deprecate those pagination fields in the request/response for now?
• node.TxSearch accepts a page params >= 1, which means you would always need to provide page=1 as part of the request, should we just default to page=1 if page is not provided as part of the request?

[alexanderbez]

should we just deprecate those pagination fields in the request/response for now?

Yes, mark them as deprecated and remove their usage from the implementation.

node.TxSearch accepts a page params >= 1, which means you would always need to provide page=1 as part of the request, should we just default to page=1 if page is not provided as part of the request?

Yes, just default to 1 when omitted 👍

[xloem]

https://lcd-osmosis.blockapsis.com/cosmos/tx/v1beta1/txs?events=message.action=%27superfluid_delegate%27&pagination.limit=50

looks like this example is broken now !

[alexanderbez]

This has been posted and mentioned in many places -- SearchTxs is a CometBFT proxy RPC query, so it does NOT support the SDK's native pagination logic.

[xloem]

It returns an empty set whether pagination parameters are passed or not. Is there a new example of a successful query?

[alexanderbez]

The pagination params won't do anything. An example query could be message.module='bank'

[xloem]

Empty list: https://lcd-osmosis.blockapsis.com/cosmos/tx/v1beta1/txs?events=message.module=%27bank%27