Implement the chain queries required by the relayer

[ancazamfir]

Summary

The relayer needs to perform different types of queries. Some of them are partially implemented.

Problem Definition

The following queries need to be implemented. All should allow for height and prove parameters to indicate the height of the query and whether a proof is required.

Queries required for the relayer:
On start:

• query client state (NOTE: this has TM data) - query ClientState given client id cosmos/ibc-proto-rs#21
• query connections per client - query connections given client id #151
• query channels per connection - Query channels per connection #505
• query per (channel, port) for send and recv sequence numbers for ordered channels
• query unrelayed acks for unordered channels - Implement Packet lifecycle in relayer v0 #379
• query all packet commitments for (channel, port) - Implement Packet lifecycle in relayer v0 #379

For IBC Event:

• query client consensus state (NOTE: this has TM data) - query ConsensusState given client id and height ibc-rs#129
• query connection given connection_id -query connection given connection_id #130
• query channel given channel_id - query channel given channel id and port id ibc-rs#130
• query packet commitment - Implement Packet lifecycle in relayer v0 #379

For a list of query paths to fetch this data from the abci app, see https://github.com/cosmos/ics/tree/master/spec/ics-024-host-requirements#path-space

Various:

• unbonding_period from the Cosmos staking module

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[ebuchman]

Still trying to get a get sense of the key space, the store layout, and the actual abci queries to fetch everything.

But just a note we can query for entire substores with eg.

curl 'http://localhost:26657/abci_query?path="store/ibc/subspace"&data="connections"'

I believe this will give us all the items under ?path="store/ibc/key"&data="connections/<identifier>" for each value of <identifier>.

I'm not sure if there's a way to fetch just the list of identifiers; it seems with this you fetch all the underlying values too.

[ebuchman]

Ok, so far we have been thinking about the relayer as something which interacts with two primary API endpoints from Tendermint on default port 26657:

• abci_query, a raw abci query for Cosmos-SDK specific paths which returns protobuf encoded values
• broadcast_tx_commit, a raw transaction send that takes a protobuf encoded signed tx

Both of these are generic to tendermint. However the SDK also exposes it's own REST API on default port 1317 that allows you to query SDK-specific data and send SDK-specific transactions, all in JSON without any amino or protobuf encoding. It will even sign the transactions for you, assuming it has access to the relevant key. Documentation for this API in the pre-IBC version of the SDK is here: https://cosmos.network/rpc/v0.37.9. I'm not sure if the latest version of the docs is built somewhere, but you can see all the endpoints by grepping for r.HandleFunc in the x/ibc dir in the SDK. For each IBC module, they're contained in the client/rest/tx.go and client/rest/query.go files.

For instance you can query all connections with curl localhost:1317/ibc/connections which returns JSON of all connections and you can send a ConnOpenInit transaction by POSTing JSON data to localhost:1317/ibc/connections/open-init.

In the past (ie. v0.37.9 and before, where v0.37.9 is the latest version used for the Cosmos Hub), this REST API server ran as a "proxy server", ie. a separate process from full nodes, eg. on a local machine, so it would do the raw abci queries and transaction sends to a remote full node, and expose a REST API locally so that local applications (eg. some wallet UI) could have a more convenient interface and wouldn't have to worry about protobuf and signing and that, it just deals with JSON. We called it a "proxy server" since it proxied a nice local JSON REST API into the underlying raw tendermint rpc communication with a remote full node. This way the local REST API server could also be given control of the user's local keys, so it could sign txs for the user and broadcast them to the full node. It's depicted in this diagram as the "Light Client Daemon" on the "Light Node", since it was able to run a light client and so could verify everything it got from full nodes before exposing it in JSON over the REST API.

With the latest SDK, the rest server now runs as part of the full node, which means it can't be used to sign txs (since youre not going to give the full node your private key), but in principle it could still be used for queries. However if we want to use the Rust light client, we'd still need to be able to work with protobuf so we can verify Merkle proofs about the data. So ultimately we may still need the relayer to use raw abci queries and txs, but in the meantime the REST API can be used for local development and testing of the core relayer algorithm since it effectively unblocks sending txs and anything else blocked by amino/proto stuff.

We could imagine, if we can still run the proxy server locally and use remote full nodes (might take a bit of work on the Go), that we could build a relayer process in Rust which uses local proxy servers for each full node it wants to talk to, and delegates all protobuf decoding, light client logic, merkle proof verification, and transaction signing to the proxy servers, leaving only the core relayer logic for it to handle, ie. how to turn events from one chain into txs for another.

Of course this is a bit contrived of a setup, and probably not what we want. However in the meantime, since we can run the full nodes locally too for testing, we could consider using this API as a way to unblock developing and testing the core relayer logic against real Gaia apps.

The API can be turned on for simd from the API Configuration section of the eg. simapp/config/app.toml

For instance, to query for all clients from it:

$ curl localhost:1317/ibc/clients
{
  "height": "0",
  "result": [
    {
      "type": "ibc/client/tendermint/ClientState",
      "value": {
        "id": "ethbridge",
        "trust_level": {
          "numerator": "1",
          "denominator": "3"
        },
        "trusting_period": "1209600000000000",
        "unbonding_period": "1814400000000000",
        "MaxClockDrift": "10000000000",
        "frozen_height": "0",
        "last_header": {
          "signed_header": {
            "header": {
              "version": {
                "block": "2",
                "app": "2"
              },
              "chain_id": "chain_B",
              "height": "10",
              "time": "2020-07-10T06:05:02.142281Z",
              "last_block_id": {
                "hash": "0000000000000000000000000000000000000000000000000000000000000000",
                "parts": {
                  "total": "9223372036854775807",
                  "hash": "0000000000000000000000000000000000000000000000000000000000000000"
                }
              },
              "last_commit_hash": "E7AAD01A1AF897B05BCF78C7563B5D1ADC2939D543DAC949A5C8712156D19BF8",
              "data_hash": "6D6E28B8B98B5327042EA50A57DD46E6CC851C72E528BDEAA6EFDEEEFE66A0B8",
              "validators_hash": "60854EC04E71A8FD0AF7E57D3DE84B4C87EDCA0CF382FD0DB847754F79B73492",
              "next_validators_hash": "60854EC04E71A8FD0AF7E57D3DE84B4C87EDCA0CF382FD0DB847754F79B73492",
              "consensus_hash": "E5E566C41ED57E3FF8CC10F184178788B8FAA602B07CF1F425217BD8179F1F24",
              "app_hash": "41CAFAE31CC70F5801FA1016A2DD54A9BCB8201B5B389919FE9976762532C516",
              "last_results_hash": "092E058630247ED6009863A12EEE117D26CD9D08B5ADCAAB37F2AB35DB475A37",
              "evidence_hash": "73865DB08F49D58428905D389AB4CA4B96E45A3206C7A69D43A5DC7372E60714",
              "proposer_address": "25EF56CA795135E409368E6DB8110F22A4BE05C2"
            },
            "commit": {
              "height": "10",
              "round": "1",
              "block_id": {
                "hash": "6B9DC374300F3093EA4C0710D2CCD82A6AAB1FEFDFEB832F374CC42D4EBDE206",
                "parts": {
                  "total": "3",
                  "hash": "87080A39CFE336A663EA5C0D9BDEEF493ABDC18B7C3E3E0C8E661354C1831432"
                }
              },
              "signatures": [
                {
                  "block_id_flag": 2,
                  "validator_address": "25EF56CA795135E409368E6DB8110F22A4BE05C2",
                  "timestamp": "2020-07-10T06:05:02.142281Z",
                  "signature": "JMLsYYJi6oLAgZ0SELwXoLqFKeq6TQKkuXehX4r33f0DjxD6fuBBPYC7RUhGcQ3ARQyAnolOKzKfSBaRDAAkDQ=="
                }
              ]
            }
          },
          "validator_set": {
            "validators": [
              {
                "address": "25EF56CA795135E409368E6DB8110F22A4BE05C2",
                "pub_key": {
                  "type": "tendermint/PubKeyEd25519",
                  "value": "FfL2qaxy/V247i+s3j9GUpEyOfgvVkJs0QsogtqBKCo="
                },
                "voting_power": "10",
                "proposer_priority": "0"
              }
            ],
            "proposer": {
              "address": "25EF56CA795135E409368E6DB8110F22A4BE05C2",
              "pub_key": {
                "type": "tendermint/PubKeyEd25519",
                "value": "FfL2qaxy/V247i+s3j9GUpEyOfgvVkJs0QsogtqBKCo="
              },
              "voting_power": "10",
              "proposer_priority": "0"
            }
          }
        },
        "proof_specs": [
          {
            "leaf_spec": {
              "hash": 1,
              "prehash_value": 1,
              "length": 1,
              "prefix": "AA=="
            },
            "inner_spec": {
              "child_order": [
                0,
                1
              ],
              "child_size": 33,
              "min_prefix_length": 4,
              "max_prefix_length": 12,
              "hash": 1
            }
          },
          {
            "leaf_spec": {
              "hash": 1,
              "prehash_value": 1,
              "length": 1,
              "prefix": "AA=="
            },
            "inner_spec": {
              "child_order": [
                0,
                1
              ],
              "child_size": 32,
              "min_prefix_length": 1,
              "max_prefix_length": 1,
              "hash": 1
            }
          }
        ]
      }
    },
    {
      "type": "ibc/client/localhost/ClientState",
      "value": {
        "id": "localhost",
        "chain_id": "ibc-test",
        "height": "353"
      }
    }
  ]
}

[ancazamfir]

I agree we may use the REST API to make some progress. But the IBC standard way is to use gRPC/ protobuf regardless of the chain type. I think we still need to work on the protobuf.

[ebuchman]

Yep just outlining it as a way to possibly unblock progress

[greg-szabo]

I’m also thinking about this from a chain generalization perspective. What if the Chain trait has an implementation for Tendermint chains by default? It can always be overwritten for other chains and we kind of base everything on Tendermint. Then we can introduce a CosmosSDKChain implementation of of it where we override some of the query functions. I expect some unimplemented functions in either case until we get the full list of queries in order but at least this enables us to write whichever implementation is easy, and they get sorted into the right place automatically.

[ebuchman]

Not sure what a Tendermint chain would consist of since Tendermint doesn't specify how IBC state is encoded or where to find it or how to send transactions so that might be too generic. It seems the Chains we use have to be somewhat based on the ABCI app framework which determines how state is encoded and how txs are formed and signed ...

[greg-szabo]

I might be overgeneralizing but based on how we started working on the queries, I thought that I should be able to implement a Tendermint-based blockchain with IBC but without using the Cosmos SDK. All the pieces are there, we can even make it compatible with the SDK. But I get that it's a lot of work for not much benefit.

I don't mind using the SDK as long as it's its own Chain implementation and the Chain trait does not depend on it.

So Chain trait + CosmosSDKChain + MockChain is what we're looking for now? (I'll add this to #157 too.)

[ebuchman]

I thought that I should be able to implement a Tendermint-based blockchain with IBC but without using the Cosmos SDK.

That's right, but you'll still have to pick a "way" of doing things. It won't be the "tendermint way", because tendermint doesn't define it. Maybe one day Substrate will run on Tendermint and you can have the SubstrateTendermintChain. Or if you use Lotion instead of the CosmosSDK it will be a LotionChain, etc.

[ancazamfir]

All queries listed have been implemented