Integration tests for lightwalletd RPCs that are only used when a wallet connects

[conradoplg]

Motivation

lightwalletd waits to call some RPC methods until a wallet connects to lightwalletd.

We need to test how lightwalletd actually uses these methods as part of our integration tests.

Specifications

Tasks

1. Work out which RPCs aren't covered by existing tests:
   • empty state integration test
   • Add lightwalletd full sync integration test #3511
   • Send transactions in lightwalletd integration tests #3512
2. Work out how to test live RPCs that are only used when a wallet connects.

We think these RPCs are only used when a wallet connects, but there might be more:

• getrawtransaction
• getaddressutxos
• getrawmempool

Related Work

[ftm1000]

Hey team! Please add your planning poker estimate with ZenHub @conradoplg @dconnolly @gustavovalverde @jvff @oxarbitrage @teor2345 @upbqdn

[oxarbitrage]

I did some tests today with 2 different available wallets.

The setup is:

• Zebra with rpc server enabled listening at 127.0.0.1:8232
• Lightwalletd with Zebra as the backend. Lightwalletd will listen at 127.0.0.1:9067 by default.

zecwallet-light-cli

I didnt tested everything specially i did not tested sending transactions which will be a good test to do here. But, most commands work even if we dont have API calls, they return empty results, for example here are few executions (note that the wallet crated some addresses at the first init):

(main) Block:455184 (type 'help') >> list
[]
(main) Block:455333 (type 'help') >> sendprogress
{
  "id": 0,
  "sending": false,
  "progress": 0,
  "total": 0,
  "txid": null,
  "error": null
}
(main) Block:455475 (type 'help') >> height
{
  "height": 455502
}
(main) Block:455502 (type 'help') >> lasttxid
{
  "last_txid": null
}
(main) Block:455603 (type 'help') >> balance
{
  "zbalance": 0,
  "verified_zbalance": 0,
  "spendable_zbalance": 0,
  "unverified_zbalance": 0,
  "tbalance": 0,
  "z_addresses": [
    {
      "address": "zs1tqe6dujqkce2pazp3433uvarvhf2sgmm893k37faz8nzxt2fa7y0hvrhzxhceshgg3suv2p4juk",
      "zbalance": 0,
      "verified_zbalance": 0,
      "spendable_zbalance": 0,
      "unverified_zbalance": 0
    }
  ],
  "t_addresses": [
    {
      "address": "t1fj6ktLBU2cn8qwezHZxZwRiRBxYZVvCiW",
      "balance": 0
    }
  ]
}
(main) Block:456019 (type 'help') >> syncstatus
{
  "sync_id": 318,
  "in_prorgess": false,
  "last_error": null
}
(main) Block:456400 (type 'help') >> addresses
{
  "z_addresses": [
    "zs1tqe6dujqkce2pazp3433uvarvhf2sgmm893k37faz8nzxt2fa7y0hvrhzxhceshgg3suv2p4juk"
  ],
  "t_addresses": [
    "t1fj6ktLBU2cn8qwezHZxZwRiRBxYZVvCiW"
  ]
}
(main) Block:456652 (type 'help') >> quit

All that looks good, but i wanted to test ZecWallet Lite

When i do the same setup i get this error in zecwallet lite when connecting to my lightwalletd backend:

Error Initializing Lightclient
Error: Error with response: Status { code: Internal, message: "Unexpected compression flag: 60, while receiving response with status: 504 Gateway Timeout", metadata: MetadataMap { headers: {"server": "nginx/1.14.0 (Ubuntu)", "date": "Wed, 06 Apr 2022 22:10:32 GMT", "content-type": "text/html", "content-length": "192"} } }

Which seems to be a gRPC issue but i am not totally sure.

Any idea on what can be going on ? We can maybe ask @adityapk00 about it ?

It could be that i am not totally in sync yet with zebra ?

As an alternative i might try ZecWallet Full Node but we probably want to make it work with ZecWallet Lite as well.

[adityapk00]

This definitely looks like a grpc error although I’m not sure what exactly it is. What I’ve done in this situation is to put nginx in the middle, between ZecWallet lite and the LightwalletD server. Nginx does a very good job of logging everything, and makes it easier to find out what part of the grpc is not working.

…

On Wed, Apr 6, 2022 at 6:07 PM, Alfredo Garcia ***@***.***> wrote: I did some tests today with 2 different available wallets. The setup is: - Zebra with rpc server enabled listening at 127.0.0.1:8232 - Lightwalletd with Zebra as the backend. Lightwalletd will listen at 127.0.0.1:9067 by default. Wallet 1: [zecwallet-light-cli](https://github.com/adityapk00/zecwallet-light-cli) I didnt tested everything specially i did not tested sending transactions which will be a good test to do here. But, most commands work even if we dont have API calls, they return empty results, for example here are few executions (note that the wallet crated some addresses at the first init): (main) Block:455184 (type 'help') >> list [] (main) Block:455333 (type 'help') >> sendprogress { "id": 0, "sending": false, "progress": 0, "total": 0, "txid": null, "error": null } (main) Block:455475 (type 'help') >> height { "height": 455502 } (main) Block:455502 (type 'help') >> lasttxid { "last_txid": null } (main) Block:455603 (type 'help') >> balance { "zbalance": 0, "verified_zbalance": 0, "spendable_zbalance": 0, "unverified_zbalance": 0, "tbalance": 0, "z_addresses": [ { "address": "zs1tqe6dujqkce2pazp3433uvarvhf2sgmm893k37faz8nzxt2fa7y0hvrhzxhceshgg3suv2p4juk", "zbalance": 0, "verified_zbalance": 0, "spendable_zbalance": 0, "unverified_zbalance": 0 } ], "t_addresses": [ { "address": "t1fj6ktLBU2cn8qwezHZxZwRiRBxYZVvCiW", "balance": 0 } ] } (main) Block:456019 (type 'help') >> syncstatus { "sync_id": 318, "in_prorgess": false, "last_error": null } (main) Block:456400 (type 'help') >> addresses { "z_addresses": [ "zs1tqe6dujqkce2pazp3433uvarvhf2sgmm893k37faz8nzxt2fa7y0hvrhzxhceshgg3suv2p4juk" ], "t_addresses": [ "t1fj6ktLBU2cn8qwezHZxZwRiRBxYZVvCiW" ] } (main) Block:456652 (type 'help') >> quit All that looks good, but i wanted to test [ZecWallet Lite](https://www.zecwallet.co/index.html) When i do the same setup i get this error in zecwallet lite when connecting to my lightwalletd backend: Error Initializing Lightclient Error: Error with response: Status { code: Internal, message: "Unexpected compression flag: 60, while receiving response with status: 504 Gateway Timeout", metadata: MetadataMap { headers: {"server": "nginx/1.14.0 (Ubuntu)", "date": "Wed, 06 Apr 2022 22:10:32 GMT", "content-type": "text/html", "content-length": "192"} } } Which seems to be a gRPC issue but i am not totally sure. Any idea on what can be going on ? We can maybe ask ***@***.***(https://github.com/adityapk00) about it ? It could be that i am not totally in sync yet with zebra ? As an alternative i might try [ZecWallet Full Node](https://www.zecwallet.co/fullnode.html) but we probably want to make it work with ZecWallet Lite as well. — Reply to this email directly, [view it on GitHub](#3655 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AHUDXBORW6C4ZU75LG44HIDVDYKLPANCNFSM5PLL3MNQ). You are receiving this because you were mentioned.Message ID: ***@***.***>

[oxarbitrage]

I will try that and update how it went, thank you @adityapk00

[oxarbitrage]

Just an update here, i was able to get the nginx proxy working with the following config (ubuntu based linux):

server {
        listen       8081 http2;
        server_name  localhost;
        access_log   /var/log/nginx/lightwalletd.access.log;
        error_log   /var/log/nginx/lightwalletd.error.log;

        location / {
                include       /etc/nginx/mime.types;
                grpc_pass grpc://localhost:9067;
        }
}

The cli wallet is still working now connecting to port 8081 instead of directly 9067. I can see the requests in the access.log of nginx, for example a cli wallet info command will query:

127.0.0.1 - - [13/Apr/2022:11:14:45 -0300] "POST /cash.z.wallet.sdk.rpc.CompactTxStreamer/GetLightdInfo HTTP/2.0" 200 162 "-" "tonic/0.4.3"
127.0.0.1 - - [13/Apr/2022:11:14:45 -0300] "POST /cash.z.wallet.sdk.rpc.CompactTxStreamer/GetLatestBlock HTTP/2.0" 200 43 "-" "tonic/0.4.3"
127.0.0.1 - - [13/Apr/2022:11:14:48 -0300] "POST /cash.z.wallet.sdk.rpc.CompactTxStreamer/GetMempoolStream HTTP/2.0" 200 0 "-" "tonic/0.4.3"

Now for zecwallet lite we need to be in sync or the app will fallback to one of the default servers, from the logs of lightwalletd i can see:

2022-04-13T11:15:56.399444084-03:00 WARN::Server's latest block(502444) is behind ours(1631939)
2022-04-13T11:15:57.769178871-03:00 INFO::Mempool monitoring starting

So as soon as i get my local zebra node sync i should be able to get what api calls are called when zecwallet lite connects to a lightwalletd server backed by zebra. I will post this info as soon as i get it.

[conradoplg]

We decided that we will follow the approach from #4068 and build gRPC requests manually, so we don't need zewallet for this (that will become another ticket)