beacon.getBlockV2 in the new sszv2 can't seem to parse the response correctly

[g11tech]

Describe the bug
beacon.getBlockV2 can't seem to parse json response received from server correctly, erroring instead in the ssz v2 code.

import {networkGenesis} from "@chainsafe/lodestar-light-client/lib/networks";
import {networksChainConfig} from "@chainsafe/lodestar-config/networks";
import {getClient} from "@chainsafe/lodestar-api";
import {createIBeaconConfig} from "@chainsafe/lodestar-config";
import {fromHexString} from "@chainsafe/ssz";



const config = createIBeaconConfig(networksChainConfig.kiln, fromHexString(networkGenesis.kiln.genesisValidatorsRoot));
const api = getClient(config, {baseUrl: "https://lodestar-kiln.chainsafe.io"});

async function fetchBlock():Promise<void> {
  const data = await api.beacon.getBlockV2("0xed440b4cda86dec9055fd0129186c0f952735e479ecd47f2d4254cae07b4fe0a");
  console.log(data);
}

fetchBlock()

Code run in the new sszv2 lodestar:

bash-5.1# ts-node testBlockV2.ts 
Error: hex argument type object must be of type string
    at Object.fromHexString (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/util/byteArray.ts:19:11)
    at ByteListType.fromJson (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/type/byteArray.ts:106:19)
    at ContainerType.fromJson (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/type/container.ts:342:36)
    at ContainerType.fromJson (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/type/container.ts:342:36)
    at ContainerType.fromJson (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/type/container.ts:342:36)
    at ContainerType.fromJson (/usr/app/lodestar/node_modules/@chainsafe/ssz/src/type/container.ts:342:36)
    at Object.fromJson (/usr/app/lodestar/packages/api/src/utils/types.ts:149:44)
    at Object.request [as getBlockV2] (/usr/app/lodestar/packages/api/src/client/utils/client.ts:73:27)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async fetchBlock (/usr/app/lodestar/testBlockV2.ts:13:16)

While this works in the pre sszv2 commit:

commit 3e1f9b26fa2d5b852ab0832340e97541541cf612 (HEAD)
Author: dadepo <dadepo@gmail.com>
Date:   Sun Apr 3 21:09:35 2022 +0200

Code run in the pre sszv2:

bash-5.1# ts-node testBlockV2.ts 
{
  data: {
    message: {
      slot: 199737,
      proposerIndex: 12779,
      parentRoot: [Uint8Array],
      stateRoot: [Uint8Array],
      body: [Object]
    },
    signature: Uint8Array(96) [
      142, 242,  49,  82, 105,  13, 210,  39, 150,   1, 121,  73,
      206,  18, 132, 155, 221, 167, 163, 207,   1, 118, 139, 158,
      108, 104, 228, 196, 122, 102, 138, 240,  99,  20, 143, 231,
        1,  28, 129, 249, 172, 196, 229,  73, 176, 171,  70,  17,
       25,  39, 172, 131, 162, 108,  28,   2,  33, 197, 187, 212,
      125,   6, 169, 220,  67, 162, 114,  98,  29,  50, 211,  99,
       78, 109, 224, 229, 197, 109, 207,  30, 235, 221, 240,  25,
        7, 197, 237, 252, 223,  38, 110,  83, 126,  60, 192,  66
    ]
  },
  version: 'bellatrix'
}

[dapplion]

The execution payload returned by that URL is

        "execution_payload": {
          "parent_hash": "0x3fef46ff162d7893a4a99a290beb2a8bd990036ebcf0da1190d679381ce9cb9e",
          "fee_recipient": "0xf97e180c050e5ab072211ad2c213eb5aee4df134",
          "state_root": "0x143a76eb213e46e1e51ee78d26db32025f953b2d04f95d0b07f58d1634cfac37",
          "receipts_root": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
          "logs_bloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
          "prev_randao": "0x480057438bc0b1209ac42d55cf3180807d213a27512e5865ed8102c8bfdb83e9",
          "block_number": "214896",
          "gas_limit": "8000000",
          "gas_used": "0",
          "timestamp": "1649404344",
          "extra_data": [],
          "base_fee_per_gas": "7",
          "block_hash": "0xaa2030a604182b00e1f7bda53cc11e8100fc02d2a6f7dad387e53c77aea96b13",
          "transactions": []
        }

This JSON serialization is invalid according to current ssz version because an empty extra_data field must be 0x not []. I have tried to recreate this error with ssz-v2 but it correctly serializes an empty extra_data to 0x. What version is that node running?

It's running Lodestar/v0.35.0-beta.0/master/+35/08f98c21 it defined extra_data as List[Uint8], so empty value is []

lodestar/packages/types/src/bellatrix/sszTypes.ts

Line 49 in 4d766d1

extraData: new ListType({limit: MAX_EXTRA_DATA_BYTES, elementType: Uint8}),

[g11tech]

This JSON serialization is invalid according to current ssz version because an empty extra_data field must be 0x not []. I have tried to recreate this error with ssz-v2 but it correctly serializes an empty extra_data to 0x. What version is that node running?

It's running Lodestar/v0.35.0-beta.0/master/+35/08f98c21 it defined extra_data as List[Uint8], so empty value is []

lodestar/packages/types/src/bellatrix/sszTypes.ts

Line 49 in 4d766d1

extraData: new ListType({limit: MAX_EXTRA_DATA_BYTES, elementType: Uint8}),

updated the kiln node to latest:

@lodestar-kiln-v2:~# docker logs kiln-lodestar | grep git
Apr-08 10:05:29.915[]                 info: Lodestar version=v0.35.0-beta.0/master/+50/43111801 (git), network=mainnet

The extra data is still coming: [] as the json response of the api

[dapplion]

The extra data is still coming: [] as the json response of the api

ssz-v2 is merged only in master not latest stable. You should run 'next' tag there

[g11tech]

The extra data is still coming: [] as the json response of the api

ssz-v2 is merged only in master not latest stable. You should run 'next' tag there

its next only, 43111801 is the latest commit in master i think whiich i grepped on the log

@lodestar-kiln-v2:~# docker logs kiln-lodestar | grep git
Apr-08 10:05:29.915[]                 info: Lodestar version=v0.35.0-beta.0/master/+50/43111801 (git), network=mainnet

[dapplion]

Could you add a unit test to assert correct behavior here in Lodestar master?

[g11tech]

Could you add a unit test to assert correct behavior here in Lodestar master?

Will do, but I think I mixed up updating our kiln validating testnet node with the kiln demo node.
Sorted now, and confirm things are working. 👍

[dapplion]

@g11tech Closing as you said things are working now