Tag v1.0.2 cannot create an extrinsic using the Utility.batch call using polkadot v.0.9.11

[shedfull]

The following code works when connecting to polkadot 0.9.10 but not 0.9.11

Fails when trying compose the batch call

from substrateinterface import SubstrateInterface, Keypair
from substrateinterface.exceptions import SubstrateRequestException

try:
    substrate = SubstrateInterface(
        url="ws://127.0.0.1:9944",
        type_registry_preset='development'
    )
except ConnectionRefusedError:
    print("⚠️ No local Substrate node running, try running 'start_local_substrate_node.sh' first")
    exit()

keypair = Keypair.create_from_uri('//Alice')

call_1 = {
    'call_module': 'Balances',
    'call_function': 'transfer',
    'call_args': {
        'dest':  '5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty',
        'value': 1 * 10**15
    }
}

call_2 = {
    'call_module': 'Balances',
    'call_function': 'transfer',
    'call_args': {
        'dest':  '5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty',
        'value': 1 * 10**15
    }
}

batch_call = substrate.compose_call(
    call_module='Utility',
    call_function='batch',
    call_params={
        'calls': [call_1, call_2]
    }
)

batched_extrinsic = substrate.create_signed_extrinsic(
    call=batch_call,
    keypair=keypair,
    era={'period': 64}
)


try:
    receipt = substrate.submit_extrinsic(batched_extrinsic, wait_for_inclusion=True)

    print('Extrinsic "{}" included in block "{}"'.format(
        receipt.extrinsic_hash, receipt.block_hash
    ))

    if receipt.is_success:

        print('✅ Success, triggered events:')
        for event in receipt.triggered_events:
            print(f'* {event.value}')

    else:
        print('⚠️ Extrinsic Failed: ', receipt.error_message)


except SubstrateRequestException as e:
    print("Failed to send: {}".format(e))

Stacktrace is :

Traceback (most recent call last):
  File "thing2.py", line 36, in <module>
    batch_call = substrate.compose_call(
  File "...pyenvs/core2/lib/python3.8/site-packages/substrateinterface/base.py", line 1422, in compose_call
    call.encode({
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/base.py", line 698, in encode
    self.data = self.process_encode(self.value_serialized)
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/types.py", line 1428, in process_encode
    data += arg_obj.encode(param_value)
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/base.py", line 698, in encode
    self.data = self.process_encode(self.value_serialized)
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/types.py", line 846, in process_encode
    data += element_obj.encode(element)
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/base.py", line 698, in encode
    self.data = self.process_encode(self.value_serialized)
  File "...pyenvs/core2/lib/python3.8/site-packages/scalecodec/types.py", line 1055, in process_encode
    raise ValueError("Value for enum with type_mapping can only have one value")
ValueError: Value for enum with type_mapping can only have one value

[mario-sangar]

Confirmed it also happens with tag 1.0.3 and scalecodec-v1.0.11

[arjanz]

Release fix in https://github.com/polkascan/py-scale-codec/releases/tag/v1.0.12

[BulatSaif]

The scrip provided in issue is still failing:

ValueError: Value for enum with type_mapping can only have one value

I use latest version:

$ pip freeze | grep scal
scalecodec==1.0.43
$ pip freeze | grep sub
substrate-interface==1.3.2

How can I make it work?

[arjanz]

The scrip provided in issue is still failing:

ValueError: Value for enum with type_mapping can only have one value

I use latest version:

$ pip freeze | grep scal
scalecodec==1.0.43
$ pip freeze | grep sub
substrate-interface==1.3.2

How can I make it work?

• What is the spec_name and spec_version of the node you are connecting with?
• What happens if you omit the type_registry_preset='development' during init?

[BulatSaif]

• What is the spec_name and spec_version of the node you are connecting with?

system.lastRuntimeUpgrade
{
  specVersion: 9,300
  specName: rococo
}

I did more tests, Utility.batch works on v0.9.29 but not on v0.9.30-rc or master.
You can reproduce issue locally by using this docker-compose file:

version: '3.7'
services:
  node_alice:
    #image:  &polkadotimage paritypr/polkadot-debug:v0.9.29-94078b44 # works
    image:  &polkadotimage paritypr/polkadot-debug:v0.9.30-rc1-df2c1b63 # ValueError: Value for enum with type_mapping can only have one value
    #image:  &polkadotimage paritypr/polkadot-debug:master-c620450d # ValueError: Value for enum with type_mapping can only have one value
    ports:
      - "9944:9944"
    command: >
      --chain=rococo-local
      --ws-port 9944
      --rpc-cors all
      --ws-external
      --alice
      --tmp

  node_bob:
    image: *polkadotimage
    command: --chain=rococo-local --bob --tmp

1. docker-compose up
2. run script provided in this issue

• What happens if you omit the type_registry_preset='development' during init?

Error still present

[BulatSaif]

I notice that Session.set_keys is also don't work as before:
This script works with runtime 9290 but not with 9300

from substrateinterface import SubstrateInterface, Keypair
from substrateinterface.exceptions import SubstrateRequestException

try:
    substrate = SubstrateInterface(
        url="ws://127.0.0.1:9944",
        type_registry_preset='development'
    )
except ConnectionRefusedError:
    print("⚠️ No local Substrate node running, try running 'start_local_substrate_node.sh' first")
    exit()

keypair = Keypair.create_from_uri('//Alice')
session_key = substrate.rpc_request(method="author_rotateKeys", params=[])['result']

if type(session_key) == str:
    type_id = substrate.get_metadata_call_function('Session', 'set_keys')['fields'][0]['type']
    session_key = substrate.decode_scale("scale_info::{}".format(type_id), session_key)

call = substrate.compose_call(
    call_module='Session',
    call_function='set_keys',
    call_params={
        'keys': session_key,
        'proof': 'None'
    }
)

extrinsic = substrate.create_signed_extrinsic(
    call=call,
    keypair=keypair,
)

receipt = substrate.submit_extrinsic(extrinsic, wait_for_inclusion=True)

if receipt and receipt.is_success: # <= this line fails
   print(True)
else:
   print(False)

Error:

Traceback (most recent call last):
  File "session_rotate.py", line 36, in <module>
    if receipt and receipt.is_success:
  File "/home/ubuntu/.local/lib/python3.8/site-packages/substrateinterface/base.py", line 3539, in is_success
    self.process_events()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/substrateinterface/base.py", line 3388, in process_events
    if self.triggered_events:
  File "/home/ubuntu/.local/lib/python3.8/site-packages/substrateinterface/base.py", line 3381, in triggered_events
    for event in self.substrate.get_events(block_hash=self.block_hash):
  File "/home/ubuntu/.local/lib/python3.8/site-packages/substrateinterface/base.py", line 1526, in get_events
    storage_obj = self.query(module="System", storage_function="Events", block_hash=block_hash)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/substrateinterface/base.py", line 1462, in query
    obj.decode()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/scalecodec/base.py", line 677, in decode
    self.value_serialized = self.process()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/scalecodec/types.py", line 830, in process
    element = self.process_type(self.sub_type, metadata=self.metadata)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/scalecodec/base.py", line 760, in process_type
    obj.decode(check_remaining=False)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/scalecodec/base.py", line 677, in decode
    self.value_serialized = self.process()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/scalecodec/types.py", line 2765, in process
    'module_id': value['event']['module_id'],
KeyError: 'module_id'

[arjanz]

@BulatSaif errors confirmed, after some debugging I noticed the path used for a Call and Event in the PortableRegistry has been changed.. I will need to examine the reason behind these changes in the Substrate codebase, as this will have some more impact.

[arjanz]

@BulatSaif Found the breaking change in Substrate: paritytech/substrate#11981

[arjanz]

Issue will be addressed in polkascan/py-scale-codec#80

[arjanz]

@BulatSaif Issue should be resolved after you update scalecodec==1.0.44 Could you confirm that?

[BulatSaif]

Wow, that was fast!
Thank you @arjanz, all works.
This issue can be closed