v6 Breaking Changes

[kclowes]

Now that v5 stable is out, it's time to start collecting a list of breaking changes that we might like to see in v6. I'll update this issue as they come up.

Outstanding breaking tasks:

• Python 3.11 Support - Python 3.11 Support #2699
• Remove Infura auto provider logic from within web3 and make a template for new auto providers to use. See issue Autoprovider repo template #1736 (Remove auto provider except for local geth dev #2706)
• Remove SolidityError in favor of the more generic ContractLogicError. See issue Remove SolidityError #1901
• Remove camelCase function names in favor of snake_case. See Snake case remaining main API methods and properties #2598
• Remove Parity module (via Remove parity namespace and refs #2718)
• Remove version module (see Remove version module #2729)
• Make ipfshttpclient opt-in. (See Make ipfshhtpclient an optional library / backend for ethpm #2730 which closes ipfshttpclient is unmaintained #2629)
• Remove start_rpc and stop_rpc from geth.admin module (see Remove deprecated start_rpc and stop_rpc from w3.geth.admin module #2731)
• Standardize apply_formatter_to_array functions. See Standardize apply_formatter_to_array  #1902
• Custom web3py exceptions inherit from Web3Exception. See Add Web3Exception to all exception classes #1478
• Enable strict bytes length checking by default. See issue Flag for Stricter ABI Type Checking #1419 and soliditySha parameter encoding issue #1154.
• Don't allow users to pass bytestrings to contract functions if the bytestring is the wrong length. As it stands, if a bytestring is shorter than the required length, we pad it. If it is longer, we throw an error. See issue Enable strict byte length checking by default #1903.
• Raise specific web3 error rather than ValueError when method isn't available on the node: Raise custom exception instead of ValueError when a method is unavailable at the node #2448
• Standardize default value for block_identifier in eth.call and ContractFunctions.call. Change made in contract.call() block_identifier default change #2335 already, just needs tests.
• Opt in or out of dict to AttributeDict conversion via middleware choice. See Typing of AttributeDict return values is broken #1656.
• Rich tuple decoder Add rich tuple decoder #1353
• Clean up Async typing

shouldn't be breaking:

• Clean up AsyncHTTPProvider instantiation. - See Clean up AsyncHTTPProvider instantiation #2736
• Rearrange web3/eth.py into separate file. See Organize eth module as separate files #2753
• Remove dependency upper limits - Remove dep upper pins #2648
• Rearrange contract.py into separate files. See reorganized contract #2567
• v6 Migration Guide

[kclowes]

All custom web3 exceptions should inherit from Web3Exception. See #1478

[kclowes]

The eth-utils and web3 apply_formatter_to_array functions are slightly different. We should standardize on one or the other (probably the eth-utils version). The apply_formatter_to_array function in web3 converts the return value to an array and the eth-utils version does not. See issue #1902

[carver]

Looks like the #1419 plan is to enable strict bytes length checking by default. At which point I think #1154 could be closed.

[kclowes]

Remove Infura auto provider logic from within web3 and make a template for new auto providers to use. See issue #1736

[wolovim]

Whisper is officially out of Geth distros (latest release notes), so we probably ought to drop support too.

Not necessary to wait for v6. Support removed in #1791.

[wolovim]

v6 is a probably a good time to reflect on some client namespace topics, given:

• Parity is now OpenEthereum (and has been for a while)
• Both Parity and OE clients are in the wild
• Soon™, OE is set to diverge significantly from Parity, see: recent blog post
• Other clients on the sidelines: Nethermind, Trinity, Turbo-Geth, ...?
• Desire to encourage client diversity (related issue: Client diversity and Web3.py #1694)

[kclowes]

The SolidityError that was introduced in #1585 should be more generic since more than one language has an assert/revert feature and we want to support more than just Solidity. See #1901

[kclowes]

Drop python 3.6 support. Python 3.6 will no longer be supported after 12/23/21. See: https://endoflife.date/python

[kclowes]

Remove camelCase methods in eth and geth modules. Consider making buildTransaction -> build_transaction, estimateGas -> estimate_gas

[kclowes]

Move websockets dependency to 10.0+ which means we can get rid of loop argument deprecation warnings. Dependent on dropping python 3.6

[kclowes]

Standardize default value for block_identifier in eth.call and ContractFunctions.call. Change made in #2335 already, just needs tests.

[kclowes]

Add black

[wiseaidev]

It would be great to integrate pre-commit within tox to make the dev process faster & easier in a way that developers are not required to manually fix linting issues. To do so, The following modifications would be introduced to tox.ini:

[testenv:lint]
basepython=python3
skip_install = true
deps = pre-commit
commands=
    pre-commit run --all-files --show-diff-on-failure

instead of:

[testenv:lint]
basepython=python
extras=linter
commands=
    flake8 {toxinidir}/web3 {toxinidir}/ens {toxinidir}/ethpm {toxinidir}/tests --exclude {toxinidir}/ethpm/ethpm-spec
    isort --recursive --check-only --diff {toxinidir}/web3/ {toxinidir}/ens/ {toxinidir}/ethpm/ {toxinidir}/tests/
    mypy -p web3 -p ethpm -p ens --config-file {toxinidir}/mypy.ini

And then add code formatters and fixers in the pre-commit config file(.pre-commit-config.yaml) as local hooks like:

  - repo: local
    hooks:
      - id: flake8
        name: Check flake8 syntax
        entry: flake8 web3 ens ethpm tests --exclude ethpm/ethpm-spec
        pass_filenames: false
        language: system
        types: [python]
      - id: isort
        name: Sort imports alphabetically
        entry: isort --recursive --check-only --diff web3 ens ethpm tests
        pass_filenames: false
        language: system
        types: [python]
      - id: mypy
        name: Check mypy static types match
        entry: mypy -p web3 -p ethpm -p ens --config-file mypy.ini
        pass_filenames: false
        language: system
        types: [python]

To add black as @kclowes did in PR:#2345, just add a new local hook in a similar fashion to the previous ones:

      - id: black
        name: Run black code formatter
        entry: black ethpm --exclude ethpm/ethpm-spec web3 ens tests --check
        pass_filenames: false
        language: system
        types: [python]

pre-commit is equipped with useful built-in fixers to automatically format files given certain properties such as:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.1.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-json
      - id: pretty-format-json
        args: ['--autofix']
      - id: debug-statements
      - id: mixed-line-ending
        args: ['--fix=lf']
        description: Forces to replace line ending by the UNIX 'lf' character.
      - id: forbid-new-submodules

The good news is that these modifications are backward compatible with the current make and tox configurations, running $ make lint , however, would generate something similar to the following:

What do you think about adding pre-commit?
If y'all agree, I will be more than happy to open a PR.

[dbfreem]

@kclowes I made the change for buildTransaction build_transaction and estimateGas -> estimate_gas in the asyncify contract branch. I did deprecate the camelCase version but maybe since this is a move to v6 they could just be removed??

Also, I can take a pass at removing all the deprecated camelCase methods from geth and eth.

[kclowes]

Thank you so much, @dbfreem!

I did deprecate the camelCase version but maybe since this is a move to v6 they could just be removed??

They can be removed in v6 (master), but we'll need to add a deprecation warning in the v5 branch for the camelCase version.

Also, I can take a pass at removing all the deprecated camelCase methods from geth and eth.

That would be great! I've been looking forward to getting that done!

[dbfreem]

Could ConciseContract and ImplicitContract be removed as part of v6?

[kclowes]

Yes! Thanks for the reminder!

[kclowes]

Explore raising a web3 specific error message rather than ValueError when method isn't available on the node: #2448

[dbfreem]

Should parity be removed as part of v6? It doesn't have a DeprecationWarning saying that but the docs say that Parity will be removed in v6. If we remove it I am assuming like other things we would add the DeprecationWarning to v5 and remove in v6.

Thoughts??

[kclowes]

Remove version module. All methods are implemented elsewhere.

[kclowes]

#2629 - make ipfshttpclient opt-in

[MartinThoma]

I see all checkmarks here set 🎉🥳

[MartinThoma]

I see that this is closed and v7 is released in planning, but web3==6.0.0 is not yet on PyPI: https://pypi.org/project/web3/#history

Do you know when it will be? How is the procedure to decide when this will be?

I'm very excited about it, as web3 is currently blocking one of my projects from moving to Python 3.11 😅

[fselmo]

Hey @MartinThoma, we are in a code merge freeze and late testing phase just to make sure we don't need to make minor adjustments. If there are no complications, v6.0.0 should be right around the corner 🙂.

To clarify, what do you mean by v7 is released? We've opened an issue to start capturing future breaking changes that will go into v7 but that is the only v7-related content at the moment.

[menaitm]

Hey @fselmo, please can you publish the code-frozen v6.0.0 to PyPI as a 6.0.0-beta.12 or rc? I'd like to test against the code most likely to be released in v6

Thanks!

[kclowes]

The code frozen version is up as v6.0.0-beta.11. Let us know if you see anything!