ContractKit Rethinking

[aaronmgdr]

In the most recent Wallet and Dapp Council meeting in response to talks of contractkit's notorious bundle size @jmrossy brought up the idea of starting a new package. what would contract kit look like if we built it today.

Id like to come to a resolution on this by the time of Celo Connect. so that we can then make progress on implementation of new package or getting contractKit itself up to par.

[aaronmgdr]

I would like it so that you can choose which contracts you will be interacting with (or sets of contracts)

[gastonponti]

When we divided the contractKit, this was also raised as a possibility, and basically the weight added by all the wrappers and abi(s) were not that significant to make that division (taking in count that in that particular moment, we didn't have an automated way to publish packages, and making that division would probably generated more problems than solutions).
For what I understand, we have that automated publishing, right? In that case we could divide the contracts in different groups, but again, I don't really know if we are going to make a difference with this (we should check this out)

[aaronmgdr]

I did some research into this more and while I did make it possible to use most ContractWrappers standalone and created a Minimal Subset for basics. Ultimately the sort of composable choice i was thinking of was not something I was able to do

[aaronmgdr]

Firstly would be better support for tree shaking to enabled a smaller bundle size

[arthurgousset]

Adding this thread to the meeting notes here (for future reference): Celo Wallet and (d)App Council #8

[jmrossy]

If we were to start a new lib, here are some properties I'd like to see:

1. Uses Ethers.js as its engine, though maybe we can have it accept Web3 too
2. Minimal 3rd party code deps, for both size and security
3. Supports tree shaking well. Includes a CI step to bundle with webpack + rollup to test size changes
4. The connect modal (like what use-ck is today) is part of it, not a separate lib

This new lib could be branded as 'SDK lite'. Meaning you get a lib that handles the eccentricities of the Celo network, without trying to solve all problems for all people. So it would have some similarities to the current Connect package @gastonponti made.

[aaronmgdr]

would https://github.com/celo-tools/celo-ethers-wrapper be able to be used as a starting point then?

[gastonponti]

2++
I'm talking for what we had in the past, don't really know if we still have the same problem, but part of the biggest issue with the sdk packages, is that the utils have a lot of heavy libs.
Examples:

• bip39 (this was not here but I think we add this lib only for the Portuguese language that was not merged in the original lib)
• google-libphonenumber
• country-data
• crypto-js
• ethereumjs-util (yes, ethereumjs)
• web3-eth-abi (yes, also web3)
• web3-utils
• lodash

The vast majority are used for just a couple of functions. The things used by both utils (ethereumjs and web3), are just a few things to have them there, and for those cases, we could actually copy those functions (that won't probably change, 0x padding, address uppercase validation, etc). Same as lodash.
With bip39 and google-libphonenumber we are adding A LOT of size to our lib. And as all of this is part of the utils, every package using that package, will be huge. That's why we started with the @celo/base which the idea, was to have a package without dependencies, just the core utils that every package could add without worrying about its size.
The idea was to start moving things from the utils lib, to the base one, but instead of adding the deps for the external packages, add interfaces which the external libs passed as objects will have to comply. This way, the heavy stuff was only going to be injected if the user was needing to use some specific functions, and not having everything as default (as we have right now)

The idea of the @celo/connect package, was to be the simplest way to send a tx, where we will be injecting a web3 object just to have a few of the things resolved, but in the future we would be allowed to use etherjs instead (same as with the base package, create the connect object with the lib you'll be using for all the other stuff), or (the best option from my point of view, but requires more work) to list the rpc calls that the connect object was going to address. This way, we could be independent of any lib (or, if the dev wants, inject the one its like).

So, what is the problem right now? We started the work to make that division, but we didn't continue with that work. Today, the connect packages has the utils one inside, and as all the contracts will require the connect one, the bundle size of the utils will still bump the numbers.

My recommendation is to try to extract all the functions that the connect package uses from the utils one, into the base package (without any external lib). This way, the connect lib will be A LOT lighter. And then, add the ability to inject a ethersjs provider instead of the web3 one, just to allow the user to choose what they want.
And then, think of populate the functions that we are consuming directly from web3 into the connect package, this way we could have something completely agnostic.

TL;DR; Destroy the utils package, add some love to the connect package, and we would probably have the same results with less resources

[aaronmgdr]

Ive been building off @gastonponti's work.

• removed google-libphonenumber and country-data as dependencies by moving functions that need them to a new phone-utils package which is not needed by contractKit

• removed bip32 and bip39 bls12377js and some more by moving functions to a new cryptographic-utils package that is not needed by contractKit

• lodash must have been removed before I started

I haven't removed ethereum-utils. it didn't pop up as being the big culpre since it looks like most of its deps are shared with web3-utils. still would be good to though.

After removing and cutting package size in ContractKit itself is now the biggest source of weight (25%). this comes from the generated Contracts and the Wrappers.

At this point the only things I can see really making a difference are

A) going all in and swapping out web3 for ethers js.

B) removing wrappers and contracts not used internal and having some separate package(s) for them

C) using esm modules. although they don't seem to be supported by ts yet.

Hard parts about ethers are that A) since we exposed web3 anyone relying on that is gonna be very broken
B) changing internals.
C) the tools that generates the ABIs and interfaces for contracts is tied to web3 (probably an ethers version though?)

[aaronmgdr]

@celo/connect is better but still quiet large due to web3 dep. Also i think only a few people know it can be used. It needs more docs / readme which i plan to do.

As much progress As i've made in all this I do think @jmrossy is right about a separate package if we want it to be as tiny as possible. Something that combines use-contractkit except contractkit gets replaced with celo-ethers-wrapper.

And yes This would be browser focused and lightweight.

[vibern0]

Something that combines use-contractkit except contractkit gets replaced with celo-ethers-wrapper

That's my opinion. I don't want anything else other than a generic wallet connector and be able to set fee transactions. @celo/connect is working well for setting fees.

[vibern0]

For me (as a developer), the only things I care about is to connect/disconnect to a wallet, get the user address, signer and provider (so I know if on the wrong network) and be able to set fee currency.

[aaronmgdr]

MY worry with starting from scratch is that a completely changed interface would mean dapp developers are forced to spend time upgrading. which I know is rarely a pleasant experience.

[aaronmgdr]

I have Opened a PR for v2 of ContractKit. It manages to reduce the size considerably. while keeping breaking changes to a minimum.

The most exciting part Is that I was able due to refactoring to create a MiniContractKit which has a similar but reduced interface to ContractKit and gives access to basic contracts like GasPriceMinimum, Account, the StableTokens, Exchange and Celo Token Contracts.

I think for most dapp projects upgrading will greatly improve bundlesize, and changing imports to @celo/contractkit/lib/mini-kit will do even more.

The PR Description has more info.

We are looking at having use-contractkit default to use mini-kit. Eventually @dckesler and I were also discussing the idea of a useCK with CK replaced with celo-ethers-wrapper

#9127

[jmrossy]

@aaronmgdr Nice!!
One bit of feedback: Can the miniKit path be @celo/contractkit/minimal or @celo/contractkit/basic or something like that (no /lib and no hyphen)?
Even better would be @celo/sdk/XXX but I understand that would be a new library

[aaronmgdr]

at this moment Im not sure how to not have the lib in there but would def be happy to have it gone.

[aaronmgdr]

PS I wrote an upgrade guide (its under reivew) celo-org/docs#307

[aaronmgdr]

BTW ALL We plan on releasing contractKit v2 in the next few days.