Feature Request: Incremental Compilation

[brockelmore]

In large repos with many contracts, compilation can be the longest run time in the testing process. Since the compiler tries to be stateless, but dapptools does not, it would be helpful to watch for changes and determine if a contract needs to be recompiled. With how closely dapptools integrates git, I don't imagine this would be a massive lift.

[MrChico]

I dont see how this can be done from dapp rather than solc

[brockelmore]

Chris from the lang team seems to insist that it is devtool responsibility and that the compiler will not be adding it

[d-xo]

I think to do that we would need access to the full import tree? The compiler does not expose this afaik and building this ourselves seems like a non trivial piece of work.

With how closely dapptools integrates git, I don't imagine this would be a massive lift.

@brockelmore how do you see this working? Perhaps I'm missing something.

[brockelmore]

i mean there is the simple version and the complex version. the simple version is very dumb and basically is: changed & imported elsewhere? rebuild all contracts that depend on it. !changed ? don't rebuild. basically just a filter on the list of contracts to compile that if we have stored compilation do the simple thing and just exclude that from the contracts to build. this helps with monorepos as most of the time you don't want to actually compile every contract (and don't need to), but tools are such that it is time consuming to specify the specific contract. if it has to rebuild all dependencies that fine, a single contract is generally quick because of bytecode limits anyway

[MrChico]

It seems like to be able to do this we would need to at least:

• store a copy (or hash) of the compiler input settings to see when they change
• analyze the inheritance tree before compiling (flatten can only analyze it from the already compiled version, we would need to start parsing raw solidity)
• introduce a way to merge compiler output jsons in a safe manner. This is useful in other contexts and has been discussed before merge multiple solc files #631.

With this we can compare the hash of the srccode and only recompile the necessary files in the way you describe.

If solc could give us just the inheritance tree, that would help a lot I think.

[d-xo]

opened a solc issue regarding the import graph: ethereum/solidity#11716

[d-xo]

ok so it seems like we can use solc to parse the source files and then build the import graph without too much effort by processing the parsed soliditiy AST.