[CLI] Rewrite solc --link to use LinkerObject

[cameel]

Extracted from #10297 (comment):

@axic

Why do we have this messy code here and why isn't this part of LinkerObject? Also solc-js has this messy code duplicated.

I would prefer a clear way of adding a LinkerObject::parse(bytes) feature and unit testing, and using LinkerObject::parse(bin).link(addresses) in the cli.

You can take inspiration from https://github.com/ethereum/solc-js/blob/master/linker.js (see findLinkReferences).

[Uttam-Singhh]

Is this issue solved or I can work on it??

[cameel]

No, no one is working on it right now so if you want to try, feel free :)

[Uttam-Singhh]

Hey, @cameel I am new to this.
It would be great if you can guide me on how to get started and what I exactly have to do!

[cameel]

Great! Not sure how familiar you are with linking and library addresses in Solidity. If it's a new topic for you, I'd suggest to start by reading the Library Linking section and the chapter on Libraries in the docs.

Here's a general rundown of everything you need for this particular task:

• First, to get an idea what this is all about, try to compile a contract that uses a library and see the resulting bytecode. For example

  solc test/libsolidity/semanticTests/libraries/stub.sol --bin

  ======= test/libsolidity/semanticTests/libraries/stub.sol:C =======
  Binary:
  608060405234801561001057600080fd5b50610217806100206000396000f3fe608060405234801561001057600080fd5b506004361061002b5760003560e01c8063e420264a14610030575b600080fd5b61004a6004803603810190610045919061011a565b610060565b60405161005791906101a5565b60405180910390f35b600073__$b3157e226f2c4dddae135e6cab4ed4e747$__63b3de648b836040518263ffffffff1660e01b8152600401610099919061018a565b60206040518083038186803b1580156100b157600080fd5b505af41580156100c5573d6000803e3d6000fd5b505050506040513d601f19601f820116820180604052508101906100e99190610143565b9050919050565b6000813590506100ff816101ca565b92915050565b600081519050610114816101ca565b92915050565b60006020828403121561012c57600080fd5b600061013a848285016100f0565b91505092915050565b60006020828403121561015557600080fd5b600061016384828501610105565b91505092915050565b610175816101c0565b82525050565b610184816101c0565b82525050565b600060208201905061019f600083018461017b565b92915050565b60006020820190506101ba600083018461016c565b92915050565b6000819050919050565b6101d3816101c0565b81146101de57600080fd5b5056fea264697066735822122020a8bc2e01963760c213821310bc9bea51a3cfcc7dc46771171a6028adddb66b64736f6c63430008040033
  
  // $b3157e226f2c4dddae135e6cab4ed4e747$ -> test/libsolidity/semanticTests/libraries/stub.sol:L
  
  ======= test/libsolidity/semanticTests/libraries/stub.sol:L =======
  Binary:
  6101d4610053600b82828239805160001a607314610046577f4e487b7100000000000000000000000000000000000000000000000000000000600052600060045260246000fd5b30600052607381538281f3fe73000000000000000000000000000000000000000030146080604052600436106100355760003560e01c8063b3de648b1461003a575b600080fd5b81801561004657600080fd5b50610061600480360381019061005c91906100a1565b610077565b60405161006e91906100d9565b60405180910390f35b6000818261008591906100f4565b9050919050565b60008135905061009b81610187565b92915050565b6000602082840312156100b357600080fd5b60006100c18482850161008c565b91505092915050565b6100d38161014e565b82525050565b60006020820190506100ee60008301846100ca565b92915050565b60006100ff8261014e565b915061010a8361014e565b9250817fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff048311821515161561014357610142610158565b5b828202905092915050565b6000819050919050565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052601160045260246000fd5b6101908161014e565b811461019b57600080fd5b5056fea26469706673582212203a397998c84d74f23a1af49ff538f5a353039460fe00aa622d24ba9035d8743664736f6c63430008040033
  

  • Note the __$b3157e226f2c4dddae135e6cab4ed4e747$__ part inside C's bytecode (you need to scroll to see it above). It's a placeholder for a library address. Linking in Solidity simply means inserting an actual library address in that spot. It takes up exactly 40 characters in the hex encoding that represent 20 bytes in the decoded bytecode.
  • To make this more apparent the compiler lists all the placeholders below the bytecode, which tells you the exact name of the library and the file.
• Try telling the compiler where that library has been deployed on the blockchain:

  solc test/libsolidity/semanticTests/libraries/stub.sol --bin --libraries test/libsolidity/semanticTests/libraries/stub.sol:L=0x1234567890123456789012345678901234567890

  • This way you told the compiler to build and immediately link the contract (all in one operation).
  • Note that that the placeholder has disappeared from the produced bytecode. It has been replaced with the address you gave.
• You can also perform linking on an already compiled binary using the --link option:

  solc test/libsolidity/semanticTests/libraries/stub.sol --bin > bytecode.bin
  solc bytecode.bin --link --libraries test/libsolidity/semanticTests/libraries/stub.sol:L=0x1234567890123456789012345678901234567890

  • Note that in the --link mode the compiler modifies the bytecode in place. To see the result you need to open bytecode.bin.
• The issue here is that these two methods are implemented separately.
  • The first one uses LinkerObject::link(). See LinkerObject.h/LinkerObject.cpp.
  • The second uses CommandLineInterface::link(). See CommandLineInterface.cpp.
• The goal is to rewrite CommandLineInterface::link() to use LinkerObject::link() internally.
• Note an important detail: LinkerObject operates on binary bytecode (that's what LinkerObject::bytecode stores) while the binary CommandLineInterface::link() gets as input is hex-encoded - except for the placeholders. So you need to detect the placeholders and decode everything except them. Only then you can create a LinkerObject and use its link() method.

If you run into problems with the the implementation, please drop by on the solidity-dev channel. During the week you can find me and other developers there (though note that this Monday is a holiday so probably best to try on Tuesday :)).

[Uttam-Singhh]

Thank you! so much @cameel!
I will start working on this as soon as possible...

[cameel]

@Uttam-Singhh Are you still working on this issue? Do you need any help?

[Uttam-Singhh]

Hey @cameel
Sorry, I was not able to work on this, I tried to learn the concepts but I think so I need to learn more to work on this.
I am unassigning myself. Sorry for the inconvenience!
Looking forward to working on more issues in the future.
Thank you so much for you guidance @cameel!

[cameel]

No worries! If you want, you could try a different task, maybe it'll be easier for a start. Here are some possibilities:

• [yulopti] interactively run one iteration of the sequence between [] from fullsuite #10812 - this is a simple tool that consists of a single .cpp file. Should be pretty easy.
• [CLI] Report errors for command line options that have no effect #11629 - in this there are will probably be a lot of changes but they should be all simple (just adding checks and printing errors) and you can also just do them one at a time to make PRs simpler.
• [CLI] Allow for printing output without headings (such as bytecode) #10275
• Make usage of quotes in errors consistent #7578