Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ganache-cli related code from API & tests #849

Merged
merged 7 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ npx hardhat coverage [command-options]
| testfiles | `--testfiles "test/registry/*.ts"` | Test file(s) to run. (Globs must be enclosed by quotes and use [globby matching patterns][38])|
| sources | `--sources myFolder` or `--sources myFile.sol` | Path to *single* folder or file to target for coverage. Path is relative to Hardhat's `paths.sources` (usually `contracts/`) |
| solcoverjs | `--solcoverjs ./../.solcover.js` | Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed) |
| network | `--network development` | Use network settings defined in the Hardhat config |
| temp[<sup>*</sup>][14] | `--temp build` | :warning: **Caution** :warning: Path to a *disposable* folder to store compilation artifacts in. Useful when your test setup scripts include hard-coded paths to a build directory. [More...][14] |
| matrix | `--matrix` | Generate a JSON object that maps which mocha tests hit which lines of code. (Useful as an input for some fuzzing, mutation testing and fault-localization algorithms.) [More...][39]|

Expand Down Expand Up @@ -83,11 +82,9 @@ module.exports = {
| onPreCompile[<sup>*</sup>][14] | *Function* | | Hook run *after* filesystem and compiler configuration is applied, *before* the compiler is run. Can be used with the other hooks to be able to generate coverage reports on non-standard / customized directory structures, as well as contracts with absolute import paths. [More...][23] |
| onCompileComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* compilation completes, *before* tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. [More...][23]|
| onTestsComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated. [More...][23]|
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up. [More...][23]|
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the coverage task completes. Useful if you need to clean resources up. [More...][23]|
| configureYulOptimizer | *Boolean* | false | (Experimental) Setting to `true` should resolve "stack too deep" compiler errors in large projects using ABIEncoderV2 |
| solcOptimizerDetails | *Object* | `undefined` |(Experimental) Must be used in combination with `configureYulOptimizer`. Allows you configure solc's [optimizer details][1001]. Useful if the default remedy for stack-too-deep errors doesn't work in your case (See FAQ below). |
| client | *Object* | `require("ganache-core")` | Ganache only: useful if you need a specific ganache version |
| providerOptions | *Object* | `{ }` | Ganache only: [ganache-core options][1] |


[<sup>*</sup> Advanced use][14]
Expand Down Expand Up @@ -143,7 +140,6 @@ $ git clone https://github.com/sc-forks/solidity-coverage.git
$ yarn
```

[1]: https://github.com/trufflesuite/ganache-core#options
[2]: https://istanbul.js.org/docs/advanced/alternative-reporters/
[3]: https://mochajs.org/api/mocha
[4]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-gas
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ module.exports = {

The plugin exposes a set of workflow hooks that let you run arbitrary async logic between the main
stages of the coverage generation process. These are useful for tasks like launching secondary
services which need to connect to a running ganache instance (ex: the Oraclize/Provable bridge),
services which need to connect to a running ethereum client instance (ex: the Oraclize/Provable bridge),
or reading data from the compilation artifacts to run special preparatory steps for your tests.

The stages/hooks are (in order of execution):
Expand Down
69 changes: 26 additions & 43 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ table below shows how its core methods relate to the stages of a test run:
| Test Stage <img width=200/> | API Method <img width=200/> | Description <img width=800/> |
|---------------|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| compilation | `instrument` | A **pre-compilation** step: Rewrites contracts and generates an instrumentation data map. |
| client launch | `ganache` | A **substitute** step: Launches a ganache client with coverage collection enabled in its VM. As the client runs it will mark line/branch hits on the instrumentation data map. |
| client launch | `attachToHardhatVM` | A **pre-test** step: Enables coverage collection enabled in a HardhatEVM client. As the client runs it will mark line/branch hits on the instrumentation data map. |
| test | `report` | A **post-test** step: Generates a coverage report from the data collected by the VM after tests complete. |
| exit | `finish` | A **substitute** step: Shuts client down |


[3]: https://github.com/gotwarlost/istanbul

Expand All @@ -20,8 +20,8 @@ table below shows how its core methods relate to the stages of a test run:
disposable set of contracts/artifacts which coverage must use in lieu of the 'real' (uninstrumented)
contracts.

+ there are two complete [coverage tool/plugin implementations][5] (for Hardhat and Truffle)
which can be used as sources if you're building something similar.
+ there is a complete [coverage tool/plugin implementation][5] for Hardhat
which can be used as a source if you're building something similar.

[5]: https://github.com/sc-forks/solidity-coverage/tree/master/plugins

Expand All @@ -31,9 +31,8 @@ which can be used as sources if you're building something similar.
- [API Methods](#api)
* [constructor](#constructor)
* [instrument](#instrument)
* [ganache](#ganache)
* [attachToHardhatVM](#attachToHardhatVM)
* [report](#report)
* [finish](#finish)
* [getInstrumentationData](#getinstrumentationdata)
* [setInstrumentationData](#setinstrumentationdata)
- [Utils Methods](#utils)
Expand Down Expand Up @@ -64,13 +63,10 @@ Creates a coverage API instance. Configurable.
| ------ | ---- | ------- | ----------- |
| port | *Number* | 8555 | Port to launch client on |
| silent | *Boolean* | false | Suppress logging output |
| client | *Object* | `require("ganache-core")` | JS Ethereum client |
| providerOptions | *Object* | `{ }` | [ganache-core options][1] |
| skipFiles | *Array* | `[]` | Array of contracts or folders (with paths expressed relative to the `contracts` directory) that should be skipped when doing instrumentation. |
| istanbulFolder | *String* | `./coverage` | Folder location for Istanbul coverage reports. |
| istanbulReporter | *Array* | `['html', 'lcov', 'text', 'json']` | [Istanbul coverage reporters][2] |

[1]: https://github.com/trufflesuite/ganache-core#options
[2]: https://istanbul.js.org/docs/advanced/alternative-reporters/

--------------
Expand Down Expand Up @@ -100,33 +96,37 @@ const instrumented = api.instrument(contracts)

--------------

## ganache
## attachToHardhatVM

Enables coverage data collection on an in-process ganache server. By default, this method launches
the server, begins listening on the port specified in the [config](#constructor) (or 8555 if unspecified), and
returns a url string. When `autoLaunchServer` is false, method returns `ganache.server` so you can control
the `server.listen` invocation yourself.
Enables coverage data collection on a HardhatEVM provider. (You will need to create a hardhat provider with the correct VM settings as shown below before invoking this method.)

**Parameters**

- `client` **Object**: (*Optional*) ganache module
- `autoLaunchServer` **Boolean**: (*Optional*)
- `provider` **Object**: Hardhat provider

Returns **Promise** Address of server to connect to, or initialized, unlaunched server
Returns **Promise**

**Example**
```javascript
const client = require('ganache-cli');
const { createProvider } = require("hardhat/internal/core/providers/construction");
const { resolveConfig } = require("hardhat/internal/core/config/config-resolution");
const { HARDHAT_NETWORK_NAME } = require("hardhat/plugins")

const api = new CoverageAPI( { client: client } );
const address = await api.ganache();
const api = new CoverageAPI( { ... } );
const config = resolveConfig("./", {});

> http://127.0.0.1:8555
config.networks[HARDHAT_NETWORK_NAME].allowUnlimitedContractSize = true;
config.networks[HARDHAT_NETWORK_NAME].blockGasLimit = api.gasLimitNumber;
config.networks[HARDHAT_NETWORK_NAME].gas = api.gasLimit;
config.networks[HARDHAT_NETWORK_NAME].gasPrice = api.gasPrice;
config.networks[HARDHAT_NETWORK_NAME].initialBaseFeePerGas = 0;

// Alternatively...
const provider = await createProvider(
config,
HARDHAT_NETWORK_NAME
)

const server = await api.ganache(client, false);
await pify(server.listen)(8545));
await api.attachToHardhatVM(provider);
```

--------------
Expand All @@ -148,22 +148,6 @@ await api.report('./coverage_4A3cd2b'); // Default folder name is 'coverage'

-------------

## finish

Shuts down coverage-enabled ganache server instance

Returns **Promise**

**Example**
```javascript
const client = require('ganache-cli');

await api.ganache(client); // Server listening...
await api.finish(); // Server shut down.
```

-------------

## getInstrumentationData

Returns a copy of the hit map created during instrumentation. Useful if you'd like to delegate
Expand Down Expand Up @@ -193,7 +177,7 @@ const data = load(data); // Pseudo-code
api.setIntrumentationData(data);

// Client will collect data for the loaded map
const address = await api.ganache(client);
await api.attachToHardhatVM(provider);

// Or to `report` instrumentation data which was collected in a different process.
const data = load(data); // Pseudo-code
Expand Down Expand Up @@ -348,8 +332,7 @@ utils.save(instrumented, config.contractsDir, tempContractsDir);

## finish

Deletes temporary folders and shuts the ganache server down. Is tolerant - if folders or ganache
server don't exist it will return silently.
Deletes temporary folders. Is tolerant - if folders don't exist it will return silently.

**Parameters**

Expand Down
109 changes: 2 additions & 107 deletions lib/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ const fs = require('fs');
const path = require('path');
const istanbul = require('sc-istanbul');
const assert = require('assert');
const detect = require('detect-port');
const _ = require('lodash/lang');

const ConfigValidator = require('./validator');
Expand Down Expand Up @@ -45,20 +44,10 @@ class API {
this.onIstanbulComplete = config.onIstanbulComplete || this.defaultHook;
this.onPreCompile = config.onPreCompile || this.defaultHook;

this.server = null;
this.defaultPort = 8555;
this.client = config.client;
this.defaultNetworkName = 'soliditycoverage';
this.port = config.port || this.defaultPort;
this.host = config.host || "127.0.0.1";
this.providerOptions = config.providerOptions || {};
this.autoLaunchServer = config.autoLaunchServer === false ? false : true;

this.skipFiles = config.skipFiles || [];

this.log = config.log || console.log;
this.gasLimit = 0xffffffffff // default "gas sent" with transactions
this.gasLimitString = "0x1fffffffffffff"; // block gas limit for ganache (higher than "gas sent")
this.gasLimitNumber = 0x1fffffffffffff; // block gas limit for Hardhat
this.gasPrice = 0x01;

Expand Down Expand Up @@ -138,56 +127,6 @@ class API {
this.instrumenter.instrumentationData = _.cloneDeep(data);
}

/**
* Enables coverage collection on in-process ethereum client server, hooking the DataCollector
* to its VM. By default, method will return a url after server has begun listening on the port
* specified in the config. When `autoLaunchServer` is false, method returns`ganache.server` so
* the consumer can control the 'server.listen' invocation themselves.
* @param {Object} client ganache client
* @param {Boolean} autoLaunchServer boolean
* @return {<Promise> (String | Server) } address of server to connect to, or initialized, unlaunched server.
*/
async ganache(client, autoLaunchServer){
// Check for port-in-use
if (await detect(this.port) !== this.port){
throw new Error(this.ui.generate('server-fail', [this.port]))
}

this.collector = new DataCollector(this.instrumenter.instrumentationData);

this.providerOptions.gasLimit =
'gasLimit' in this.providerOptions
? this.providerOptions.gasLimit
: this.gasLimitString;

this.providerOptions.allowUnlimitedContractSize =
'allowUnlimitedContractSize' in this.providerOptions
? this.providerOptions.allowUnlimitedContractSize
: true;

// Attach to vm step of supplied client
try {
if (this.config.forceBackupServer) throw new Error()
await this.attachToGanacheVM(client)
}

// Fallback to ganache-cli)
catch(err) {
const _ganache = require('ganache-cli');
this.ui.report('vm-fail', [_ganache.version]);
await this.attachToGanacheVM(_ganache);
}

if (autoLaunchServer === false || this.autoLaunchServer === false){
return this.server;
}

await pify(this.server.listen)(this.port);
const address = `http://${this.host}:${this.port}`;
this.ui.report('server', [address]);
return address;
}

/**
* Generate coverage / write coverage report / run istanbul
*/
Expand Down Expand Up @@ -223,60 +162,16 @@ class API {
})
}


/**
* Removes coverage build artifacts, kills testrpc.
*/
async finish() {
if (this.server && this.server.close){
this.ui.report('finish');
await pify(this.server.close)();
}
}
async finish() { /* Just a stub now - used to shutdown ganache */}

// ------------------------------------------ Utils ----------------------------------------------

// ========
// Provider
// ========
async attachToGanacheVM(client){
const self = this;

// Fallback to client from options
if(!client) client = this.client;
this.server = client.server(this.providerOptions);

this.assertHasBlockchain(this.server.provider);
await this.vmIsResolved(this.server.provider);

const blockchain = this.server.provider.engine.manager.state.blockchain;
const createVM = blockchain.createVMFromStateTrie;

// Attach to VM which ganache has already created for transactions
blockchain.vm.on('step', self.collector.step.bind(self.collector));

// Hijack createVM method which ganache runs for each `eth_call`
blockchain.createVMFromStateTrie = function(state, activatePrecompiles) {
const vm = createVM.apply(blockchain, arguments);
vm.on('step', self.collector.step.bind(self.collector));
return vm;
}
}

assertHasBlockchain(provider){
assert(provider.engine.manager.state.blockchain !== undefined);
assert(provider.engine.manager.state.blockchain.createVMFromStateTrie !== undefined);
}

async vmIsResolved(provider){
return new Promise(resolve => {
const interval = setInterval(() => {
if (provider.engine.manager.state.blockchain.vm !== undefined){
clearInterval(interval);
resolve();
}
});
})
}

// Hardhat
async attachToHardhatVM(provider){
Expand Down
12 changes: 0 additions & 12 deletions lib/ui.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,6 @@ class AppUI extends UI {
const w = ":warning:";

const kinds = {
'vm-fail': `${w} ${c.red('There was a problem attaching to the ganache VM.')}\n` +
`${w} ${c.red('For help, see the "client" & "providerOptions" syntax in solidity-coverage docs.')}\n`+
`${w} ${c.red(`Using ganache-cli (v${args[0]}) instead.`)}\n`,


'instr-start': `\n${c.bold('Instrumenting for coverage...')}` +
`\n${c.bold('=============================')}\n`,
Expand All @@ -69,10 +65,6 @@ class AppUI extends UI {
'istanbul': `${ct} ${c.grey('Istanbul reports written to')} ./coverage/ ` +
`${c.grey('and')} ./coverage.json`,

'finish': `${ct} ${c.grey('solidity-coverage cleaning up, shutting down ganache server')}`,

'server': `${ct} ${c.bold('server: ')} ${c.grey(args[0])}`,

'command': `\n${w} ${c.red.bold('solidity-coverage >= 0.7.0 is no longer a shell command.')} ${w}\n` +
`${c.bold('=============================================================')}\n\n` +
`Instead, you should use the plugin produced for your development stack\n` +
Expand Down Expand Up @@ -103,10 +95,6 @@ class AppUI extends UI {
'istanbul-fail': `${c.red('Istanbul coverage reports could not be generated. ')}`,

'sources-fail': `${c.red('Cannot locate expected contract sources folder: ')} ${args[0]}`,

'server-fail': `${c.red('Port')} ${args[0]} ${c.red('is already in use.\n')}` +
`${c.red('\tRun: "lsof -i" to find the pid of the process using it.\n')}` +
`${c.red('\tRun: "kill -9 <pid>" to kill it.\n')}`
}

return this._format(kinds[kind])
Expand Down
2 changes: 0 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
"@solidity-parser/parser": "^0.18.0",
"chalk": "^2.4.2",
"death": "^1.1.0",
"detect-port": "^1.3.0",
"difflib": "^0.2.4",
"fs-extra": "^8.1.0",
"ghost-testrpc": "^0.0.2",
Expand All @@ -52,7 +51,6 @@
"decache": "^4.5.1",
"ethereum-waffle": "^3.4.0",
"ethers": "^5.5.3",
"ganache-cli": "6.12.2",
"hardhat": "^2.19.5",
"hardhat-gas-reporter": "^1.0.1",
"nyc": "^14.1.1",
Expand Down
Loading