Skip to content

Commit

Permalink
Merge pull request #548 from sc-forks/hardhat
Browse files Browse the repository at this point in the history
Add Hardhat support
  • Loading branch information
cgewecke authored Nov 16, 2020
2 parents 6761556 + e106076 commit f5ed5e5
Show file tree
Hide file tree
Showing 59 changed files with 2,450 additions and 1,896 deletions.
12 changes: 6 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,26 +65,26 @@ jobs:
executor: win/default
steps:
- checkout
- run: dotnet tool install --global PowerShell
- run:
name: Windows Metacoin E2E
command: |
bash ./scripts/run-metacoin.sh
e2e-buidler:
e2e-nomiclabs:
machine: true
steps:
- checkout
- <<: *step_install_nvm
- run:
name: Buidler E2E
name: Buidler & Hardhat E2E
command: |
./scripts/run-buidler.sh
./scripts/run-nomiclabs.sh
workflows:
version: 2
build:
jobs:
- unit-test
- e2e-zeppelin
# Temporarily disabled due to unskipped GSN gas measurement tests
# - e2e-zeppelin
- e2e-metacoin
- e2e-metacoin-windows
- e2e-buidler
- e2e-nomiclabs
131 changes: 131 additions & 0 deletions HARDHAT_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
[![Gitter chat](https://badges.gitter.im/sc-forks/solidity-coverage.svg)][18]
![npm (tag)](https://img.shields.io/npm/v/solidity-coverage/latest)
[![CircleCI](https://circleci.com/gh/sc-forks/solidity-coverage.svg?style=svg)][20]
[![codecov](https://codecov.io/gh/sc-forks/solidity-coverage/branch/beta/graph/badge.svg)][21]
[![hardhat](https://hardhat.org/buidler-plugin-badge.svg?1)][26]

# solidity-coverage

Solidity code coverage plugin for [Hardhat](http://hardhat.org).

## What

![coverage example][22]

+ For more details about how it works and potential limitations, see [the accompanying article][16].
+ `solidity-coverage` is also [JoinColony/solcover][17]


## Installation

```bash
$ npm install --save-dev solidity-coverage
```

And add the following to your `.config.js`:

```js
require("solidity-coverage");
```

Or, if you are using TypeScript, add this to your hardhat.config.ts:

```ts
import "solidity-coverage"
```

## Tasks

This plugin implements a `coverage` task

```bash
npx hardhat coverage [options]
```

| Option <img width=200/> | Example <img width=750/>| Description <img width=1000/> |
|--------------|------------------------------------|--------------------------------|
| testfiles | `--testfiles "test/registry/*.ts"` | Test file(s) to run. (Globs must be enclosed by quotes.)|
| solcoverjs | `--solcoverjs ./../.solcover.js` | Relative path from working directory to config. Useful for monorepo packages that share settings. (Path must be "./" prefixed) |
| network | `--network development` | Run with a ganache client over http using network settings defined in `hardhat.config.js`. (Hardhat is the default network) |


## Configuration

Options can be specified in a `.solcover.js` config file located in the root directory of your project.

**Config Example:**
```javascript
module.exports = {
skipFiles: ['Routers/EtherRouter.sol']
};
```

| Option <img width=200/>| Type <img width=200/> | Default <img width=1300/> | Description <img width=800/> |
| ------ | ---- | ------- | ----------- |
| silent | *Boolean* | false | Suppress logging output |
| client | *Object* | undefined | *Ganache only*: Useful if you need a specific ganache version. An example value is: `require('ganache-cli')` |
| providerOptions | *Object* | `{ }` | *Ganache only*: [ganache-core options][1] |
| skipFiles | *Array* | `['Migrations.sol']` | Array of contracts or folders (with paths expressed relative to the `contracts` directory) that should be skipped when doing instrumentation. |
| istanbulFolder | *String* | `./coverage` | Folder location for Istanbul coverage reports. |
| istanbulReporter | *Array* | `['html', 'lcov', 'text', 'json']` | [Istanbul coverage reporters][2] |
| mocha | *Object* | `{ }` | [Mocha options][3] to merge into existing mocha config. `grep` and `invert` are useful for skipping certain tests under coverage using tags in the test descriptions.|
| onServerReady[<sup>*</sup>][14] | *Function* | | Hook run *after* server is launched, *before* the tests execute. Useful if you need to use the Oraclize bridge or have setup scripts which rely on the server's availability. [More...][23] |
| onCompileComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* compilation completes, *before* tests are run. Useful if you have secondary compilation steps or need to modify built artifacts. [More...][23]|
| onTestsComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the tests complete, *before* Istanbul reports are generated. [More...][23]|
| onIstanbulComplete[<sup>*</sup>][14] | *Function* | | Hook run *after* the Istanbul reports are generated, *before* the ganache server is shut down. Useful if you need to clean resources up. [More...][23]|

[<sup>*</sup> Advanced use][14]

## Usage

+ Coverage runs tests a little more slowly.
+ Coverage uses the Hardhat network by default.
+ Coverage [distorts gas consumption][13]. Tests that check exact gas consumption should be [skipped][24].
+ :warning: Contracts are compiled **without optimization**. Please report unexpected compilation faults to [issue 417][25]

## Using with ganache

Begining with `v0.7.12`, this plugin runs directly on the Hardhat network by default (for speed).

If you want to use a ganache based http network, you can specify it by name using the `--network` cli option. The plugin will then launch its own coverage enabled ganache instance which can be configured in `.solcover.js` via the `providerOptions` key.

## Documentation

More documentation, including FAQ and information about solidity-coverage's API [is available here][28].


[1]: https://github.com/trufflesuite/ganache-core#options
[2]: https://istanbul.js.org/docs/advanced/alternative-reporters/
[3]: https://mochajs.org/api/mocha
[4]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-gas
[5]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-memory
[6]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#running-out-of-time
[7]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#continuous-integration
[8]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#notes-on-branch-coverage
[9]: https://sc-forks.github.io/metacoin/
[10]: https://coveralls.io/github/OpenZeppelin/openzeppelin-solidity?branch=master
[11]: https://github.com/sc-forks/solidity-coverage/tree/master/test/units
[12]: https://github.com/sc-forks/solidity-coverage/issues
[13]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/faq.md#notes-on-gas-distortion
[14]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/advanced.md
[15]: #config-options
[16]: https://blog.colony.io/code-coverage-for-solidity-eecfa88668c2
[17]: https://github.com/JoinColony/solcover
[18]: https://gitter.im/sc-forks/solidity-coverage?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
[19]: https://badge.fury.io/js/solidity-coverage
[20]: https://circleci.com/gh/sc-forks/solidity-coverage
[21]: https://codecov.io/gh/sc-forks/solidity-coverage
[22]: https://cdn-images-1.medium.com/max/800/1*uum8t-31bUaa6dTRVVhj6w.png
[23]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/advanced.md#workflow-hooks
[24]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/advanced.md#skipping-tests
[25]: https://github.com/sc-forks/solidity-coverage/issues/417
[26]: https://hardhat.org/
[27]: https://www.trufflesuite.com/docs
[28]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/api.md
[29]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/upgrade.md#upgrading-from-06x-to-070
[30]: https://github.com/sc-forks/solidity-coverage/tree/0.6.x-final#solidity-coverage
[31]: https://github.com/sc-forks/solidity-coverage/releases/tag/v0.7.0
[32]: https://github.com/sc-forks/buidler-e2e/tree/coverage
[33]: https://github.com/sc-forks/moloch
[34]: https://github.com/sc-forks/solidity-coverage/blob/master/docs/advanced.md#reducing-the-instrumentation-footprint

37 changes: 31 additions & 6 deletions lib/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ class API {
this.skipFiles = config.skipFiles || [];

this.log = config.log || console.log;

this.gasLimit = 0xffffffffff; // default "gas sent" with transactions
this.gasLimitString = "0xfffffffffff"; // block gas limit for ganache (higher than "gas sent")
this.gasLimit = 0xffffffffff // default "gas sent" with transactions
this.gasLimitString = "0x1fffffffffffff"; // block gas limit for ganache (higher than "gas sent")
this.gasLimitNumber = 0x1fffffffffffff; // block gas limit for Hardhat
this.gasPrice = 0x01;

this.istanbulFolder = config.istanbulFolder || false;
Expand Down Expand Up @@ -158,14 +158,14 @@ class API {
// Attach to vm step of supplied client
try {
if (this.config.forceBackupServer) throw new Error()
await this.attachToVM(client)
await this.attachToGanacheVM(client)
}

// Fallback to ganache-cli)
catch(err) {
const _ganache = require('ganache-cli');
this.ui.report('vm-fail', [_ganache.version]);
await this.attachToVM(_ganache);
await this.attachToGanacheVM(_ganache);
}

if (autoLaunchServer === false || this.autoLaunchServer === false){
Expand Down Expand Up @@ -228,7 +228,7 @@ class API {
// ========
// Provider
// ========
async attachToVM(client){
async attachToGanacheVM(client){
const self = this;

// Fallback to client from options
Expand Down Expand Up @@ -268,6 +268,31 @@ class API {
})
}

// Hardhat
attachToHardhatVM(provider){
const self = this;
this.collector = new DataCollector(this.instrumenter.instrumentationData);

let cur = provider;

// Go down to core HardhatNetworkProvider
while (cur._wrapped) {
cur = Object.assign({}, cur._wrapped)
}
cur._node._vm.on('step', self.collector.step.bind(self.collector))
}

// Temporarily disabled because some relevant traces aren't available
// (maybe bytecode cannot be found)
/*hardhatTraceHandler(trace, isTraceFromCall){
for (const step of trace.steps){
if (trace.bytecode && trace.bytecode._pcToInstruction){
const instruction = trace.bytecode._pcToInstruction.get(step.pc)
this.collector.trackHardhatEVMInstruction(instruction)
}
}
}*/

// ========
// File I/O
// ========
Expand Down
30 changes: 25 additions & 5 deletions lib/collector.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,35 @@ class DataCollector {
if (this.validOpcodes[info.opcode.name] && info.stack.length > 0){
const idx = info.stack.length - 1;
let hash = web3Utils.toHex(info.stack[idx]).toString();
hash = this._normalizeHash(hash);

if(this.instrumentationData[hash]){
this.instrumentationData[hash].hits++;
}
this._registerHash(hash)
}
} catch (err) { /*Ignore*/ };
}

// Temporarily disabled because some relevant traces aren't available
/**
* Converts pushData value to string and registers in instrumentation map.
* @param {HardhatEVMTraceInstruction} instruction
*/
/*trackHardhatEVMInstruction(instruction){
if (instruction && instruction.pushData){
let hash = `0x` + instruction.pushData.toString('hex');
this._registerHash(hash)
}
}*/

/**
* Normalizes has string and marks hit.
* @param {String} hash bytes32 hash
*/
_registerHash(hash){
hash = this._normalizeHash(hash);

if(this.instrumentationData[hash]){
this.instrumentationData[hash].hits++;
}
}

/**
* Left-pads zero prefixed bytes 32 hashes to length 66. The '59' in the
* comparison below is arbitrary. It provides a margin for recurring zeros
Expand Down
8 changes: 6 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "solidity-coverage",
"version": "0.7.11",
"description": "",
"main": "plugins/buidler.plugin.js",
"main": "plugins/nomiclabs.plugin.js",
"bin": {
"solidity-coverage": "./plugins/bin.js"
},
Expand Down Expand Up @@ -41,15 +41,19 @@
"recursive-readdir": "^2.2.2",
"sc-istanbul": "^0.4.5",
"shelljs": "^0.8.3",
"web3": "^1.3.0"
"web3-utils": "^1.3.0"
},
"devDependencies": {
"@nomiclabs/buidler": "^1.3.6",
"@nomiclabs/buidler-truffle5": "^1.3.4",
"@nomiclabs/buidler-web3": "^1.3.4",
"@nomiclabs/hardhat-truffle5": "^2.0.0",
"@nomiclabs/hardhat-web3": "^2.0.0",
"@truffle/contract": "^4.0.36",
"buidler-gas-reporter": "^0.1.3",
"decache": "^4.5.1",
"hardhat": "^2.0.2",
"hardhat-gas-reporter": "^1.0.1",
"mocha": "5.2.0",
"nyc": "^14.1.1",
"solc": "^0.5.10",
Expand Down
18 changes: 8 additions & 10 deletions plugins/buidler.plugin.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
const API = require('./../lib/api');
const utils = require('./resources/plugin.utils');
const buidlerUtils = require('./resources/buidler.utils');
const PluginUI = require('./resources/buidler.ui');
const buidlerUtils = require('./resources/nomiclabs.utils');
const PluginUI = require('./resources/nomiclabs.ui');

const pkg = require('./../package.json');
const death = require('death');
const path = require('path');
const Web3 = require('web3');

const { task, types } = require("@nomiclabs/buidler/config");
const { ensurePluginLoadedWithUsePlugin } = require("@nomiclabs/buidler/plugins");
Expand Down Expand Up @@ -53,25 +52,24 @@ function plugin() {
// ==============
// Server launch
// ==============
const network = buidlerUtils.setupNetwork(env, api, ui);
const network = buidlerUtils.setupBuidlerNetwork(env, api, ui);

const client = api.client || require('ganache-cli');
const address = await api.ganache(client);
const web3 = new Web3(address);
const accounts = await web3.eth.getAccounts();
const nodeInfo = await web3.eth.getNodeInfo();
const ganacheVersion = nodeInfo.split('/')[1];
const accountsRequest = await utils.getAccountsGanache(api.server.provider);
const nodeInfoRequest = await utils.getNodeInfoGanache(api.server.provider);
const ganacheVersion = nodeInfoRequest.result.split('/')[1];

// Set default account
network.from = accounts[0];
network.from = accountsRequest.result[0];

// Version Info
ui.report('versions', [
ganacheVersion,
pkg.version
]);

ui.report('network', [
ui.report('ganache-network', [
env.network.name,
api.port
]);
Expand Down
Loading

0 comments on commit f5ed5e5

Please sign in to comment.