Testing and Code Coverage of Solidity Smart Contracts

It’s no secret that continuous integration and continuous deployment is an important part of the application development process. Integrating testing into this workflow is also an important part, to protect against any unwanted regression when developing new features or refactoring existing code.

Having all of your unit tests go green and be passing is one thing, but it’s also important to know just how much of your code-base has been covered by these tests. Having 100% passing tests is nice but if the tests only cover 10% of your codebase you’re still not going to catch regression  in areas which are not covered.

Being able to measure how much of your codebase is covered by tests is equally as important.

When you add in the very real possibility of financial loss if a smart contract is not properly tested and audited, it becomes even more important to make sure that every part of your code which can be tested, is tested.

At EdgeFund we chose to use Travis CI as our continuous integration tool, along with coveralls to ensure that our smart contracts were tested and coverage was as close to 100% as possible.

Travis CI is super easy to set up and use with your application. Our code is available on GitHub so we were able to connect to Travis quite simply through the GitHub  interface. The process to hook up coveralls is very similar, if you’re on github.

Once you have your repository connected to both travis and coveralls, it’s simply a case of creating a .travis.yml file which describes your build process. It should also be noted that we are using ganache-cli which is part of the truffle suite in order to provide us with a test blockchain against which to develop and test our smart contracts.


sudo: required
language: node_js

  - "8"

  - docker

  - docker pull trufflesuite/ganache-cli
  - docker run -d -p 8545:8545 trufflesuite/ganache-cli:latest -h

  - npm install -g truffle
  - npm install -g coveralls
  - npm install

  - npm run coverage && cat coverage/lcov.info | coveralls

The last line is where the tests and code coverage is being ran. This makes use of a script we have defined in our package.json file.

”coverage”: “./node_modules/.bin/solidity-coverage”

We’re using a module called solidity-coverage which is an npm module we have listed as a dev dependency in the package.json file.

After the coverage script has ran and saved its output to a coverage directory, we read out the contents of a file in there called lcov.info and send it to the coveralls module. This module is responsible for interpreting and sending our test coverage output to the coveralls website.

As we can see below in our coveralls report, we have 100% coverage on 3 of our contracts, and 1 contract which needs a bit more attention.

coveralls.io coverage report for edgefund-core repository master branch

All in all it took less than 1 hour to get everything hooked up and connected with Travis and Coveralls and now we have a continuous integration process where we can see the effect our changes have on our overall coverage percentage. This is especially useful when it comes to reviewing pull requests. We can see for example if someone has added a new feature, but has forgotten to add associated tests.

It’s unlikely you’ll have 100% test coverage across all code paths, nor should it be expected, but being able to see the effect code changes have on the overall percentage is definitely a useful thing to be aware of when developing a new smart contract.

Gareth Oates

Gareth Oates

Oslo, Norway