I just spent the evening improving Plume's continuous integration, and it wasn't as easy as I could have imagined.
That's why I decided to write a small article to summarize how to setup continuous integration for a Rust project, with Travis CI (and how to get nice coverage reports with Codecov too).
This solution works for Rust project using workspaces too.
Travis CI uses
.travis.yml as its configuration file. The simplest config for a Rust project is:
language: rust cache: cargo
Actually the second line is not even required, but it will make future build faster by caching artifacts.
Travis CI has a predefined preset for Rust projects, that install all the required tools with
rustup for you. In their doc they explain how to choose the Rust version(s) to
install. But what is not obvious is that you can specify any valid toolchain here, it will be
For instance, Plume needs a very specific Nightly version to be installed. Before, we were just telling Travis to install Rust
nightly, and it was installing the latest
Nightly build. That was not a real problem, because Plume has a
rust-toolchain file, and when the build was started with the wrong Rust version, the correct one was installed,
so nothing ever broke. The only issue was that we were downloading Rust twice (and Rust is not a very light thing). To avoid that, you can directly specify the toolchain you
want to use:
rust: - nightly-2018-07-17
Building and testing in parallel
Then I wanted to both build Plume and run the tests at the same time. You can achieve that in Travis CI by running multiples "jobs" in the same "stage". Here is a simplified version of Plume's config to understand how to do that:
jobs: include: - stage: test and build script: cargo test --all - stage: test and build script: cargo build
So all you have to do, is to put two (or more) jobs descriptions in the
jobs.include list, with the same value for
stage, and they will run in parallel.
If you want to learn more about jobs and stages, I advise you to check out the official documentation.
Code Coverage with kcov and Codecov
That part was the most difficult to figure out. The solution I came with uses kcov and Codecov.
First of all, you need to install kcov. But because the distribution packages are totally outdated, you'll have to compile it from source (it doesn't take too much time
fortunately). Then you'll have to run it for all the test executables in your workspace. These executables are generated by cargo when running
cargo test --all, and can be
target/debug. Finally, you'll have to upload the result to Codecov (they are providing a great Bash script to do that easily). Here is the script to achieve that:
sudo: true dist: trusty addons: apt: packages: - libcurl4-openssl-dev - libelf-dev - libdw-dev - cmake - gcc - binutils-dev - zlib1g-dev - libiberty-dev jobs: include: - stage: test and build env: - RUSTFLAGS='-C link-dead-code' script: cargo test --all after_success: - | wget https://github.com/SimonKagstrom/kcov/archive/master.tar.gz && tar xzf master.tar.gz && cd kcov-master && mkdir build && cd build && cmake .. && make && sudo make install && cd ../.. && rm -rf kcov-master && for crate in WORKSPACE_MEMBERS lib; do for file in target/debug/$crate-*[^\.d]; do mkdir -p "target/cov/$(basename $file)"; kcov --exclude-pattern=/.cargo,/usr/lib --verify "target/cov/$(basename $file)" "$file"; done; done && bash <(curl -s https://codecov.io/bash) && echo "Uploaded code coverage"
after_success part is where all the steps I described above lies. But there are few other thing that were added to our config. First of all, we install some dependencies
kcov, since we are building it from source (
addons.apt). Then we are compiling with a special flag:
link-dead-code. It tells Rust to keep
dead code in our executable. This way, even functions that are not called during our tests will be in the coverage report, which will be more accurate.
The most complex line of the script is probably the
for loop near the end. It actually iterates over a list of workspace members (you should replace
the name of the crates in your workspace) and
lib, tries to find a test executable that start with that name and run
kcov for it. Test executable are either named after the
name of the crate (for unit tests) or
lib-something (for integration tests).
The last step is to add the Codecov token to your Travis CI environment so that you can upload reports. You can find your token at https://codecov.io/gh/OWNER/REPO/settings, and
add it in Travis at https://travis-ci.org/OWNER/REPO/settings (the variable should be named
That's it! You should now be able to build, test and get coverage reports for any Rust project (even with workspaces). I hope this little article helped you, and if you found an error please tell me.