Code coverage in end-to-end/integration Go tests
I like writing integration tests for my Go code. They give me confidence that everything works as expected. This was one of the reasons I built Gnomock (an integration and end-to-end toolkit based on Docker): to easily write tests for code that uses databases or other external services, like AWS S3 or Splunk.
I often put integration and end-to-end tests either into a separate package, or
main package (
main_test in the project root). I had one problem
with it: code coverage reports were not good enough, they skipped entire
packages and gave unexpected results. This was the default behavior when using
the following command:
In the next examples, I will use a demo app, which has an integration test, and two packages:
$ go test -race -cover PASS coverage: 0.0% of statements ok github.com/orlangure/crud-demo 20.876s
My tests took 20 seconds to run, but no code was covered, which I knew was
wrong. The issue was that the coverage report ignored sub-packages
handlers), which had all the important things in them.
Turns out there is a
-coverpkg flag that allows to select a package for the
$ go test -race -cover -coverpkg ./handlers PASS coverage: 58.1% of statements in ./handlers ok github.com/orlangure/crud-demo 20.617s
It is even possible to report coverage for all sub-packages:
$ go test -race -cover -coverpkg ./... PASS coverage: 58.7% of statements in ./... ok github.com/orlangure/crud-demo 19.845s
This information is more useful than
coverage: 0.0%, but not useful enough.
To add more details, coverage report can be saved into a file for further
$ go test -race -cover -coverpkg ./... -coverprofile cover.out PASS coverage: 58.7% of statements in ./... ok github.com/orlangure/crud-demo 21.038s
The output looks the same, but a new file appears in the current directory. We can extract a more detailed report from it:
$ go tool cover -func cover.out <skipped>/handlers/handlers.go:14: CreateThingHandler 63.6% <skipped>/handlers/handlers.go:36: GetThingByNameHandler 57.1% <skipped>/handlers/handlers.go:63: GetThingByIDHandler 55.6% <skipped>/main.go:14: main 0.0% <skipped>/models/things.go:19: String 0.0% <skipped>/models/things.go:24: Connect 75.0% <skipped>/models/things.go:39: CreateThing 80.0% <skipped>/models/things.go:54: GetThingByName 85.7% <skipped>/models/things.go:74: GetThingByID 85.7% total: (statements) 58.7%
From this report it is clear that
String function from
things.go file has
zero coverage. This can be improved easily by adding another test. What about
other functions with non-zero coverage? What did the tests miss?
There is another useful
go tool feature:
$ go tool cover -html cover.out
This report includes coverage for every line of code. It appears that none of the errors were tested 😿
Coverage reports can even include a heat map and tell you how many times
each line of code was hit! This option is set by default when
-race flag is
used, or can be allowed with
-covermode atomic flag. Read more about
supported flags here.
Integration tests are great to build confidence in your code, but coverage
reports of such tests in Go can be tricky. Use
flags to get more details from test coverage reports, and use
go tool cover -html to view the results in a browser.
If your code relies on external services, like databases or AWS, Gnomock can be useful for writing integration or end-to-end tests.