Testing Your Tests: Mutation Testing

If you’re into testing you should read the post Mutation Testing. I’m a advocate of testing in general and TDD in specific.   I’ve done mutation testing manually, breaking code to make sure my tests caught the break, but finding a tool to do it automatically seemed like a win.

What does mutation testing buy you? It makes sure your tests cover semantic changes that they might have missed.  Does your fluid API always return the target object?  Do you handle null arguments as expected?  Are all the sad paths really covered?

Having read the above I immediately tried applying the tools to one of my more stable frameworks – almost-functional.  I found that pitest had a gradle plugin and put it in place.  It was really easy to hook up, took all of maybe thirty minutes, and immediately yielded results!  I was happy to see that my coverage percentage was in line with my overall coverage (90% +) but caught some things well worth adding to improve the stability of my API.

So, yeah, if you’re a Java dev and want your tests done right, I’d advise you read the above and try it on your project.  I’ve even been chatting with folks at sites like codecov to see what it would take for them to support pitest.

Advertisements

One thought on “Testing Your Tests: Mutation Testing

  1. Pingback: Together: PIT, SonarQube and Gradle | My name is Ozymandias

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s