I came across mutation testing quite some while ago, and decided to set it up in a project. The goal of mutation testing is to pick up some flaws in written unit tests and mark them as mutant.

How does it work

The idea is to modify your code covered codebase in a small way, and check if your current test suite will pick up the mutation.

For example:

<?php
function doSomething() {
    return new ResultDto();
}

Your code would be changed to:

<?php
function doSomething() {
    return null;
}

And your mutation testing framework will run your test framework again. If all your tests still pass, one mutant has escaped.

However mutation testing is not only limited to return values, it will also change parameters

<?php
class Foo {
	public function doSomething() {
	    $this->service->bar(true);
	}
}

Your mutation testing framework would change the parameter to false

<?php
class Foo {
	public function doSomething() {
	    $this->service->bar(false);
	}
}

Now that behavior has changed, your unit test framework will run again to see if it still passes.

Why would you use mutation testing

It is useful to use mutation testing if you are unsure about the quality of your test suite. It exposes missing behavioral changes that are not under test. It’s also useful to quickly check your own new code to see if you missed something.

Why would you not use mutation testing

If you are a practitioner of TDD I would not use mutation testing. With TDD you have defined your behavior in advance, how would it even be possible for a mutation testing framework to pick up any behavioral problem? Right? :) Your tests should have defined what to expect from your codebase.

The benifits are slim and it slows down your continuous integration suite.