Nerdy tidbits from my life as a software engineer

Tuesday, July 7, 2009

How Much Should You Mock?

I managed to incite a small riot a few weeks ago when I got involved in a somewhat heated debate internally about how far one should go with their unit testing.  How much should you be mocking in your unit tests?  Should you mock the file system?  Should you mock a database?  Should you mock calls to a web service?

The basic problem is that at the lowest layers, you end up reaching a point where you must interact with an outside dependency.  This is the point where there’s nothing left to mock: you have abstracted everything you possibly can and now you must interact with the outside world.  The arguments many people make are, because the interaction with the external dependency is something that you don’t own, you can’t possibly simulate it’s behavior, and therefore you can’t test it.  Or, because the dependency can’t be abstracted any further, that being able to test it is so difficult that there are no benefits to the extra work.  Both of these are no doubt true on at least a few levels.  What people prefer to do is wrap these dependencies with abstractions, and then test those abstractions rather than the boundaries.  The last bit at least, I agree with completely.

Here’s my problem.  Often times, the reason that external dependencies are so hard to be test is because they’re not designed to be tested.  Take the Tfs Client API, for instance.  This is an insanely difficult library to test because it is filled with sealed objects that have non-virtual methods and private setters.  Ack!  The only way to test this is to mimic the hierarchy by using a nicely designed bridge pattern and reference the object model via our own abstractions.  But this is not ideal.  Why would we choose to wrap an API with a testable object model instead of using the actual object model?  As long as we can mock the original hierarchy, this becomes a large waste of time and serves no purpose.

Sadly, many of the boundaries in our applications run into walls just like this.  And to me, this is the major reason that mocking them becomes so difficult.  It’s not that there is no value in mocking these dependencies, it’s just that there’s no real practical way to do it most of the time. 

…but if there was, my question is: why would we choose not to?  Just because it’s not our system doesn’t mean we shouldn’t try and test our interaction with it.  Somewhere in our code, we are making assumptions that external systems are going to behave a certain way.  They’re going to throw certain exceptions in certain cases, they’re going to return different values based on different inputs, etc.  Because we have to code against these behaviors, why would we choose not to test against them, too?  There are, perhaps, limits to our zealotry – but I don’t think that means we shouldn’t try.

So my answer to the question is this: you should test and mock everything that you reasonably can.  This is not a substitute for an integration test or a nicely layered design – it’s a companion to it.  And my other plea is: remember that other people will be writing code against yours, so if you want their code to be robust, make sure your API is testable.  If only it were easier to test the boundaries of my applications, that riot might have been avoided.