What is a unit test? Wikipedia describes unit testing as
testing individual units of code in isolation. If the code has external
dependencies, you simulate the dependencies using mock objects.
For example, if I am testing code which gets data from a
database, hopefully access to the database is via something like ODBC or JDBC.
In which case, it is possible to use a fake database (file system or memory
based) rather than say an Oracle or SQL Server driver.
If my database connection is hard coded to a particular
machine or assumes the machine is localhost then my first step is to refactor
the code to remove this dependency.
Part of the purpose of having unit test cases is so that we
can safely change the code and know we didn't break any existing functionality.
So if we need to modify the code to be able to add unit tests we have a bit of
a Catch-22 situation. The truth of the matter is, if we have been changing the
code without unit tests, changing it one more time in order to add unit tests
is actually a step in the right direction and no worse than previous
development.
Another important feature of unit tests are speed. If I am
adding a new feature and I want to be sure it hasn't broken anything, I want to
know as soon as possible. I don't want to write the feature, run the tests and
check the results tomorrow. Ideally, I want to know in seconds. Realistically,
I might have to live with minutes at first.
Test runs should be automated. If I have to make a change
and figure out what tests to run, run them and check the results there is a
strong chance I will stop running them. Especially if I'm on a tight timeline.
Ideally, I would check in my code. This will fire a trigger
which builds my code (not the entire product, just my code) and run the unit
tests against it. Putting such a build system in place is a great deal of work
but worth the effort. Every minute it takes to create this build system should
be weighed against how much time developers spend testing their code before
they check in, how many minutes testers spend finding bugs, how much time
developers take understanding the bug and fixing it. Numerous studies have
shown fixing bugs is much more expensive than never introducing them in the
first place.
So what do we need so far?
First, we need a unit test framework. You wouldn't create
your own replacement for JDBC/ODBC. So why create your own unit test framework.
There are plenty of them out there.
Second, we need mocking frameworks for the technologies we
are utilizing. Which mock object frameworks you require depends on what you are
using in your application. If it is a web application, you might need to mock
out the web server. If it accesses a database, you will need to mock out the
database.
Third, we need a build system to automate the running and
reporting of the unit tests. Reporting the results is important too. Most
systems will either report back to the source control client or send you an
email. If the tests run in, literally, seconds, you can afford to reject the
checkin if a unit test fails. If it takes more than say 5 seconds, you might
want to send an email when a checkin fails.
Fourth, we need commitment from management and the team. If
you don't believe there is benefit to unit testing there will be no benefit to
unit testing. Training people on how to create good unit tests and maintain
them is critical. If I'm starting a new project and writing tests from the
beginning it is easy but the majority of you will be adding unit tests to
existing code.
The first three things are relatively easy to obtain. There
are plenty of technologies and examples of people using them. The fourth
requirement is the biggest reason adopting unit testing fails. If you don’t get
buy in from everyone involved it just won’t work. The developers need to
understand this will benefit them in the long run. The testers need to
understand that less testing will be required and they need to focus on things
unit testing will not catch. There will always be plenty of things to test. So
there should be no fear unit testing will replace integration or system testing. Management has
to understand if they cut timelines for a project, they will not give
developers time to write the unit tests. If you reward the Project Manager for
getting the project out on time, he will get the project out on time even if it
means giving developers no time for unit test creation. As a Project Manager,
if reducing the number of issues AFTER the project has shipped is not a metric
I’m evaluated on, I’m happy to ship a product which will make the next project
difficult to get out on time.
So, you have the tools and you have buy in from everyone.
Now what? If you have 100,000+ lines of code, where do you start writing unit
tests? The answer is actually really simple. For example piece of code a
developer touches, they should add unit tests. Bug fixing is the best place to
start. I would FIRST write a unit test which would have caught the bug. Then I’d
fix the bug and see the unit test pass.
By focusing on unit tests for bug fixes it reduces the need
for regression testing, it focuses on the features customers are using and the
developers are in that code anyways. If we need to refactor the code to support
unit testing, might as well happen as we are changing the code. The code was
broken when we started the bug fix. So
we’ll have to manually test the fix without unit tests. Hopefully, with a unit
test in place, it will be the last time we manually test changes to this code.
If we are modifying the code for feature creation, not bug
fixing, we want to write unit tests to confirm the current behaviour. Once we
have a test which passes with the current code, we can add the feature and the
tests should continue to pass.
At this point we know what we need and where to start. So
let’s cover some of the how to write a unit test.
First, a unit test is going to be a function/method which calls
our code. We want the name of the unit test to reflect what it is testing. When
results are published they will go out to the developer but they will also be
seen by the backup developer, project management and various other people as
well. If I got an email telling me test17() failed I’m going to have to open
the code and read what test17() is testing. You added comments and kept them up
to date, right? Or course you didn’t. The comments shouldn’t be necessary. The
test name should tell me what it is doing. If the test method was called, callingForgotPasswordWhenNoEmailInUserPreferences()
then we all know what is being tested.
Second, what failed? Most unit test frameworks has assert
statements. There is the basic fail() call but there are also things like
AssertTrue, AssertEquals, AssertNotNull, etc. They can be called with just what
you are checking or with a message and what you are checking. You don’t want to
code any more than you have to but enough that someone receiving the results
will know what failed. If the requirement for my software is “When a user
clicks the Forgot Password button but they have not set an email address in their
preferences, they should be presented with a message telling them to contact
the system administrator.” Then the result message from my example here might
be something like, “callingForgotPasswordWhenNoEmailInUserPreferences() failed.
Was expecting: ‘No email address was set for your account. Please contact the
System Administrator.’ but received: ‘No email address.’”. From this is it
pretty clear what was expected and what we received instead. Failing to tell
the user how to proceed should be considered a show stopper for the customer.
On the other hand, if the result was: “callingForgotPasswordWhenNoEmailInUserPreferences()
failed. Was expecting: ‘No email address was set for your account. Please
contact the System Administrator.’ but received: ‘No email address was set for
your account. Please contact the system administrator.’” the customer might
consider this acceptable. We might even update the unit test case to ignore
case so the test becomes a pass.
Unit test frameworks are pretty well established now. The
general structure of a unit test is:
- set up for the test
- run the test
- assert the test passed
- clean up so the next test starts at the same point
The set up would be things like creating mock objects, initializing
the inputs for the test, etc. The running of the test would be a call to the
method being testing. Next would be an assert statement confirming that we received
the expected results or side effect. Finally, clean up (often called tear down)
the environment so it is at the exact same condition it was before the set up
occurred.
Often you will group similar tests in one test suite. If I
have 12 tests and they all require the same set up I will put them all in one
suite. The code will then have one setUp() method that creates the environment
for each test, one method for each test (12 methods in total for this example)
and one tearDown(). The setUp() method will create any mock objects, initial
global variables, etc. The test method will create anything particular to that
test, call the method being tested then make an assert call. The tearDown()
method will then clean up the environment so it is just like it was before the setUp()
method was called. This is important because most unit test frameworks do no
guarantee the order the tests will be run. Assuming one test starts where a
previous test left off is just bad practice. I have worked on a project with
45,000 unit tests. All test are run as part of the nightly build. Rather than
running all the tests on one machine, they are distributed to 238 different
machines. If they all ran on one machine they would take 378 hours (over 2
weeks) to run. By distributing them over 238 computers they run in
approximately 3 hours. However, if test1932 depends on test1931 and the two
tests get sent to different machines, test1932 will not run correctly. Each
test must be independent of all other tests. This will not seem important at
first but 1 year later you might find yourself needing weeks (possibly months)
to refactor all your unit tests. Moments like these often cause management to
abandon unit testing.
This is unit testing is a nutshell. I will warn you, ‘the
devil is in the details.’ Hiring someone who has gone through the pains of
setting up a unit test framework is always a good idea. Either find a good
consultant or hire someone full time to work on the framework for you. Some unit
test frameworks are jUnit for Java, cppUnit for C++, nUnit for .NET, etc. Gerard
Meszaros has written an excellent book called “xUnit Test Patterns: Refactoring
Test Code”. In it he talks about “Test Smells”. Essentially, you can sometimes
look at a piece of code and say, “This code stinks.” A code or test ‘smell’ is
an indicating that the code has problems, i.e. it stinks. I have found reading Gerard
Meszaros book I know what to look for before I do it. Originally the book was
designed for people who created unit tests, found the tests have issues, i.e.
they ‘smell’ and are looking to fix them, i.e. refactor. By reading the book, I
avoid creating the bad unit tests in the first place.
Good luck and have fun!
