Migrating from JUnit 3 to JUnit 4: Nothing but Good News

Migrating from JUnit 3 to JUnit 4: Nothing but Good News

By Charles A. Sharp, OCI Senior Software Engineer

August 2007

TODO: Migrate the JUnit 3.8.1 tests to 4.4...

There are a lot of JUnit 3 tests out there. The emphasis on unit testing and test driven development has resulted in developers creating a significant corpus of JUnit tests using the JUnit 3 test framework. The migration of these unit tests to JUnit 4, though, has been fairly slow. Some of the reason for this is the less than rapid move from JDK 1.4.x to Java 5 (or later).

Another reason is the lack of enthusiasm (admittedly difficult to summon) for learning new features and migrating to a new version of a framework that serves so well just as it is today.

The time approaches, however, when the features offered by JDK 5, 6, and before long, 7, will be sufficiently compelling that migration to a later Java release will occur. When that Java migration day arrives, the ability to migrate to JUnit 4 arrives with it.

So what happens to your JUnit 3 unit tests should you want to step up to JUnit 4? Do you have to modify all your tests to keep them running? (No.) How much modification is necessary? (Some but not much.) Are there any advantages, other than avoiding version obsolescence, to migrate to JUnit 4? (A very definite yes.)

This article assumes that you're considering a move to JUnit 4 and would like to gain some idea of the amount of work necessary to modify a JUnit 3 test to take advantage of the features of JUnit 4. To that end, it provides information to answer the previous questions more completely.

Prerequisites

Before we get to the details of test migration, let's look at what you need to be working with. On the Java side, the only prerequisite is Java 5 or later. Simple enough. As always, the Java JDK is available for download at Sun's Java website.

Why do you need 1.5 or later? Primarily for annotations. JUnit 4 also uses generics and the related autoboxing to good advantage but the most visible new aspects of JUnit 4 derive from the use of annotations. With the release of JDK 1.5 in the fall of 2004, the JUnit creators changed JUnit to an annotation-based framework. As it stands today, JUnit is an excellent example of leveraging the power of annotations -- but the implication of an annotation-based technology is that the developer is running JDK 1.5 or later.

On the JUnit side, you should get the latest release of JUnit; currently that's 4.4. This can be obtained from the JUnit website. Once you have JUnit 4 installed, ensure that the JUnit 4 jar (the current jar is junit-4.4.jar) replaces the JUnit 3.x jar in any classpaths used in testing.

And that's it for prerequisites, excepting the not-so-minor assumption that you know what a JUnit test is. This article deals with the migration from JUnit 3.x to 4.x. If you are new to JUnit and are looking for an introduction to the JUnit testing framework, I would recommend a look at the JUnit Cookbook, the introductory article for many of us. The book Pragmatic Unit Testing by Andrew Hunt and David Thomas also provides introductions (and more) to both Unit Testing and JUnit.

The Good News of JUnit 4

I might as well get this in black and white. I consider just about everything concerning JUnit 4 to be good news. And the first piece of the good news is that all 3.x unit tests run without modification using the JUnit 4 jar.

JUnit 3 uses junit.framework.* packages and JUnit 4 uses an org.junit.* package hierarchy. By using different package structures, the JUnit folk have ensured JUnit 3 tests continue to run without complaint. JUnit 4 ships with both packages so you can easily make your fundamental first step of migration which is to use the new JUnit 4 jar in place of your current JUnit 3 jar. After doing that all your JUnit tests continue to run as before -- no pain involved.

This is excellent news as it permits developers to migrate their tests gradually and gracefully rather than forcing a big-bang, change-everything-all-at-once approach so often needed to move to a newer release of an established piece of software.

There are two distinct phases in migrating to JUnit 4. Phase one is the conversion of existing JUnit 3 test cases to their JUnit 4 analogues. The second phase is taking advantage of the new features of JUnit 4 as you refactor existing tests and write new ones.

Conversion: Changing the Past

Imports

Since the JUnit 4 package structure is different, you'll need to change the junit.framework.* imports to org.junit.* imports. The org.junit packages are similar in structure, and clearly documented in the javadocs. The new annotations follow the form of org.junit.. For example, the @Test annotation is imported using import org.junit.Test;.

New with JUnit 4 is the need to import the static Assertion methods. In JUnit 3, a test class would extend the TestCase class which in turn extended the Assert class and made the assertion methods available. JUnit 4 tests no longer extend TestCase, so you must specifically import the asserts. To obtain the asserts, use another JDK 1.5 feature, the static import. This looks like:

     import static org.junit.Assert.*;

The org.junit.Assert class comprises only static methods that define all Assert methods used in JUnit 3.8 and a couple of new ones discussed below. The static import as used above makes them all available. Of course, simply importing the methods used by the test is also a clean option.

From a conversion standpoint, the presence of all the JUnit 3 asserts in JUnit 4 means that you do not have to convert any of the asserts used in the tests being migrated.

Tests

Tests are naturally the focus of unit testing, so let's look at them next. Here are some rules for creating test methods using JUnit 3.x. Let's see how the rules work in JUnit 4.

JUnit 3JUnit 4
Must be public, [return void], and require no arguments Still true.
Must begin with the name "test". Ah! The first major departure. Instead of using a naming convention to identify a test method, in JUnit 4, any method annotated with "@Test" is a test method. Which means that you can name your test method anything. For those who think that test method names starting with "test" make a lot of sense, well, your method names can still start with "test".

For ease and speed of conversion, you simply need to add the @Test before the public void of your test methods and you're in business.
May optionally throw exceptions which will be reported as errors by JUnit. Still true for errors. But on the whole, exception testing in JUnit 4 is more straight forward than in JUnit 3. If you wish to indicate that you expect an exception to be thrown by a test method, you can now add an "exception" parameter with the expected exception class as the value to the @Testannotation.

It looks like this: @Test(exception=ArrayIndexOutOfBounds.class).

This example states that somewhere in the test method an ArrayIndexOutOfBounds exception will be thrown. If this does not happen, the test fails. In JUnit 3, this type testing was accomplished by try/catch for the success path. If the catch clause was not entered, the test failed. For example, here's a JUnit 3.8.1 test that expects an exception: 
    public void testExceptionThrow() {
      try {
          String[] ary = new String[1];
          ary[0] =  "zero";
          ary[1] =  "one";
          fail("ArrayIndexOutOfBoundsException was not thrown");
      } catch (ArrayIndexOutOfBoundsException e) {
        // The test passes when this clause is entered.
      }
    }

Here's the same test using JUnit 4:

    @Test(expected=java.lang.ArrayIndexOutOfBoundsException.class)
    public void testExceptionThrow() {
      String[] ary = new String[1];
      ary[0] =  "zero";
      ary[1] =  "one";
    }

It is worth mentioning that if your test needs to examine the contents of the thrown exception, you will need to use the JUnit 3 approach. The @Test exception parameter does not provide the ability to examine the thrown exception.
Asserts are reported as failures  Here's a difference you may not like. JUnit 4 no longer distinguishes failures from errors. A test either passes, fails, or is ignored. (@Ignore is discussed in a bit.) Sometimes it is useful to know if the test failed due to error or an assertion, but that doesn't happen now. As an aside, the failures note where the failure occurred so it isn't really that difficult to find - - but for automated testing, the red flag went away.
OK to use assert several times in the same method.  Still true.

For converting a test method, that's all you need to know. There is one more thing, though, that is very nice about the @Test annotation. JUnit 4 now gives you the ability to specify a timeout value in milliseconds using the timeout parameter on the @Test annotation. Here's what it looks like: @Test(timeout=50). This construct specifies a timeout value of 50 milliseconds. If the test method execution time exceeds the annotated timeout value, the test fails.

One possible question is, "Can you specify both timeout and exception parameters on a single @Test annotation?" Yes, indeed. But no more than one of each.

Test Fixtures, aka setUp and tearDown

Tests many times require some programmatic preparation before running. This preparation is often matched by some clean up afterwards. The JUnit 3 tests accomplish this by the creation of a test fixture, that is, a test class containing a set of tests with methods named setUp and tearDown.

To convert a JUnit 3 test fixture to JUnit 4, the setUp method, which is executed before each test method, is replaced by a method with the @Before annotation. The tearDown method is replaced by a method annotated with @After.

Just as the @Test methods no longer need to follow a naming convention, neither do the @Before and @After methods. A side effect of this is that you can now have more than one method annotated with @Before or @After -- as many as you want. Each of the @Before methods will be invoked before each test and each of the @After methods will be invoked after each test. But Note Well! Even though you can have more than one method, there is no guaranteed order of invocation. This means that any dependencies in the setUp or tearDown requirements for the tests must still be handled by the logic in your @Before and @After methods.

Suites

In JUnit 3, if you add a static public method named "suite" that returns an object implementing the Test interface (this object is usually an instance of a TestSuite class), then JUnit will run all the tests in the returned collection of tests. This permits the tester to group tests into logical chunks that can be grouped together. It is a bit trickier to talk about converting JUnit 3 test suites because over time, several patterns of suites have developed. The simplest case is a class that comprises nothing but the suite() method, like this:

  1.     import junit.framework.TestSuite;
  2.  
  3.     public class JRandomJUnit3TestSuite {
  4.      public static Test suite() {
  5.          TestSuite suite = new TestSuite();
  6.          suite.addTestSuite(TestsForOneClass.class);
  7.          suite.addTestSuite(TestsForAnotherClass.class);
  8.          return suite;
  9.      }
  10.     }

You can convert a test suite that follows this pattern to a JUnit 4 suite by the use of two new annotations, @RunWith and @Suite.SuiteClasses. The org.junit.runner.RunWith annotation takes a default value of a Runner class. This class will be used to run the tests rather than the default JUnit 4 Runner. The org.junit.runners.Suite class is a runner that permits the tester to specify the classes containing tests to be run. (SuiteClasses is a nested annotation of Suite.) These two annotations permit you to generate the equivalent of the above test suite like this:

  1.     import org.junit.runner.RunWith;
  2.     import org.junit.runners.Suite;
  3.  
  4.     @RunWith(Suite.class)
  5.     @Suite.SuiteClasses({
  6.       TestsForOneClass.class,
  7.       TestsForAnotherClass.class
  8.       })
  9.     public class JRandomJUnit4TestSuite {
  10.     }

Notice that the annotations are annotating the class. Until this example, the annotations have been applied to methods.

A JUnit 4 feature that relates to suites is the ability to do a one-time setUp and tearDown.

There are two groups of activities that are often necessary to run tests. One group comprises activities that really only need to be done once before all the tests are run and once after they complete. The other group consists of setup that needs to be performed before each test and teardown that cleans up after each test. As we've seen, the "each test" group is handled with the @Before and @After annotations.

The "just once" group is handled by the @BeforeClass and @AfterClass annotations introduced in JUnit 4. Please note, however, that unlike the @Before and @After methods, the signatures on methods annotated with @BeforeClass and @AfterClass are public static void, not public void. The @BeforeClass methods (like the @Before annotation, there can be more than one method annotated with @BeforeClass) are run before any tests and the @AfterClass methods are run after all tests in the class complete.

One more thing about the @BeforeClass and @AfterClass annotations. All similarly annotated superclass methods bookend the subclass methods. This is to say that all superclass methods annotated with @BeforeClass are executed, then the @BeforeClass methods of the current class are run. After the tests are run, the @AfterClass methods of the current class are invoked, then the @AfterClass methods of the superclass are invoked. Here are two classes that illustrate this:

  1.     import org.junit.BeforeClass;
  2.     import org.junit.AfterClass;
  3.  
  4.     public class SuperSample {
  5.       @BeforeClass
  6.       public static void abrMethod() {
  7.           System.out.print("abr");
  8.       }
  9.       @AfterClass
  10.       public static void raMethod() {
  11.           System.out.println("ra");
  12.       }
  13.     }

and

  1.     import org.junit.BeforeClass;
  2.     import org.junit.AfterClass;
  3.     import org.junit.Test;
  4.  
  5.     public class SubSample extends SuperSample {
  6.       @BeforeClass
  7.       public static void acMethod() {
  8.           System.out.print("ac");
  9.       }
  10.  
  11.       @Test
  12.       public void adMethod() {
  13.           System.out.print("ad");
  14.       }
  15.  
  16.       @AfterClass
  17.       public static void abMethod() {
  18.           System.out.print("ab");
  19.       }
  20.     }

Running the SubSample class with a TestRunner will print the string: "abracadabra".

A final note on the @AfterClass and @BeforeClass annotations. The JUnit FAQ entry for running setup() and tearDown() for all tests [rightly] points out that using these may indicate some unwarranted dependencies among your group of tests. Aside from undesired dependencies, you also need to take care that no tests inadvertently modify the environment set up by the @BeforeClass setup methods.

On the other hand, there is an excellent example of a good use of the @BeforeClass annotation in the article by Elliotte Harold, An early look at JUnit 4. When discussing suite-wide initialization, he illustrates how to redirect the undesirable System.err output being generated by the use of a third-party class into a file, thereby avoiding pollution of the unit test output.

Running the tests

Once you have converted and recompiled a test, how do you run it? Unlike JUnit 3, JUnit 4 comes with only a text runner. It has neither swingui nor awtui to graphically show your test results. It appears the majority of use for JUnit is either automated, that is to say, using a text-based output, or is run via an IDE. Regardless, the JUnit authors have put their efforts into other arenas than the GUI runners and they are no longer available.

This means that you can run your JUnit tests from your IDE, ant, or from the command line. The JUnit FAQ, which I highly recommend reading, gives a pointer to a page describing IDE integration.

There is also a SourceForge project, JUnit 4 Extensions, that provides, among other interesting features, the JUnit 3.8 UI runners.

New Features of JUnit 4

While converting existing tests is the thrust of this article, once you have installed JUnit 4, you gain several other immediately useful features you should be aware of.

Asserts

The first thing you might notice if you happen to be looking at the JUnit 3 and JUnit 4 Assert class javadocs side-by-side, is that it appears many assertEquals methods have disappeared. Not to worry.

JUnit 4 takes advantage of the autoboxing available in JDK 5 so assertEquals(,) is now handled by assertEquals(Object, Object). For example, if your JUnit 3 test uses assertEquals(int, int), it will work unchanged in JUnit 4.

Here's a timesaver. You can now compare Arrays. This works for byte[], char[], int[], short[], long[], and Object[]; with optional Messages, as expected.

With the introduction of JDK 1.4, the JUnit 3 assert method was deprecated since it conflicted with the new Java assert keyword. With JUnit 4 this is no longer true. Even better, for those folk who like to use asserts in their methods, if asserts are enabled (the JVM -ea option) and an assert fails during a test, JUnit 4 will report it as a failed test.

JUnit 4.4 introduces a new assert, assertThat, and some assumptions: assumeThat, assumeNotNull, assumeNoException, and assumeTrue. The 4.4 release notes give credit to Joe Walnes for the assertThat assertion mechanism. Those familiar with the jMock project will probably recognize it.

The assertThat assertions require a parameter of org.hamcrest.Matcher type. JUnit 4.4 provides several of these Matcher classes by including a third-party package, org.hamcrest.core as well as including some in the package, org.junit.matchers. According to the release notes, this is the first time a third-party package has been shipped with JUnit. A good start on using this assertion is found in a blog post by Joe Walnes, in which he explains the rationale and intended usage of assertThat. The javadoc accompanying the org.hamcrest.core packages in the JUnit 4.4 distribution is adequate for understanding the Matcher classes in that package. The javadoc for the org.junit.matchers classes is notably absent.

@Ignorance is bliss

It's not difficult in JUnit 3 to ignore a test. You simply change the name of a test method from "public void testSomething" to "public void xxtestSomething" or some such thing. If you are consistent with your method names, you can even disable all of your test methods with a single global replacement command in your editor. Thus I can't say that the @Ignore annotation iseasier but it sure looks better. When you wish to skip one or more tests in a class, simply add the @Ignore before each @Test to be skipped. As a bonus, @Ignore will take a String value parameter that documents why the test is being ignored.

If you wish to ignore all the tests in a class, simply annotate the class with @Ignore.

Runners

JUnit 4.4 introduces some refactoring of the default JUnit 4 test runner. Quoting from the release notes:

The old version was named TestClassRunner, and the new is named JUnit4ClassRunner. Likewise, OldTestClassRunner is now JUnit3ClassRunner. ... The rules for determining which runner is applied by default to a test class have been simplified:

  • If the class has a @RunWith annotation, the annotated runner class is used.
  • If the class can be run with the JUnit 3 test runner (it subclasses TestCase, or contains a public static Test suite() method), JUnit38ClassRunner is used.
  • Otherwise, JUnit4ClassRunner is used.

This default guess can always be overridden by an explicit @RunWith(JUnit4ClassRunner.class) or @RunWith(JUnit38ClassRunner.class) annotation. The old class names TestClassRunner and OldTestClassRunner remain as deprecated.

If you just wish to see the console output of a test class, use: java -cp org.junit.runner.JUnitCore ClassToBeTested

Final Things

In closing, it is somewhat standard to give credit to Kent Beck and Erich Gamma for their JUnit framework. And quite rightly. Their work in providing the JUnit testing framework has proved to be incredibly important. I'm certainly grateful.

References