Posts Tagged ‘unit testing’

Integration tests from the trenches

April 28th, 2013 No comments

This post is the written form of one of my submission for Devoxx France 2013. As it was only chosen as backup, I lacked the necessary motivation to prepare it. The subject is important though, so I finally decided to write it down.

In 2013, if you’re a standard developer, it is practically a given that you test your code. Whether you’re a practicioner of TDD or just create them afterwards, most realize a robust automated test harness is not optional but mandatory.

Unit Test vs Integration Test

There’s some implicit behind the notion of ‘test’ though. Depending on the person you ask, there may be some differences of what a test is. Let us provide two definitions to prevent potential future misunderstanding.

Unit test
A unit test tests a method in isolation. In this case, the unit is the method. Outside dependencies should be mocked to provide known input(s). This input can be set different values, including bordercase values, to check all code branches.
Integration test
An integration test tests the collaboration of two (or more) units. In this case, the unit is the class. This means that as soon as you want to check the execution result of more than one class, it is an integration test.

On one hand, achieving 100% UT sucess with 100% code coverage is not enough to guarantee a bug-free application. On the other hand, a complete IT harness is not feasible in the context of real-world project since it would cost too much. The consequence is that both are complementary.

Knowing the good IT & UT ratio is part of being a successful developer. It depends on many factors, including some outside the pure technical, such as the team skills and experience, application lifetime and so on.

Fail-fast tests

Most IT are usually inherently slower than UT, as they require some kind of initialization. For example, in modern-day applications, this directly translates to some DI framework bootstrap.

It is highly unlikely that IT be successful while UT fail. Therefore, in order to have the shortest feedback loop in case of a failure, it is a good idea to execute the fastest tests first, check if they fail, and only execute slowest ones then.

Maven provides the way to achieve this with the following plugins combination:

  1. maven-surefire-plugin to execute unit tests; convention is that their names end with Test
  2. maven-failsafe-plugin to execute integration tests; names end with IT

By default, the former is executed during the test phase of the Maven build lifecycle. Unfortunately, configuration of the latter is mandatory:


Note the Maven mojo is already bound to the integration-test phase, so that it is not necessary to configure it explicitly.

This way, UT – whose name pattern is *Test, are executed during the test phase while IT – whose name patter is *IT, are executed in the integration-test phase.

System under test boundaries

The first step before writing IT is to define the SUT boundaries. Inside lies everything that we can manage, outside the rest. For example, the database clearly belongs to the SUT because we can easily provide test data, whereas a web service does not.

Inside the SUT, mocking or initializing subsystems depend on the needed/wanted integration level:

  1. To UT a use-case, one could mock DAOs in the service class
  2. Alternatively, one could initialize a test database and use real world data. DBUnit is the framework to use in this case, providing ways to initialize data during set up and check results afterwards.

Doing both would let us test corner cases in the mocking approach (faster) and the standard case in the initialization one (slower).

Outside the SUT, there is an important decision to make regarding IT. If external dependencies are not up and stable most of the time, the build would break too often and throws too many red herrings. Those IT will probably we deactivated very quickly.

In order to still benefit from those IT but so that they do not get ignored (or even worse, removed), they should be put in a dedicated module outside the standard build. When using a continuous integration server, you should set two jobs: one for the standard build and one for those dependent IT.

In Maven, it is easily achieved by creating a module at the parent root and not referencing it in the parent POM. This way, running mvn integration-test will only launch tests that are in the standard modules, those that are reliable enough.

Fragile tests

As seen above, fragility in IT comes from dependency on external systems. Those fragile tests are best handled as a separate module, outside the standard build process.

However, another cause of fragility is unstability. And in any application, there’s a layer that is fundamentally unstable and that is the presentation layer. Whatever the chosen technology, this layer is exposed to end-users and you can be sure there will be many changes there. Whereas your service API are your own, GUI is subject to users whims, period.

IT that uses GUI, whether dedicated GUI tests or end-to-end tests should thus also be considered fragile and isolated in a dedicated module, as above.

My personal experience, coupled by some writings by Gojko Adzic, taught me to bypass the GUI layer and start my tests at the servlet layer. Whether you use  Spring Test provides a bunch of Fakes regarding the Servlet API.

Tests ordering

Developers unfamiliar with IT should probably be more than astounded by this section title. In fact, as UT should be context-independent and not be ordered in any case.

For IT, things are a little different: I like to define some UT as user stories. For example, user logs in then performs some action and then another. Those steps can of course be defined in the same test method.

The negative side of this, is that if the test fails, we have no way of easily knowing what went wrong and during which step. We could remedy to that by isolating each step in a single method and ordering those methods.

TestNG – a JUnit offshoot fully integrated in Maven and Spring, let us do that easily:

public class MyScenarioIT {

    public void userLogsIn() {
    @Test(dependsOnMethod = "userLogsIn")
    public void someAction() {
    @Test(dependsOnMethod = "someAction")
    public void anotherAction() {

This way, when an error occurs or an assert fails, log displays the faulty method: we just need to have adequate name for each method.

Framework specifics UT

Java EE in-container UT

Up until recently, automated UT were executed independently of any container. For Spring applications, this was no big deal but for Java EE intensive applications, you had to fake and mock dependencies. In the end, there was no guarantee really running the application inside the container would produce expected results.

Arquillian brought a paradigm shift, with the ability to produce UT for applications using Java EE. If you do not use it already, know that Arquillian let you automate creation of the desired archive and its deployment on one or more configured application servers.

Those can be either existing or downloaded, extracted and configured during UT execution. The former category would need to be segregated in a separate module as to prevent breaking the build while the latter can safely be assimilated to regular UT.

Spring UT

Spring Test provide a feature to assemble different Spring configuration fragments (either XML or Java classes). This means that by designing our application with enough modularity, we can use desired production-ready and tests-only configuration fragments.

As an example, by separating our data source and DAO in two different fragments, we can reuse the regular DAO fragment in UT and use a test fragment declaring an in-memory database. With Maven, this means having the former in src/main/resources and the latter in src/test/resources.

@ContextConfiguration(locations = {"classpath:datasource.xml", "classpath:dao.xml"})
public class MyCustomUT extends AbstractTestNGSpringContextTests {



This is the big picture regarding my personal experience regarding UT. As always, they shouldn’t be seen as hard and fast rules but be adapted to your specific project context.

However, tools listed in the article should be a real asset in all cases.

Send to Kindle

PowerMock, features and use-cases

October 7th, 2012 No comments

Even if you don’t like it, your job sometimes requires you to maintain legacy applications. It happened to me (fortunately rarely), and the first thing I look for before writing as much as a single character in a legacy codebase is a unit testing harness. Most of the time, I tend to focus on the code coverage of the part of the application I need to change, and try to improve it as much as possible, even in the total lack of unit tests.

Real trouble happens when design isn’t state-of-the-art. Unit testing requires mocking, which is only good when dependency injection (as well as initialization methods) makes classes unit-testable: this isn’t the case with some legacy applications, where there is a mess of private and static methods or initialization code in constructors, not to mention static blocks.

For example, consider the following example:

public class ExampleUtils {

    public static void doSomethingUseful() { ... }

public class CallingCode {

    public void codeThatShouldBeTestedInIsolation() {



It’s impossible to properly unit test the codeThatShouldBeTestedInIsolation() method since it has a dependency on another unmockable static method of another class. Of course, there are proven techniques to overcome this obstacle. One such technique would be to create a “proxy” class that would wrap the call to the static method and inject this class in the calling class like so:

public class UsefulProxy {

    public void doSomethingUsefulByProxy() {


public class CallingCodeImproved {

    private UsefulProxy proxy;

    public void codeThatShouldBeTestedInIsolation() {



Now I can inject a mock UsefulProxy and finally test my method in isolation. There are several drawbacks to ponder, though:

  • The produced code hasn’t provided tests, only a way to make tests possible.
  • While writing this little workaround, you didn’t produce any tests. At this point, you achieved nothing.
  • You changed code before testing and took the risk of breaking behavior! Granted, the example doesn’t imply any complexity but such is not always the case in real life applications.
  • You made the code more testable, but only with an additional layer of complexity.

For all these reasons, I would recommend this approach only as a last resort. Even worse, there are designs that are completely closed to simple refactoring, such as the following example which displays a static initializer:

public class ClassWithStaticInitializer {

    static { ... }

As soon as the ClassWithStaticInitializer class is loaded by the class loader, the static block will be executed, for good or ill (in the light of unit testing, it probably will be the latter).

My mock framework of choice is Mockito. Its designers made sure features such as static method mocking weren’t available and I thank them for that. It means that if I cannot use Mockito, it’s a design smell. Unfortunately, as we’ve seen previously, tackling legacy code may require such features. That’s when enters PowerMock (and only then – using PowerMock in a standard development process is also a sure sign the design is fishy).

With PowerMock, you can leave the initial code untouched and still test to begin changing the code with confidence. Here’s the test code for the first legacy snippet, using Mockito and TestNG:

public class CallingCodeTest {

    private CallingCode callingCode;

    protected void setUp() {


        callingCode = new CallingCode();

    public IObjectFactory getObjectFactory() {

        return new PowerMockObjectFactory();

    public void callingMethodShouldntRaiseException() {


        assertEquals(getInternalState(callingCode, "i"), 1);

There isn’t much to do, namely:

  • Annotate test classes (or individual test methods) with @PrepareForTest, which references classes or whole packages. This tells PowerMock to allow for byte-code manipulation of those classes, the effective instruction being done in the following step.
  • Mock the desired methods with the available palette of mockXXX() methods.
  • Provide the object factory in a method that returns IObjectFactory and annotated with @ObjectFactory.

Also note that with the help of the Whitebox class we can access the class internal state (i.e. private variables). Even though this is bad, the alternative – taking chance with the legacy code without test harness is worse: remember our goal is to lessen the chance to introduce new bugs.

Features list of PowerMock is available here for Mockito. Note that suppressing static blocks is not possible with TestNG right now.

You can find the sources for this article here in Maven format.

Send to Kindle
Categories: Java Tags: , , ,

Dependency Injection on GUI components

April 9th, 2012 No comments

For some, it goes without saying but a recent communication made me wonder about the dangers of implicitness in it.

In my book Learning Vaadin, I showed how to integrate the Vaadin web framework with the Spring DI framework: in order to make my point, I wired Vaadin UI components together. I did this because I didn’t want to go into the intricacies of creating a service layer. I had some comments questioning whether injecting UI components was relevant. The question applies equally to Vaadin and Swing. IMHO, I think not and if there’s a second version of Learning Vaadin, I will either write a warning in big bold font or choose a more complex example.

Why so? Here’s my reasoning so far. There are some reasons to use DI. To just name a few:

  • If I wanted to be controversial, I’d say that some use it because it’s hype (or because they’ve always done so)… but people that know me also know I’m not this kind of guy.
  • Some developers (or Heaven forbids, architects) use DI because it breaks dependencies
  • As for myself, the main reason I use DI is to make my classes more testable.

Now, UI components may have to collaborate with 3 other kinds of components: other UI components, GUI behavior and services. Since I consider the Vaadin framework to be top quality, I won’t unit test available components, only the behavior I’ve coded (more info on how to separate UI components and their behavior here) and services. Therefore, wiring  UI components together has no real value for me.

Injecting behavior and services is another matter: since those will likely need to be unit-tested and have dependencies themselves (either on the service layer or the data access layer), I need them to be provided by the DI context so they’ll be injectable themselves. At this point, there are basically two solutions:

  • Either wire only services and behaviors. This will lead to a dependency to the DI framework in the UI components.
  • Or wire all components at the root and let the framework handle injection. This may be overkill in some cases. This is the choice I made when writing the example for Learning Vaadin.

I hope this post will clarify the whole inject UI components thing for you dear readers.

Send to Kindle

Shoud you change your design for testing purposes?

January 22nd, 2012 2 comments

As Dependency Injection frameworks go, the standard is currently CDI. When switching from Spring, one has to consider the following problem: how do you unit test your injected classes?

In Spring, DI is achieved through either constructor injection or setter injection. Both allow for simple unit testing by providing the dependencies and calling either the constructor or the desired setter. Now, how do you unit test the following code, which uses field injection:

public class MyMainClass {

  private MyDependencyClass dependency;

  // dependency is used somewhere else

Of course, there are some available options:

  1. you can provide a setter for the dependency field, just as you did in Spring
  2. you can use reflection to access the field
  3. you can use a testing framework that does the reflection for you (PrivateAccessor from JUnit addons or Powermock come to mind)
  4. you can increase the field visibility to package (i.e. default) and put the test case in the same package

Amazingly enough, when Googling through the web, the vast majority of unit testing field-injected classes code demonstrate the increased visibility. Do a check if you do not believe me: they do not display the private visibility (here, here and here for example). Granted, it’s a rather rethorical question, but what annoys me is that it’s implicit, whereas IMHO it should be a deeply thought-out decision.

My point is, changing design for testing purpose is like shooting yourself in the foot. Design shouldn’t be cheaply sold. Else, what will be the next step, changing your design for build purposes? Feels like a broken window to me. As long as it’s possible, I would rather stick to using the testing framework that enable private field-access. It achieves exactly the same purpose but at least it keeps my initial design.

Send to Kindle
Categories: Development Tags:

Mockito’ spy() method and Spring

May 21st, 2009 4 comments

Mockito is a mocking framework (see Two different mocking approaches) that is an offshoot of EasyMock. Whatever the mocking framework one uses, a common feature is the ability to mock interfaces, through the JDK Proxy class. This is well and nice, but one has to explicitly mock every method that one wants to use in the course of the test.

What if I want to mock an already existing implementation, with some methods providing behaviours that suit me ? Today, I ran across this case: Read more…

Send to Kindle