• Should tests be ordered or not?

    Everything ordered and tidy

    Most of our day-to-day job is learned through mentorship and experience and not based upon scientific research. Once a dogma has permeated a significant minority of practitioners, it becomes very hard to challenge it.

    Yet, in this post, I’ll attempt to not only challenge that sometimes tests must be ordered but prove that in different use-cases.

    Your tests shall not be ordered (shall they?)

    Some of my conference talks are more or less related to testing, and I never fail to point out that TestNG is superior to JUnit if only because it allows for test method ordering. At that point, I’m regularly asked at the end of the talk why method ordering matters. It’s a widespread belief that tests shouldn’t be ordered. Here are some samples found here and there:

    Of course, well-written test code would not assume any order, but some do.
    --JUnit Wiki - Test execution order
    Each test runs in its own test fixture to isolate tests from the changes made by other tests. That is, tests don't share the state of objects in the test fixture. Because the tests are isolated, they can be run in any order.
    --JUnit FAQ - How do I use a test fixture?
    You’ve definitely taken a wrong turn if you have to run your tests in a specific order [...]
    --Writing Great Unit Tests: Best and Worst Practices
    Always Write Isolated Test Cases
    The order of execution has to be independent between test cases. This gives you the chance to rearrange the test cases in clusters (e.g. short-, long-running) and retest single test cases.
    --Top 12 Selected Unit Testing Best Practices

    And this goes on ad nauseam

    In most cases, this makes perfect sense. If I’m testing an add(int, int) method, there’s no reason why one test case should run before another. However, this is hardly a one-size-fits-all rule. The following use-cases take advantage of test ordering.

    Tests should fail for a single reason

    Let’s start with a simple example: the code consists of a controller that stores a list of x Foo entities in the HTTP request under the key bar.

    The naive approach

    The first approach would be to create a test method that asserts the following:

    1. a value is stored under the key bar in the request
    2. the value is of type List
    3. the list is not empty
    4. the list has size x
    5. the list contains no null entities
    6. the list contains only Foo entities

    Using AssertJ, the code looks like the following:

    // 1: asserts can be chained through the API
    // 2: AssertJ features can make the code less verbose
    @Test
    public void should_store_list_of_x_Foo_in_request_under_bar_key() {
        controller.doStuff();
        Object key = request.getAttribute("bar");
        assertThat(key).isNotNull();                    // #1
        assertThat(key).isInstanceOf(List.class);       // #2
        List list = (List) key;
        assertThat(list).isNotEmpty();                  // #3
        assertThat(list).hasSize(x);                    // #4
        list.stream().forEach((Object it) -> {
            assertThat(object).isNotNull();             // #5
            assertThat(object).isInstanceOf(Foo.class); // #6
        });
    }

    If this test method fails, the reason can be found in any of the previous steps. A customary glance at the failure report is not enough to tell exactly which one.

    Single tests result

    To know that, one has to analyze the stack trace then the source code.

    java.lang.AssertionError: 
    Expecting actual not to be null
    
    	at ControllerTest.should_store_list_of_x_Foo_in_request_under_bar_key(ControllerTest.java:31)
    

    A test method per assertion

    An alternative could be to refactor each assertion into its own test method:

    @Test
    public void bar_should_not_be_null() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        assertThat(bar).isNotNull();
    }
    
    @Test
    public void bar_should_of_type_list() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        assertThat(bar).isInstanceOf(List.class);
    }
    
    @Test
    public void list_should_not_be_empty() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        assertThat(list).isNotEmpty();
    }
    
    @Test
    public void list_should_be_of_size_x() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        assertThat(list).hasSize(x);
    }
    
    @Test
    public void instances_should_be_of_type_foo() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        list.stream().forEach((Object it) -> {
            assertThat(it).isNotNull();
            assertThat(it).isInstanceOf(Foo.class);
        });
    }

    Now, every failing test is correctly displayed. But if the bar attribute is not found in the request, every test will still run and still fail, whereas they should merely be skipped.

    Unordered tests results

    Even if the waste is small, it still takes time to run unnecessary tests. Worse, it’s waste of time to analyze the cause of the failure.

    A private method per assertion

    It seems ordering the tests makes sense. But ordering is bad, right? Let’s try to abide by the rule, by having a single test calling private methods:

    public void should_store_list_of_x_Foo_in_request_under_bar_key() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        bar_should_not_be_null(bar);
        bar_should_of_type_list(bar);
        List<?> list = (List) bar;
        list_should_not_be_empty(list);
        list_should_be_of_size_x(list);
        instances_should_be_of_type_foo(list);
    }
    
    private void bar_should_not_be_null(Object bar) {
        assertThat(bar).isNotNull();
    }
    
    private void bar_should_of_type_list(Object bar) {
        assertThat(bar).isInstanceOf(List.class);
    }
    
    private void list_should_not_be_empty(List<?> list) {
        assertThat(list).isNotEmpty();
    }
    
    private void list_should_be_of_size_x(List<?> list) {
        assertThat(list).hasSize(x);
    }
    
    private void instances_should_be_of_type_foo(List<?> list) {
        list.stream().forEach((Object it) -> {
            assertThat(it).isNotNull();
            assertThat(it).isInstanceOf(Foo.class);
        });
    }

    Unfortunately, it’s back to square one: it’s not possible to just know in which step the test failed just at a glance.

    Single test result

    At least the stack trace conveys a little more information:

    java.lang.AssertionError: 
    Expecting actual not to be null
    
    	at ControllerTest.bar_should_not_be_null(ControllerTest.java:40)
    	at ControllerTest.should_store_list_of_x_Foo_in_request_under_bar_key(ControllerTest.java:31)
    

    How to skip unnecessary tests, and easily know the exact reason of the failure?

    Ordering it is

    Like it or not, there’s no way to achieve skipping and easy analysis without ordering:

    // Ordering is achieved using TestNG
    
    @Test
    public void bar_should_not_be_null() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        assertThat(bar).isNotNull();
    }
    
    @Test(dependsOnMethods = "bar_should_not_be_null")
    public void bar_should_of_type_list() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        assertThat(bar).isInstanceOf(List.class);
    }
    
    @Test(dependsOnMethods = "bar_should_of_type_list")
    public void list_should_not_be_empty() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        assertThat(list).isNotEmpty();
    }
    
    @Test(dependsOnMethods = "list_should_not_be_empty")
    public void list_should_be_of_size_x() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        assertThat(list).hasSize(x);
    }
    
    @Test(dependsOnMethods = "list_should_be_of_size_x")
    public void instances_should_be_of_type_foo() {
        controller.doStuff();
        Object bar = request.getAttribute("bar");
        List<?> list = (List) bar;
        list.stream().forEach((Object it) -> {
            assertThat(it).isNotNull();
            assertThat(it).isInstanceOf(Foo.class);
        });
    }
    

    The result is the following:

    Ordered tests results

    Of course, the same result is achieved when the test is run with Maven:

    Tests run: 5, Failures: 1, Errors: 0, Skipped: 4, Time elapsed: 0.52 sec <<< FAILURE!
    bar_should_not_be_null(ControllerTest)  Time elapsed: 0.037 sec  <<< FAILURE!
    java.lang.AssertionError: 
    Expecting actual not to be null
    	at ControllerTest.bar_should_not_be_null(ControllerTest.java:31)
    
    Results :
    
    Failed tests: 
    	ControllerTest.bar_should_not_be_null:31 
    Expecting actual not to be null
    
    Tests run: 5, Failures: 1, Errors: 0, Skipped: 4
    

    In this case, by ordering in unit test methods, one can achieve both optimization of testing time and fast failure analysis by skipping tests that are bound to fail anyway.

    Unit testing and Integration testing

    In my talks about Integration testing, I usually use the example of a prototype car. Unit testing is akin to testing every nut and bolt of the car, while Integration testing is like taking the prototype on a test drive.

    No project manager would take the risk of sending the car on a test drive without having made sure its pieces are of good enough quality. It would be too expensive to fail just because of a faulty screw; test drives are supposed to validate higher-levels concerns, those that cannot be checked by Unit testing.

    Hence, unit tests should be ran first, and only integration tests only afterwards. In that case, one can rely on the Maven Failsafe plugin to run Integration tests later in the Maven lifecycle.

    Integration testing scenarios

    What might be seen as a corner-case in unit-testing is widespread in integration tests, and even more so in end-to-end tests. In the latest case, an example I regularly use is the e-commerce application. Steps of a typical scenario are as follow:

    1. Browse the product catalog
    2. Put one product in the cart
    3. Display the summary page
    4. Fill in the delivery address
    5. Choose a payment type
    6. Enter payment details
    7. Get order confirmation

    In a context with no ordering, this has several consequences:

    • Step X+1 is dependent on step X e.g. to enter payment details, one must have chosen a payment type first, requiring that the latter works
    • Step X+2 and X+1 both need to set up step X. This leads either to code duplication - as setup code is copied-pasted in all required steps, or common setup code - which increases maintenance cost (yes, sharing is caring but it’s also more expensive).
    • The initial state of step X+1 is the final state of step X i.e. at the end of testing step X, the system is ready to start testing step X+1
    • Trying to test step X+n if step X failed already is time wasted, both in terms of server execution time and and of failure analysis time. Of course, the higher n, the more waste.

    This is very similar to the section above about unit tests order. Given this, it makes no doubt for me that ordering steps in an integration testing scenario is far from a bad practice but good judgement.

    Conclusion

    As in many cases in software development, a rule has to be contextualized. While in general, it makes no sense to have ordering between tests, there are more than a few cases where it does.

    Software development is hard because the “real” stuff is not learned by sitting on universities benches but through repeated practice and experimenting under the tutorship of more senior developers. If enough more-senior-than-you devs tend to hold the same opinion on a subject, chances are you’ll take that for granted as well. At some point, one should single out of such opinion and challenge it to check whether it’s right or not in one’s own context.

  • Are you guilty of over-engineering?

    Construction site

    If you listen to other language communities - such as Python or Ruby, it seems Java developers have a strong tendency of over-engineering.

    Perhaps they’re just jealous of our superior platform (wink), perhaps there is some very slight reason that they believe so. I do believe so. And it’s quite interesting that I realized it by doing code review - while I may be guilty of over-engineering myself when writing code. But I’m working on it.

    Of course, you’re a “simple” developer, and only architects have the power to design, so only they can be guilty of over-engineering, aren’t they? I’m afraid that’s not the case: developers do that all the time. In this article, I’ll focus on one symptom that I commented a lot in my reviews, but it can be extended to many others.

    I’ve been taught early in my career to design my class hierarchy to be as extensible as possible. It translates into a parent interface, implemented by a concrete class. Sometimes, there might even be an abstract class in between. It looks like the following diagram:

    For example, the Spring framework uses this design a lot, throughout its packages. One great example is the ViewResolver interface, which has a rich children hierarchy and a lot of different implementing classes e.g. InternalResourceViewResolver and VelocityLayoutViewResolver among others. Other examples abound (bean registry, context, etc.) in different parts of the framework.

    However, please note quite important facts about Spring:

    1. There are a lot of different children classes organized within a structured hierarchy
    2. It’s a framework, and it’s meant to be open for extension and closed to modification by definition

    Back to our current project, let’s say a regular web app. Locate one of the defined interface. From this point, it’s quite easy to find its children implementations; in most cases, there’s only one and it’s prefixed with Default (or alternatively suffixed with Impl). Also, most of the time, the interface resides in the x.y.z package and the implementation in the x.y.z.impl. One might wonder about such utter lack of imagination. Well, coming up with a relevant name for the implementation is hard because there is no semantic difference between it and the interface i.e. the former doesn’t specialize the later. The only possible conclusion is that the interface is not necessary…

    Some architects might argue that even if useless now, the abstraction might be useful at a later time. Yet, adding unnecessary code only because it might become useful in the future just decreases the signal to noise ratio of the code base - it hinders readability for no reason. Moreover, this extra code needs to be created, tested and maintained, hence increased costs and waste. It’s also opposite to the Agile way of “just enough” to achieve the goal. Finally, if (and only if) the need arises, introducing a parent interface is quite easy by just changing the code of the app. It’s a no-brainer, thanks to refactoring features of any IDE worthy of the name.

    I would qualify the above approach as over-engineering if one designs it on purpose to prepare for an hypothetical future but as Cargo Cult if one does it just because it has always been done like this.

    Useless interfaces are but one simple example of over-engineering. There are many more. I’d happy to read the ones you’ve found (or practiced yourself).

  • Adventures in SEO with Vaadin

    Vaadin logo

    TL;DR: Vaadin was hardly SEO-friendly in the past. Not anymore, with the new Volga library.

    Bookmarking pages

    Bookmarking is as old as www itself. Being able to save an URL is part of the ADN of websites. Regarding web apps, this is somewhat different. For example, in an e-commerce webapp, while it does make sense to bookmark a specific product, bookmarking a specific step of the checkout process does not.

    Following the shop example, here’s what happens in a traditional servlet-based context:

    1. A servlet is mapped on a specific subcontext such as /product/*
    2. When the URL /product/really-cool-product is called, the doGet() method of that servlet is called
    3. The method parses the URL to read the really-cool-product part - which should be an unique key for the product
    4. It delegates to a whole chain of components that loads the product from the datastore
    5. It forwards to a JSP along with the relevant product data
    6. This JSP generates the HTML

    Single-Page Applications

    Come SPAs. By definition, they serve all content under the same URL. This makes bookmarking specific pages of the application impossible because there are no pages per se. In general, SPAs handle this problem with fragment identifiers. The above URL becomes /product#really-cool-product, problem solved. In Vaadin, this directly translate to usage of the Page.getCurrent().setUriFragment() method or of the Navigator API.

    Unfortunately, this doesn’t work at all with the crawling part of SEO. Fragments are not discriminatory parts of an URL: #really-cool-product and #another-cool-product do point to the same URL so bots such as Google Bot won’t crawl both.

    The fragment identifier functions differently than the rest of the URI: namely, its processing is exclusively client-side with no participation from the web server.
    -- Wikipedia

    Distinct URLs for SPAs

    Back to square one, both /product/really-cool-product and /product/another-cool-product paths are required. This problem is not unique to Vaadin, but common to all server- and client-side SPA frameworks. What is required is:

    1. To have the client change the browser’s URL with no server interaction - no page reload nor AJAX calls
    2. To have the server handle paths

    In JavaScript, the answer is to use the History API. I assume everyone is familiar with the following snippet:

    window.back();
    window.go(-1);
    

    This is however absolutely not standard. This should be replaced by the following:

    window.history.back();
    window.history.go(-1);
    

    The history object implements the History API. In particular, it API makes it possible to add entries in the browser history via the pushState() method.

    Suppose http://mozilla.org/foo.html executes the following JavaScript:
    var stateObj = { foo: "bar" };
    history.pushState(stateObj, "page 2", "bar.html");
    This will cause the URL bar to display http://mozilla.org/bar.html, but won't cause the browser to load bar.html or even check that bar.html exists.
    --Mozilla Developer Network

    On the server-side, handling different paths is quite trivial - even though some designs are better than others.

    Beyond distinct URLs

    Distinct URLs is only the emerged part of the iceberg regarding SEO.

    One wants to have dedicated meta headers for each dedicated URL, such as <title> and <meta name="description">. Even further, social medias have their own dedicated meta headers, e.g.:

    Volga, the SEO-friendly Vaadin library

    Implementing the steps above from scratch in your Vaadin project is definitely not trivial. Rejoice, for comes Volga, a ready-to-use library that handles the brunt of things for you.

    To use it, just add this snippet to your POM:

    <dependency>
        <groupId>org.vaadin</groupId>
        <artifactId>volga</artifactId>
        <version>0.1</version>
    </dependency>
    

    Important part of the API include:

    org.vaadin.volga.VolgaDetails
    Holds a set of metadata headers
    org.vaadin.volga.VolgaServlet
    Set the VolgaDetails for the root path and provides bindings between a path and other VolgaDetails objects. This way, each specific path can be set their own VolgaDetails.
    org.vaadin.volga.Volga
    Holds the previously defined mappings
    org.vaadin.volga.VolgaUI
    Handles the initial configuration
    org.vaadin.volga.SeoBootstrapListener
    Fills page metadata headers from a VolgaDetails object

    For more details, please check this example project on Github. It’s deployed online and here are results shown on Google search that proves that it works.

    Demo application as seen on Google Search

    This works for Twitter as well:

    Twitter card for main view Twitter card for second

    Categories: Java Tags: SEOvaadinvolga
  • Limits of the technical debt analogy

    The triangle of project management is a well-known model, used through and through since ages.

    Triangle of Project Management

    I assume most software developers, even junior ones, are familiar with those:

    • Time
    • Cost
    • Scope

    As for my personal experience in the software industry, all projects have been behind schedule, with only a few exceptions for small projects (some man-months). The reason might probably because initial estimations were too low, either by accident or on purpose, but that is a subject best tackled in another post.

    Now, Project Managers also have been taught about the Triangle of Project Management. The idea behind this figure is that when things start to go badly, in one or more of the edges, one has to make tradeoffs on the other(s) edge(s). For example, when the project runs behind schedule, scope might be reduced to keep it on track.

    However, the above triangle is wrong - or at least incomplete. The real model should look like this:

    Triangle of Project Management

    This version of the model is rarely shown, plus quality is hard to define, hard to measure and doesn’t appear in most dashboards. In essence, when schedules run tight and/or costs start to over-burn, nobody cares about Quality anymore (if anyone did before). It’s a death march toward delivery here and now, and nothing should stand in the way.

    I wrote it’s hard to define Quality. That’s true even for software engineers. Even widely accepted metrics such as link:{% post_url 2014-10-05-your-code-coverage-metric-is-not-meaningful %}[code coverage^] are not meaningful. Lack of Quality however, is universally acknowledged by programmers: a seemingly simple feature takes ages to develop, because the existing code is a tangled mess, or there are no/not enough tests, or…​ (insert you favorite here).

    Because software engineers are despite widespread rumors not only creative people but also communicative ones, they came up with the concept of technical debt:

    Technical debt (also known as design debt or code debt) is "a concept in programming that reflects the extra development work that arises when code that is easy to implement in the short run is used instead of applying the best overall solution".

    Technical debt can be compared to monetary debt. If technical debt is not repaid, it can accumulate 'interest', making it harder to implement changes later on. Unaddressed technical debt increases software entropy.

    — Wikipedia

    That was a stroke of genius! Even people who were clueless about code, software development and software quality could understand finance fundamentals. If you get indebted to the Great Bank of Software, then you’ll have to repay your debt later…​ plus interest.

    However, I’m afraid this idea has only marginal benefits compared to the previous situation, because of one very persistent trend, silos. Think about organizations work: when an idea forms, it gets translated into a project, with a dedicated budget. To steer the project is the Project Manager. Now, if the project succeeds (and yes, it happens), the application becomes part of the assets portfolio and is considered a product. It’s assigned a Product Owner, who in most cases is not the same person as the Project Manager. In layman’s terms, the Project Manager incurs the debt and ru(i)ns to the next project while the Product Owner has to pay the debt!

    An "interesting" development is for the PO to continue to incur technical debt to lower maintenance costs in order and thus improve his career path, get promoted as fast as possible and pass the buck to the next poor sap in the line.

    In this great age of silos of everything, projects, departments, information systems, budgets and of course responsibilities, even if everyone can understand technical debt, many choose not to care about it - and leave the next guy to handle the problem they created in the first place.

    Of course it’s not specific to software, as political figures unfortunately prove every day…​

    Categories: Development Tags: technical debtrefactoringsoftware engineering
  • Scala vs Kotlin: inline and infix

    This is the third post in the Scala vs Kotlin comparison serie:

    1. Pimp my library
    2. Operator overloading

    This week, I’d like to address two features: inline and infix - not because they’re related but because neither of them would be enough to fill a post.

    Inlining comes from C (and then C++). In those languages, a hint could be provided to the compiler through the inline keyword. By doing so, it may replace an inlined function call by the function body itself in order to skip the overhead of a function call.

    Infix notation is to be in line with prefix and postfix, it’s relative to the place of the operator compared its 2 operands. Hopefully, the following example is clear enough:

    • Prefix: + 2 2
    • Postfix: 2 2 +
    • Infix: 2 + 2

    Scala

    Scala offers inlining through the @inline annotation on a function. As for C/C++, this is a an hint to the compiler. As per the ScalaDocs:

    An annotation on methods that requests that the compiler should try especially hard to inline the annotated method.

    The compiler has the final say in whether the function will be inlined, or not. On the opposite site, a function can be annotated with @noinline to prevent inlining altogether:

    An annotation on methods that forbids the compiler to inline the method, no matter how safe the inlining appears to be.

    As for infix annotation, it’s interestingly quite different from the definition above. In this context, it means that dot and parentheses can be omitted while calling functions that have a single parameter. There are some additional constraints:

    • Either the function must have no side-effects - be pure
    • Or the parameter must be a function
    1
    2
    val isLess1 = 1.<(2)
    val isLess2 = 1 < 2
    

    Lines 1 and 2 are equivalent. Obviously, line 2 is much more readable. Thanks to infix annotation, Scala doesn’t need operators, as every function can not only look but be called like an operator.

    Kotlin

    In Kotlin, inlining is set with the inline keyword. However, it’s much more than just a compiler hint: it’s a requirement. Whenever inline is used, the compiler will inline the function, no matter what.

    As such, it’s very important to use inlining only on small functions. Other limitations might include keeping its use to code under our control, e.g. to use it only for application code or code that is not part of a library’s public API.

    Note that inlining affects both the function itself as well as arguments that are lambdas. To make lambda arguments not inlined, use the noinline keyword.

    Infix notation is not automatic in Kotlin as it requires the function to be marked with the infix keyword. Additionally, the function needs to be attached to a class, either because it’s a member or an extension. Of course, the single parameter still applies.

    // Defined in Kotlin's runtime
    infix fun and(other: kotlin.Int): kotlin.Int { /* compiled code */ }
    
    val bool1 = 1.and(2)
    val bool2 = 1 and 2
    

    Be aware that infix notation only looks similar to an operator, it’s still a regular method call underneath.

    // This is valid
    val bool3 = 1 < 2
    
    // This is not valid, because < is an operator
    val bool4 = 1.<(2)
    
    Categories: Development Tags: scalakotlin
  • Starting with Cucumber for end-to-end testing

    Cucumber logo

    This week, my team decided to create a smoke test harness around our web app to avoid the most stupid regressions. I was not in favor of that, because of my prior experience with the fragility of end-to-end testing. But since we don’t have enough testers on our team, that was the only sane thing to do. I stepped forward to develop that suite.

    A simple TestNG MVP

    At first, I wanted to make a simple working test harness, so I chose technologies I was familiar with:

    • Selenium to manage browser interactions.
    • TestNG for the testing framework.

    TestNG is a much better choice that JUnit (even compared to the latest 5th version) for end-to-end testing because it lets you order test methods.

    However, the problem with this approach is readability when failures happen. Stack traces are hardly understandable by team members that are not developers. After having developed something that worked, I thus wanted to add some degree of usability for all team members.

    Migrating to Cucumber

    Cucumber is a BDD tool available on the JVM. It can be integrated with Selenium to have a thin BDD layer on top of GUI testing.

    Project structure

    Cucumber is based on 3 main components :

    • A feature, as its name implies, is a high-level feature of the system. It contains different scenarios, smaller-grained features that realize a feature. Each scenario is made in turn by a combination of step_s: _Given, When and Then that are well-know to BDD practitioners.
      In Cucumber, a feature is written in its own file Gherkin
    • The step definition is a Java file that implements in code the steps described in the feature
    • Last but not least, the test class is a JUnit (or TestNG) test class that binds the 2 former components

    For example, let’s analyze how to create a test harness for an e-commerce shop.

    The following is an excerpt of the feature to handle the the checkout:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    Feature: Checkout
      A customer should be able to browse the shop,
                                   put an item in the cart,
                                   proceed to checkout,
                                   login
                                   and pay by credit card
    
      Scenario: Customer goes to the homepage and chooses a category
        Given the homepage is displayed
        Then the page should display 5 navigation categories
        And the page should display the search box
        When the first displayed category is chosen
        Then the customer is shown the category page
    
      # Other scenarios follow
    
    • On line 1 stands the name of the feature. The name should be short and descriptive.
    • Follows on line 2 a longer text that describes the feature in detail. It’s only meant for documentation.
    • The title of the scenario is set on line 8.
    • Initialization is set on line 9 with the Given keyword
    • On line 10 & 11 are some assertions via the Then keyword. Note that And could be replaced by Then but it feels more readable.
    • On line 12 is an interaction, the When keyword is used
    • And on line 13, we assert the state of the app again

    The corresponding step definition to the previous feature could look like that (in Kotlin):

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    class HomePageStepDef @Autowired constructor(page: HomePage) : En {
    
        init {
            Given("^the homepage is displayed$") { page.displayHomePage() }
            Then("^the page should display (\\d+) navigation categories$") { numberOfCategories: Int ->
                val displayedCategories = page.displayedCategories
                assertThat(displayedCategories).isNotNull().isNotEmpty().hasSize(numberOfCategories)
            }
            And("^the page should display the search box$") {
                val searchBox = page.searchBox
                assertThat(searchBox).isNotNull().isNotEmpty().hasSize(1)
            }
            When("^the first displayed category is chosen$") { page.chooseFirstDisplayedCategory() }
            Then("^the customer is shown the category page$") {
                assertThat(page.url).isNotNull().isNotEmpty().containsPattern("/c/")
            }
        }
    }
    

    Let’s forget for now about line 1 from the above snippet from above, included @Autowired and focus on the rest.

    For each Given/When/Then line in the feature file, there’s a corresponding method with a matching regexp in the class file:

    • Cucumber matches the step defined in the feature with the method by using the first parameter - the regexp.
    • Parameters can be defined in the feature and used in the step. As an example, compare line 10 of the first snippet with line 8 of the second: the regexp will capture the number of categories so it can be easily changed in the feature without additional development.
    • The second method parameter is the lambda that will get executed by Cucumber.
    • Given I’m using a Java runtime v8, those methods are default methods implemented in the En interface. There’s one such interface for each available language, so that step definitions can be implemented in your own language.
    • The class has no direct dependency on the Selenium API, it’s wrapped behind the Page Object pattern (see below).

    Finally, here’s the entry point test class:

    @RunWith(Cucumber::class)
    @CucumberOptions(
            features = arrayOf("classpath:feature/guest_checkout.feature"),
            glue = arrayOf("de.sansibar.glue"),
            format = arrayOf("pretty"))
    class GuestCheckoutIT
    

    As can be seen, it’s empty: it just provides the entry point and binds a feature to the step definitions package. At that point, running the test class in the IDE or through Maven will run the associated Cucumber feature.

    Improving beyond the first draft

    So far, so good. But the existing code deserves to be improved.

    Coping with fragility

    I cheated a little for this one as it was already implemented in the first TestNG MVP but let’s pretend otherwise.

    If you’ve read the step definition class above, you might have noticed that there’s no Selenium dependency anywhere in the code. All of it has been hidden in a class that represents the page:

    class HomePage(driver: WebDriver, private val contextConfigurator: ContextConfigurator): AbstractPage(driver) {
    
        val displayedCategories: List<WebElement> by lazy { driver.findElements(className("navigationBannerHome")) }
        val searchBox: List<WebElement> by lazy { driver.findElements(id("input_SearchBox")) }
    
        fun displayHomePage() {
            val url = contextConfigurator.url
            driver.get(url)
        }
    
        fun chooseFirstDisplayedCategory() {
            displayedCategories[0].click()
        }
    }

    This approach is known as the Page Object pattern.

    Mixing selectors and tests into the same class makes tests brittle, especially in the early stage of the project when the GUI changes a lot. Isolating selectors into a dedicated class let us buffer changes into that class only.

    There are a couple of good practices there - suggested by colleagues and from my personal experience:

    • Use id attributes on elements used for selection. This makes it less likely to break the test by changing the structure of the DOM.
    • Use coarse-grained methods mapped to a business case. For example, instead of having a whole bunch of selectTitle(), fillFirstName(), fillLastName(), submitRegistration(), etc. methods for each registration field, have a single register() method that inputs and submits the data. Again, this isolates possible breaking changes in the page class.

    Improved design

    The Page needs to select components through the Selenium API, thus it needs a reference to a WebDriver. This is a problem when a single feature contains several scenarios as this reference needs to be shared among all scenarios. Possible solutions to this include:

    1. A single scenario per feature. Every scenario will have to define its starting point. For our e-commerce checkout scenario, this defeats the purpose of testing itself.
    2. A single scenario containing steps of all scenarios. In this case, all scenarios will be merged into a very long one. That makes for a hard-to-read scenario and an even harder to read (and maintain) class.
    3. To be able to have multiple scenarios per feature while putting methods into their relevant step definitions, one needs to share the same driver instance among all step definitions. This can be achieved by applying the Singleton pattern to a dedicated class.
    4. The last alternative is to use… DI! Actually, Cucumber integrates quite nicely to integrate with some commons DI frameworks, including Weld and Spring.

    This is great news, as it’s possible to use the libraries we already use in development in the tests. Regular readers know me as a Spring proponent, so I naturally used it as the DI framework. Those are dependencies that are required for that in the POM:

    <!-- Cucumber Spring integration -->
    <dependency>
      <groupId>info.cukes</groupId>
      <artifactId>cucumber-spring</artifactId>
      <version>${cucumber.version}</version>
      <scope>test</scope>
    </dependency>
    <!-- Spring -->
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>${spring.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-beans</artifactId>
      <version>${spring.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
      <version>${spring.version}</version>
      <scope>test</scope>
    </dependency>
    <!-- Spring test - mandatory but not available via transitivity -->
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-test</artifactId>
      <version>${spring.version}</version>
      <scope>test</scope>
    </dependency>
    

    At this point, it’s quite easy to create a standard Spring configuration file to generate the driver, as well as the page objects and their necessaries dependencies:

    @Configuration
    open class AppConfiguration {
    
        @Bean open fun driver() = ChromeDriver()
        @Bean open fun contextConfigurator() = ContextConfigurator(properties)
        @Bean open fun homePage(contextConfigurator: ContextConfigurator) = HomePage(driver(), contextConfigurator)
        // Other page beans
    }
    

    In thie configuration, the driver bean is a singleton managed by Spring and the single instance can be shared among all page beans.

    Those are also singletons part of the Spring bean factory. That’s the reason of the @Autowired annotation in the step definition constructor. But why don’t step definitions get created in the Spring configuration class? Because they are to be created by the Cucumber framework itself using package scanning - yuck. Note that they don’t need to be self-annotated, it’s part of Cucumber’s magic but they still are part of the Spring context and can be injected.

    Screen capture

    A common mistake made when testing is to think nothing will break, ever. I fell into this trap when I was more junior, and now I try to prepare for failure.

    In order to achieve that, I wanted to take a screenshot when a test fails so that it will be easier to fix the failure. Cucumber provides a dedicated lifecycle - before and after code run around each test through respective @Before and @After annotations. Note those are not the same as JUnit’s, and Cucumber doesn’t parse JUnit’s own annotations.

    The easy way would be to create an @After-annotated method in each step definition. Yet, that would just be code duplication. Cucumber also offers hooks, classes which annotated methods are run around each step definition test method. The only constraint for hooks is to place them in the same package as the step definitions so they can be discovered and managed by Cucumber Spring package scanning as for step definitions.

    private val LOGGER = LoggerFactory.getLogger(ScreenCaptureHook::class.java)
    private var DATE_FORMAT = "yyyy-MM-dd-HH-mm-ss"
    
    class ScreenCaptureHook @Autowired constructor(driver: WebDriver) {
    
        private val screenshotFolder = File(File("target", "e2e"), "screenshots")
        private val takesScreenshot = driver as TakesScreenshot
    
        @Before
        fun ensureScreenshotFolderExists() {
            if (!screenshotFolder.exists()) {
                val folderCreated = screenshotFolder.mkdirs()
                if (!folderCreated) {
                    LOGGER.warn("Could not create takesScreenshot folder. Screen capture won't work")
                }
            }
        }
    
        @After
        fun takeScreenshot(scenario: Scenario) {
            if (scenario.isFailed) {
                LOGGER.warn(scenario.status)
                LOGGER.info("Test failed. Taking takesScreenshot")
                val screenshot = takesScreenshot.getScreenshotAs(FILE)
                val format = SimpleDateFormat(DATE_FORMAT)
                val imageFile = File(screenshotFolder, format.format(Date()) + "-" + scenario.id + ".png")
                FileUtils.copyFile(screenshot, imageFile)
            }
        }
    }
    
    • Cucumber unfortunately doesn’t distinguish between a failed test and a skipped one. Thus, screenshots will be taken for the failed test as well as the skipped tests ran after it.
    • There’s no global hook in Cucumber. The ensureScreenshotFolderExists() will thus be ran before each step definition test method. This way requires to manage state, so as to initialize only once.

    Conclusion

    The final result is a working end-to-end testing harness, that non-technical people are able to understand. Even better, using Cucumber over TestNG let us improve the code with Dependency Injection. The setup of Cucumber over Selenium was not trivial, but quite achievable with a little effort.

    Despite all the above good parts, the harness fails… sometimes, for reasons unrelated to code, as all end-to-end tests go. That’s the reason I’m extremely reluctant to set them up in the build process. If that would happen every so often, the team would lose confidence in the harness and all would have been for nothing. So far, we’ll use it manually before promoting the application to an environment.

  • Performance cost of reflection

    In Java, it’s widely admitted that reflection - usage of the java.reflect API, comes at a high cost in terms of performance. Older Java versions had huge performance overhead, while it seems that newer versions bring it in the acceptable range. But what does “acceptable” really mean?

    This is the question I asked when commenting on a performance review that advised to replace code based on reflection by standard code. As many of our decisions are not based on facts but on beliefs, I decided to perform some tests to get metrics in Java 8.

    Testing protocol

    In order to get realistic metrics through an unchallenged protocol, I used the excellent JMH testing framework. JMH has several advantages, among others:

    • An existing Maven artifact is readily available
    • Methods to benchmark have only to be annotated with @Benchmark
    • It handles the warming up of the JVM
    • It also handles the writing the results on the console

    Here’s a JMH snippet:

    @Benchmark
    public void executePerformanceTest() {
    	// Code goes here
    }
    

    JMH will take care of executing the above executePerformanceTest() and taking care of measuring the time taken.

    The code

    To highlight the cost of reflection, let’s check the difference between the time needed to access attributes with reflection and to call simple getters without.

    // With reflection
    Field firstName = clazz.getDeclaredField("firstName");
    Field lastName = clazz.getDeclaredField("lastName");
    Field birthDate = clazz.getDeclaredField("birthDate");
    Field.setAccessible(new AccessibleObject[] { firstName, lastName, birthDate }, true);
    firstName.get(person);
    lastName.get(person);
    birthDate.get(person);
    
    // Without reflection
    person.getFirstName();
    person.getLastName();
    person.getBirthDate();
    

    Checking possible optimizations

    I was wondering if immutable data structures were compiled into optimized bytecode that could decrease the performance overhead of reflection.

    Thus I created the same basic data structure in two-different ways:

    • One mutable with a no-args constructor and setters
    • One immutable with final attributes and constructor initialization

    Results

    Running the tests on my machine yields the following results:

    # Run complete. Total time: 00:26:55
    
    Benchmark                                    Mode  Cnt         Score        Error  Units
    BenchmarkRun.runImmutableWithReflection     thrpt  200   2492673.501 ±  37994.941  ops/s
    BenchmarkRun.runImmutableWithoutReflection  thrpt  200  26499946.587 ± 242499.198  ops/s
    BenchmarkRun.runMutableWithReflection       thrpt  200   2505239.277 ±  27697.028  ops/s
    BenchmarkRun.runMutableWithoutReflection    thrpt  200  26635097.050 ± 150798.911  ops/s
    

    For table-minded readers:

      Non-reflection Reflection
    Mutable 26,635,097.050 ± 150,798.911 2,505,239.277 ± 27,697.028
    Immutable 26,499,946.587 ± 242,499.198 2,492,673.501 ± 37,994.941

    And for bar chart interested readers (note the scale is linear):

    Test results

    Conclusion

    Whatever the way results are displayed, it will take about 10 times more to access a field with reflection than without… on my machine. My guess is that this can be extrapolated to be true on any machine.

    There’s a corollary conclusion: whatever you think holds true at a given time, you should always do some fact-checking to ensure a solid foundation for your decisions.

    If you want to run the tests yourself, the project is available online in Maven format.

    Categories: Java Tags: performancesoftware engineering
  • You don't talk about refactoring club

    The first rule of Fight Club is: You do not talk about Fight Club. The second rule of Fight Club is: You do not talk about Fight Club.

    — Tyler Durden

    I guess the same could be said about refactoring. That would first requires to define what I mean by refactoring in the context of this post:

    Refactoring is any action on the codebase that improves quality.

    Which in turn requires to define what is quality. Everyone I’ve talked with agrees on this: it’s quite hard to do. Let’s settle for now for the following tentative explanation:

    Quality is a feature of the codebase (including and not limited to architecture, design, etc.) which the lack of stalls further meaningful changes in the codebase. At the limits:

    • 100% quality means changing the codebase to develop a new feature would require the smallest possible time;
    • 0% quality means the time to do it would be infinite.

    Given this definition, refactoring includes:

    • Improving the design of classes
    • Adding unit tests
    • Removing useless code
    • Following accepted good practices
    • Anything related to improve readability
    • etc.

    Now back to the subject of this post. Should we ask the customer/manager if a refactoring is necessary? Should we put a refactoring sprint in the backlog? I’ve witnessed first hand many cases where it was asked. As expected, in nearly all cases, the decision was not to perform the refactoring. Taking ages to implement some feature? No design change. Not enough test harness? No tests added. Why? Because the customer/manager has no clue what refactoring and quality means.

    Let’s use a simple analogy: when I take my car to the mechanic, do I get to choose whether he’ll check if the repairs have been correctly executed? Not at all. Checks are part of the overall package I get when I choose a professional mechanic. If choice was made possible, some people who probably opt not to do the checks - to pay less. So far, so good. But then if trouble happened, and probability is in favor of that, the mechanic would be in deep trouble. Because he’s the professional and didn’t do his job well.

    Developers would also get into trouble if they delivered applications with no tests or with a messy codebase; not their customer nor their manager - especially not their managing (depending on their kind of manager if you catch my drift). So I wonder why developers have to let people that don’t know about code taking such important decisions.

    As a professional developer, you and no one else are responsible for the quality of the application you deliver. Your name is in the source code and the commit history, not your manager’s. Stop searching for excuses not to refactor: don’t ask, do it. Refactoring is part of the software development package, period.

    That doesn’t mean that you have to hide the fact that you’re refactoring, only that it’s up to you to decide if the code is good enough or need to be improved.

    Categories: Development Tags: refactoringsoftware engineering
  • Scala vs Kotlin: Operator overloading

    Last week, I started my comparison of Scala and Kotlin with the Pimp my library pattern. In the second part of this serie, I’d like to address operator overloading.

    Overview

    Before to dive into the nitty-gritty details, let’s try first to tell what it’s all about.

    In every language where there are functions (or methods), a limited set of characters is allowed to define the name of said functions. Some languages are more lenient toward allowed characters: naming a function \O/ might be perfectly valid.

    Some others are much more strict about it. It’s interesting to note that Java eschewed the ability to use symbols in function names besides $ - probably in response to previous abuses in older languages. It definitely stands on the less lenient part of the spectrum and the Java compiler won’t compile the previous \O/ function.

    The name operator overloading is thus slightly misleading, even if widespread. IMHO, it’s semantically more correct to talk about operator characters in function names.

    Scala

    Scala stands on the far side of leniency spectrum, and allows characters such as + and £ to be used to name functions, alone or in combinations. Note I couldn’t find any official documentation regarding accepted characters (but some helpful discussion is available here).

    This enables libraries to offer operator-like functions to be part of their API. One example is the foldLeft function belonging to the TraversableOnce type, which is also made available as the /: function.

    This allows great flexibility, especially in defining DSLs. For example, mathematics: functions can be named π, or . On the flip side, this flexibility might be subject to abuse, as \O/, ^_^ or even |-O are perfectly valid function names. Anyone for an emoticon-based API?

    def ∑(i: Int*) = i.sum
    
    val s = ∑(1, 2, 3, 5) // = 11
    

    Kotlin

    Kotlin stands on the middle of the leniency scale, as it’s possible to define only a limited set of operators.

    Each such operator has a corresponding standard function signature. To define a specific operator on a type, the associated function should be implemented and prepended with the operator keyword. For example, the + operator is associated with the plus() method. The following shows how to define this operator for an arbitrary new type and how to use it:

    class Complex(val i: Int, val j: Int) {
        operator fun plus(c: Complex) = Complex(this.i + c.i, this.j + c.j)
    }
    
    val c = Complex(1, 0) + Complex(0, 1) // = Complex(1, 1)
    

    Conclusion

    Scala’s flexibility allows for an almost unlimited set of operator-looking functions. This makes it suited to design DSL with a near one-to-one mapping between domains names and function names. But it also relies on implicitness: every operator has to be known to every member of the team, present and future.

    Kotlin takes a much more secure path, as it allows to define only a limited set of operators. However, those operators are so ubiquitous that even beginning software developer know them and their meaning (and even more so experienced ones).

    Categories: Development Tags: scalakotlin
  • Scala vs Kotlin: Pimp my library

    I’ve been introduced to the world of immutable data structures with the Scala programming language - to write I’ve been introduced to the FP world would sound too presumptuous. Although I wouldn’t recommend its usage in my day-to-day projects, I’m still grateful to it for what I learned: my Java code is now definitely not the same because Scala made me aware of some failings in both the language and my coding practices.

    On the other hand, I became recently much interested in Kotlin, another language that tries to bridge between the Object-Oriented and Functional worlds. In this serie of articles, I’d like to compare some features of Scala and Kotlin and how each achieve it.

    In this article, I’ll be tackling how both offer a way to improve the usage of existing Java libraries.

    Scala

    Let’s start with Scala, as it coined the term Pimp My Library 10 years ago.

    Scala’s approach is based on conversion. Consider a base type lacking the desired behavior. For example, Java’s double primitive type - mapped to Scala’s scala.Double type, is pretty limited.

    The first step is to create a new type with said behavior. Therefore, Scala provides a RichDouble type to add some methods e.g. isWhole().

    The second step is to provide an implicit function that converts from the base type to the improved type. The signature of such a function must follow the following rules:

    • Have a single parameter of the base type
    • Return the improved type
    • Be tagged implicit

    Here’s how the Scala library declares the Double to RichDouble conversion function:

    private[scala] abstract class LowPriorityImplicits {
        ...
        implicit def doubleWrapper(x: Double) = new runtime.RichDouble(x)
        ...
    }
    

    An alternative is to create an implicit class, which among other requirements must have a constructor with a single parameter of base type.

    The final step step is to bring the conversion in scope. For conversion functions, it means importing the function in the class file where the conversion will be used. Note that in this particular case, the conversion function is part of the automatic imports (there’s no need to explicitly declare it).

    At this point, if a function is not defined for a type, the compiler will look for an imported conversion function that transforms this type to a new type that provides this function. In that case, the type will be replaced with the conversion function.

    val x = 45d
    val isWhole = x.isWhole // Double has no isWhole() function
    
    // But there's a conversion function in scope which transforms Double to RichDouble
    // And RichDouble has a isWhole() function
    val isWhole = doubleWrapper(x).isWhole
    

    Kotlin

    One of the main reasons I’m cautious about using Scala is indeed the implicit part: it makes it much harder to reason about the code - just like AOP. Homeopathic usage of AOP is a life saver, widespread usage is counter-productive.

    Kotlin eschews implicitness: instead of conversions, it provides extension methods (and properties).

    Let’s analyze how to add additional behavior to the java.lang.Double type.

    The first step is to provide an extension function: it’s a normal function, but grafted to an existing type. To add the same isWhole() function as above, the syntax is the following:

    fun Double.isWhole() = this == Math.floor(this) && !java.lang.Double.isInfinite(this)
    

    As for Scala, the second step is to bring this function in scope. As of Scala, it’s achieved through an import. If the previous function has been defined in any file of the ch.frankel.blog package:

    import ch.frankel.blog.isWhole
    
    val x = 45.0
    val isWhole = x.isWhole // Double has no isWhole() function
    
    // But there's an extension function in scope for isWhole()
    val isWhole = x == Math.floor(x) && !java.lang.Double.isInfinite(x)
    

    Note that extension methods are resolved statically.

    Extensions do not actually modify classes they extend. By defining an extension, you do not insert new members into a class, but merely make new functions callable with the dot-notation on instances of this class.

    We would like to emphasize that extension functions are dispatched statically, i.e. they are not virtual by receiver type. This means that the extension function being called is determined by the type of the expression on which the function is invoked, not by the type of the result of evaluating that expression at runtime.

    Conclusion

    Obviously, Scala has one more indirection level - the conversion. I let anyone decide whether this is a good or a bad thing. For me, it makes it harder to reason about the code.

    The other gap is the packaging of the additional functions. While in Scala those are all attached to the enriched type and can be imported as a whole, they have to be imported one by one in Kotlin.

    Categories: Development Tags: scalakotlin