Do we really need the DAO?

December 23rd, 2012 4 comments

This may seem like a stupid question, especially after years of carefully creating them. Yet these thoughts about DAO arose in my mind when I watched Adam Bien’s Real World JavaEE rerun on Parley’s.

In his talk, Adam says he doesn’t use DAOs anymore – even though he has one ready so as to please architects (them again). My first reaction was utter rejection: layered architecture is at the root of decoupling and decoupling is a requirement for an evolutive design. Then, I digested the information and thought, why not?

Let’s have a look at the definition of the DAO design pattern from Oracle:

Use a Data Access Object (DAO) to abstract and encapsulate all access to the data source. The DAO manages the connection with the data source to obtain and store data.

Core J2EE Patterns – Data Access Object

Back in the old days, either with plain JDBC or EJB, having DAO was really about decoupling. Nowadays and when you think about that, isn’t it what JPA’s EntityManager is all about? In effect, most JPA DAO just delegate their base methods to the EntityManager.

public void merge(Person person) {
    em.merge(person);
}

So, for basic CRUD operations, the point seems a valid one, doesn’t it? That would be a fool’s conclusion, since the above snippet doesn’t take into account most cases. What if a Person has an Address (or more than one)? In web applications, we don’t have the whole object graph, only the Person to be merged, so we wouldn’t merge the entity but load it by it’s primary key and update fields that were likely changed in the GUI layer. The above snippet should probably look like the following:

public Person merge(Person person) {
    Person original = em.find(Person.class, person.getId());
    original.setFirstName(person.getFirstName());
    original.setLastName(person.getLastName());
    em.flush();
    return original;
}

But what about queries? Picture the following:

public class PersonService {

    public List findByFirstNameAndLastName(String firstName, String lastName) {

        CriteriaBuilder builder = em.getCriteriaBuilder();

        CriteriaQuery select = builder.createQuery(Person.class);

        Root fromPerson = select.from(Person.class);

        Predicate equalsFirstName = builder.equal(fromPerson.get(Person_.firstName), firstName);

        Predicate equalsLastName = builder.equal(fromPerson.get(Person_.lastName), lastName);

        select.where(builder.and(equalsFirstName, equalsLastName));

        return em.createQuery(select).getResultList();
    }
}

The findByFirstNameAndLastName() method clearly doesn’t use any business DSL but plain query DSL. In this case, and whatever the query strategy used (JPA-QL or Criteria), I don’t think it would be wise to use such code directly in the business layer and the DAO still has uses.

Both those examples lead me to think DAO are still needed in the current state of things. Anyway, asking these kind of questions are of great importance since patterns tend to stay even though technology improvements make them obsolete.

Categories: Java Tags:

Short-term benefits in the Information Service business

December 16th, 2012 No comments

It’s a fact that companies and their management have money in mind. However, they’re many ways to achieve this goal. In my short career as an employee, I’ve been faced with two strategies, the short-term one and the long-term one.

Here are some examples I’ve been directly confronted with:

  • A manager sends a developer on an uninteresting mission a long way from home. At first, he tells the developer it won’t last more than 3 months but in effect, it lasts more than one year. Results: the manager meets his objectives and gains his bonus, the developer is demotivated and talks very negatively about his mission (and about the manager and the company as well)
  • A business developer bids for and wins a contract regarding a proprietary and obsolete technology that needs maintenance. Results: the business developer gains a hefty chunk of the contract money, developers who are tasked with the maintenance become increasingly irrelevant regarding mainstream technology
  • Coming to a new company, a developer is proposed a catalog to choose his laptop from. The catalog contains real-life (read “not outdated”) laptops (such as Mac Book Pro) as well as real-life hardware (8Gb RAM). Results: the developer is highly motivated to begin with, the company pays the hardware a little more than lower-grade one
  • A developer informs his boss there’s been an error in his previous payslip: overtime he has made the previous month has been paid again. The boss says it’s ok and that the bonus is now becoming permanently included in his salary. Results: the developer is happily surprised and tells his friends, the boss pays extra salary each month.

What these experience have in common is a trade-off between short-term gain and long-term (or middle-term) gain.

In the first two examples, the higher-up meets his objectives thus ensuring short-term profit. In the first example, this is balanced with trust issues (and negative word-of-mouth); in the second example with a sharp decrease in human resources asset. On the contrary, the two last examples balance marginally higher costs with loyalty and good publicity from the developer.

I understand that stakeholders want short-term profit. What I fail to understand is that in the long run, it decreases company value: when all you have in your IS company is a bunch of demotivated developers looking to jump out of the train, you can have all the contracts in the world, you won’t be able to deliver, period. Of course you could invest massively in recruiting, but what’s the added value?

Kepping your developers happy and motivated has a couple long-term consequences that have definite value:

  • Developers will speak highly of the company, thus making the company more attractive to fellow developers
  • Developers will speak highly of the company, thus building trust between the company’s customers and the company itself
  • Developers will be more motivated, thus more productive by themselves
  • Developers will be more motivated, hence more collaborative (and so even more productive)
  • etc.

The problem lies in that those benefits are hardly quantifiable and translatable into cash equivalent by any means. Average companies will likely ignore them while smart ones will build upon them. Are you working for the first kind or the second?

Note 1: this article reflects my experience in France and Switzerland. Maybe it’s different in other European countries and North America?
Note 2: in Paris (France), there’s a recent trend of companies caring more about long-term profits. I can only hope it will get stronger

Categories: Miscellaneous Tags:

The cost and value of knowledge

December 2nd, 2012 No comments

I was a student in France when I discovered the Internet and the World Wide Web. At that time, I didn’t realize the value of the thing, even though I taught myself HTML from what was available at the time (even if I don’t remember the sites – they probably have disappeared into Limbo).

When I began my professional life,  a project director of mine joked about the difference between the developer and the expert: “The expert knows about the cabinet where the documentation is”. Some time after that, I found this completely untrue, as one of my senior developer constantly found the answers to my questions: he used Google through and through and it helped me tremendously. The cabinet had become the entire Internet and everything could be found provided time and toil. I soon became a master of the Google query.

At this time, when I had a problem, I asked Google, found the right answer and reproduced it on my project. This was all good but it didn’t teach you much – script kiddies can do the same. At the same time, however, I found an excellent site for French Java developers: “Développons en Java” by Jean-Michel Doudoux. This site was the first I knew of where you could learn something step by step, not only reproduce a recipe. I eagerly devoured the whole thing, going as far as copying it on my computer to be able to read it when offline (a very common occurence then). This was made possible by M. Doudoux passion and will to make his knowledge available!

Even better, a recent trend, from which Coursera has been the best marketer, cooperates with well-established universities to make standard in-the-room courses available… for free. For the first time in history, knowledge has become a costless commodity and it’s something that can be compaired to Gutenberg’s press during the Renaissance. Where previously books had to be written by hand thus preventing diffusion of the understanding they contained, printed press made books at relatively low costs available everywhere.

The first natural reaction is the “Wow!” effect. I personally am very interested in foreign languages, Japan and pyschology: I could learn so much about any (or all) of these subjects without even leaving my home. Moreover, so many people are kept away from education, not by a lack of intellectual prowess, but just because they haven’t been born rich enough or in a part of Earth where there’s no university. Imagine if all people could learn about humanities, democracy, engineering, whatever. This could well provoke a new Age of Enlightnemet, but this time not only in Europe but encompassing the whole globe.

However, there’s a negative side to this coin. When something is made free, people tend to give it no value and waste is sure to follow. There are plenty of examples everywhere, the worst I know of is the free water policy for Qatari households in Qatar. Thus, we have to be very careful about that – knowledge has been acquired with much effort, period. If it’s made freely available, there’s the risk that people treat it as a common commodity.

Worse, both public and private sectors invest in research and knowledge. If it isn’t sold for a profit (even marginal), what are the odds both (at least the latter) would continue their investments. The business model for such services is still to be found. A hint would be recruiting: if the course is one of the best, recruiters would perhpas be interested in getting their paws on contact data of those who passed with flying colors?

My conclusion to these points is that providing people with free knowledge is something very commendable, but it shouldn’t lower its value, only its cost.

Categories: Miscellaneous Tags:

Drupal 7 to Jekyll, an epic journey

November 25th, 2012 1 comment

There’s a recent trend in blogging that consists of dropping PHP platforms such as WordPress and Drupal in factor of static HTML pages generated when needed.

Static blogs have many advantages over PHP engines:

  • They’re much faster since there’s no computation overhead at runtime for composing the rendered page
  • There’s no security issues regarding SQL injection
  • Finally, generating the site removes the need for a staging area with possibly mismatched configurations

morevaadin.com is one of my site, designed with the Open Publish distribution on top of Drupal 7. It’s designed only to display articles about the Vaadin framework: there’s no comment management, no interaction, no fancy stuff involved whatsoever. In essence, there’s no need for a Drupal platform and a bunch of HTML files with some JavaScript would be more than enough. Here’s about my journey into porting the existing site, with the different steps I went through to reach my goal (spoiler: at the time of this writing, it’s not a success).

First step

Having no prior experience, I choosed Jekyll as my target platform for it’s supported by Github.

Before doing anything, I went through the motion of creating a Ubuntu VM, with the Ruby and RubyGems packages.

sudo apt-get install ruby
sudo apt-get install rubygems

Blog migration explained

My first strategy was to follow instructions from this blog post, which seemed to exactly match my target. In essence, I was to get the SQL dump from my Drupal site, execute it on a local MySQL base and execute some Ruby script wrapping a SQL one.

The problem is that the propsed SQL script was not adapted to the structure of my base: I fixed the prefixes and the node types, but I had still no success running the script.

Other scripts

I found other scripts on the Internet, but none was adapted to the structure of my database and I couldn’t get any running despite my best efforts.

Eureka!

Back to the basics: given that no hand-crafted script was able to run on my database, I decided to look further into Jekyll documentation and found there was a gem associated with a Drupal 6.1 migrator.

So, I installed Jekyll – a Ruby gem – and I could finally run the script.

sudo gem install jekyll
ruby -rubygems -e 'require "jekyll/migrators/drupal"; Jekyll::Drupal.process($MYSQL_INSTANCE, $LOGIN, $PASSWORD)

At this point, I got no errors running the Ruby script but no output whatsoever. However, I tweaked the /var/lib/gems/1.8/gems/jekyll-0.11.2/lib/jekyll/migrators/drupal.rb script a bit, where I commented out the lines concerning the node and entity types (30-31), as well as the whole tag things (lines 53-60 and 69) and I finally obtained some results: a _drafts and a _posts folder, with the latter full of .md files!

Running Jekyll with jekyll --server finally generated a _site folder, with a load of pages.

Problems

There are some problems with what I got:

  • Some pages have unparseable sections (REXML could not parse this HTML/XML)
  • Pages have no extensions, despite being HTML, so the browser doesn’t recognize the MIME type when they’re run under Jekyll. The normal way would be to have index.html under folders named as the post
  • There’s no index

Cloning Octopress and Jekyll

Given my very limited knowledge (read none) of what happens under the cover and my unwillingness to further change the script, I tried the same on a ready-to-use template named Octopress, with the possible intention to migrate the few pages I had (not very satisfying from a software developer point-of-view but pragmatic enough for my needs).

sudo apt-get install git
git clone https://github.com/imathis/octopress

Unfortunately, there was an error in the script :-( So, in a desperate attempt, I wanted to give a try to Jekyll Bootstrap.

git clone https://github.com/plusjade/jekyll-bootstrap.git

And lo, I finally got no error, an output and something to view in the browser at http:/0.0.0.0/index.html!

Present situation

I’ve been tweaking and updating included files (those in the _included folder) and I’m beginning to get something bearing some similarity to the original layout.

To obtain that, I had to install Pygments, the code prettifier:

sudo apt-get install python-pygments

At this point, there are basically 2 options: either manually port other posts and finalize the layout (as well as work toward the original theme), or invest in the configuration process to make it work.

At present, I’m leaning toward the first option. Feedback welcome!

Categories: Technical Tags: ,

Devoxx 2012 – Final day

November 17th, 2012 No comments

– Devoxx last day, I only slept 2 hours during the previous night. Need I really say more? –

Clustering your applications with Hazelcast by Talip Ozturk

Hazelcast is an OpenSource product used by many companies (even Apple!).

HashMap is a no-thread safe key-value implementation. If you need thread safety, you’ll use ConcurrencyHashMap. When you need to distribute your map against distributed JVMs, you use Hazel.getMap() but the rest is the same as for the Map interface (but not interface itself).

– a demo is presented during the entire session –
Hazelcast let you add nodes very easily, basically just by starting another node (and it takes care of broadcasting).

Hazelcast alternatives include Terracotta, Infinispan and a bunch of others. However, it has some unique features: for example, it’s lightweight (1,7 Mb) without any dependency. The goal is to make distributed computing very easy.

In a Hazelcast cluster, there’s a master, but every node knows the topology of the cluster so that when the master eventually dies, its responibilities can be reassigned. Data is backed up on other nodes. Typical Hazelcast topology include data nodes and lite members, which do not carry data.

Enterprise edition of Hazelcast is the Community Edition plus Elastic Memory configuration and JAAS Security. It’s easy to use Hazelcast in tests with the Hazelcast.newHazelcastInstance(config) statement. Besides, an API is available to query about the topology. Locking is either done globally on the cluster or on a single key. Finally, messaging is also supported, though not through the JMS API (note that messages aren’t persistent). A whole event-based API is available to listen to events related to Hazelcast.

A limitation of Hazelcast is that objects put into it have to serializable. Moreover, if an object is taken from Hazelcast and then modified, you have to put it back into Hazelcast so that it’s also updated inside the cluster.

Hazelcast doesn’t persist anything, but ease the task of doing it on your own through the MapStore and MapLoader. There are two ways to store: write-behind which is asynchronous and write-through which is synchronous. Reads are read-through which is basically lazy-loading.

About long GC-pauses:

Killing a node is good, but letting it hang around like a zombie is not

There’s a plugin to support Hibernate second-level cache and it can also be tightly integrated with Spring.

mgwt – GWT goes mobile by Daniel Kurka

The speaker’s experience has taught him that mobile apps are fated to die. In fact, he has used and developed many mobile applications. The problem, is that you’re either limited by a too small number of applications, or the boundary goes higher and you cannot find them anymore. In the past, Yahoo put web pages into a catalog, while Google only crawled and ranked them. App stores looks like crap, they don’t provide a way to search for applications you’re itnerested in. Besides, when you’re looking for public transports schedule, you have to install the application of the specific company. Worse, some sites force you to install the application instead of providing the needed data.

When developing mobile applications, you have to do it for a specific platform. As Java developers, we’re used to the “Develop Once, Run Anywhere”. Also, there’s already such an universal platform, it’s the browser. PhoneGap tries to resolve the problem and provides HTML and JavaScript as the sole language. Now, you get “Build Once, Wrap with PhoneGap Around, Run Anywhere”.
GWT is a framework where Java is compiled into JavaScript instead of bytecode. Better, it’s compiled into optimized JS, and that’s important because mobile access may dry your battery dry. Finally, there’s GWT PhoneGap integration so you can write awesome webapps, wrap them into PhoneGap and release them on a store. PhoneGap works with your app composed of HTML, JS and CSS and plugins that are native and let you access mobile device features. Two important things to remember: data passed between app and device is done as strings, and calls are asynchronous. PhoneGap uses W3C standards (when possible); it’s an intermediate that will have to die when the mobile web is finally there.

Yet, PhoneGap doesn’t solve the core “too many apps” problem. GWT’s compiler compiles Java into JavaScript (a single file per browser). Remember that GWT’s code is optimized that would be hard to achieve manually: that’s very important when running on mobiles to due limited battery issue. mgwt is about writing GWT applications, aimed specifically for mobiles. In order to create great apps, be wary of performance. There are 3 areas of performance:

  • Startup performance, a matter of downloading, parsing, executing and rendering
  • Runtime performance which is really impacted by layout. As a rule, never leave native code. So, if you use JavaScript layout, you’re doing just that. CSS3 provides all the means necessary to layout. Likewise, animations built in JavaScript are bad and prefer CSS instead.

Both are taken into account by the GWT compiler. Note that mgwt provides plenty of themes, one for each mobile platform.

– And now ends Devoxx 2012 edition for me. It was enriching as well as tiring. You can find the recap of the previous days here:

Categories: Event Tags:

Devoxx 2012 – Day 4

November 15th, 2012 No comments

Unitils: full stack testing solution for enterprise applications by Thomas de Rycke and Jeroen Horemans

There are different types of test: unit tests (testing in isolation), integration tests (testing a subsystem) and system tests (testing the whole system). The higher you get in the hierarchy, the longer it will to execute, so you should have a lot of UT, some IT and just a few system tests.

Unit tests have to be super fast to execute, be easy to read and refactor, they also should tell you what went wrong without debugging. On the other hand, you’ve to write a lot of tests. Not writing unit tests will get you faster in the first phase, but when the application grows, it will drastically slows your productivity down. In enterprise settings, you’re better off with UT.

In order to avoid an explosion of testing frameworks, Unitils testing framework was chosen. Either JUnit or TestNG can be hooked into Unitils. The core itself is small (so as to be fast), the architecture being based on modules.

Let’s take for example a Greetings Card Sending application. – the demoed code is nicely layered, with service and dao and uses Spring autowiring and transactions through annotations – Unitils is available from Maven, so adding it to your classpath is just a matter of adding the right dep to the POM. Unitils provide some annotations:

  • @TestedObjectto set on the target object
  • @Mockto inject the different collaborators (using EasyMock)
  • @InjectByType to hint at the injection strategy for Unitils to use

Even though testing POJOs (as well as DTOs) is not very cost-effective, a single test class can be created to check for JavaBean conventions.

Unit testing becomes trivial if you learn how to write testable code: designing good OO and dependency injection, seams in your code and avoid at all the use of global state (time for example). What about integration and systems? They tend to get complex and you’ll need a “magic” abstract superclass. Moreover, when an IT fails, you don’t know if the code is buggy or if the IT itself is. Besides, you’ll probably need to learn of frameworks (JSF comes with its own testing framework, JSFUnit).

As software developers, the same problems come back:

  • Database: is my schema up-to-date? How to manage my referential integritry to set up and tear down data?
  • Those operations are probably slow
  • Finally, how to deal with external problems?

Unitils provides a solution for each of these problems, through the use of a dedicated module for each. For example, a Webdriver module lets you use Selenium through Unitils, a DBUnit module does the same for DBunit and a Mail module creates a fake SMTP server.
– speakers tells us that with Unitils, there’s no need for a magic abstract superclass anymore: IMHO, it has been replace with a magic black box testing framework –
Unitils benefits from a whole set of modules, which covers a single dedicated area: Swing, Batch, File, Spring, Joda Time,etc.

Modern software development anti-patterns by Martijn Verburg and Ben Evans

There are 10 anti-patterns to cover:

Anti-pattern Diabolical programmers Voice of reason What to do
Conference-Driven Delivery Real pros hack code and write their slides minutes before their talks PPPPPP In order to improve your presentation skills, you can rehearse in front of the mirror. Let’s call it Test-Driven presentations. Begin with a supportive crew, then grow into larger and larger audience.
Mortgage-Driven Development In order for others not to steal your job, don’t do any documentation. Even better, control your source code i.e keep the source on a USB key Don’t succumb to fear: proper communication is key. Developers who communicate have the most success. If you want one of your idea to go into source control, you’ll have to communicate about it.
Distracted by shiny Always use the latest tech, it’ll put you ahead. Consider using the latest version of Eclipse, packed full of plugins Prototype and evaluate: learn to separate the myth from the reality. As an example, web frameworks are an area you should tread carefully. Code reviews are an asset, especially if everyone do them so as to share best practices.
Brown bag sessions are good, in order to test every possible options.
Design-Driven Design UML Code generators are awesome. Print your UML diagrams on gigantic sheets, put them on the wall, and if someone asks you a question, reply that it’s obvious Design for what you need to know: don’t try to do too much upfront. Successful teams are able to navigate between methodologies that suit them. Less source code: pay your junior developers to produce code, and pay your senior to remove it. The less code, the less chances to have bugs. Be wary of maintainability, though (Clojure is a good example).
Pokemon pattern Use allof the GoF patterns The appropriate design pattern is your friend. Using a design pattern is adding a feature to a language which is missing it. As an example, Java concurrency works, but since mutability is deeply rooted, it’s hard. Of course, you could use actors (like in Scala), but you definitely would add a feature. Whiteboard, whiteboard and whiteboard, to communicate with your colleagues. As a tip, carry whiteboard markers at all time (or alternatively pen and pencil).
Tuning by Folklore Performance tune by lightning black candles. Remove string concatenations, add DB connections and so on Measure, don’t guess. It’s the best approach to improve performance. If it sounds boring, it’s because it is. Remember that the problem is probably not where you think it is. Shameless plug: go to jClarity
Deity All the code in one file is easier to search. VB 6 is one of the most popular language because of it Discrete components based on SOLID principles: Single Responsibility, Open to extension, Litskov substitution, etc. Discuss with your team, don’t go screaming WTF, you’ll probably have issues. You need to use naming based on the domain model so as other developers can read your code
Lean Startup Ninja A ninja ships its code when it compiles!Continuous delivery is a business enabler. Tools are available to achieve continuosu delivery: Ant/Gradle/Maven + Jenkins + Puppet + Vagrant
CV++ The Tiobe index is your friend. Just be good at the principles. Try different languages, because when learning another language can be applied to the language you’re currently programming in. Being a Software Developer is better than a Programmer. The former understands the whole lifecycle of an application, which is much better.
I haz Cloud Nothing can go wrong if you push your applications in the cloud Evaluate and prototype There are many providers available: EC2, Heroku, JElastic, CloudBees, OpenShift, etc. Again, a brown bag session is a great way to start
Can Haz Mobile If you go into the mobile game, you’ll be a millionaire
HTML5 for the win UX is so important. What are your UX needs? Think about what pisses you off…
Big Data MongoDB is web scale What type of data are you storing? Non-functional matters. Business doesn’t care about distributed databases, but about their reports!

Simplicity in Scala Design by Bill Venners

The main thing is to show how to:

  • Design to simplify tasks for your users
  • Design for busy teams. Don’t assume your users are expert in your libraries. It’s something like driving a car you’ve never driven before: it should be so obvious users shouldn’t have to check the documentation
  • Make it obvious, guessable or easy to remember, by order of preference. The choice process for the plusOrMinusin ScalaTest is a good example.
  • Design for readers, then writers, in that order. To choose the name for the invokePrivateis a good example: better use something semantically significant than some symbol
  • Make errors impossible (or difficult at last). In particular, use Scala’s rich type system to achieve this. Using the compiler to flag errors is the best
  • Exploit familiarity. James Gosling used familiar C++ keywords in Java, but without memory management so as to leverage this principle. So is the way ScalaTest is using JUnit constructs but enforcing a label on each test.
  • Document with examples
  • Minimize redudancy. A Zen of Python states:

    “There should be one and preferably only one obvious way to do it.”

    . A good practice would be to put a Recommended Usage blocks for each class.

  • Maximize consistency. ScalaTest offers a GivenWhenThen trait as well as the Suite class. But when you mixin the latter with the former, you do not need to set the name of the test. However, enforcing the Single Responsibility Principle and introducing Specsolves the problem.
  • USe symbols when your users are already experts in them. Scala provides + instead of plus, and * instead of multiply. On the contrary, in SBT some symbols prove to be problematic: ~=, <<=, <+=, <<++=. Likewise, it’s better to use foldLeft() than /:. However, there are situations where you shouldn’t be able to use words at all (like in algebra).
  • Use a functional style by default, but switch to an imperative style where it improves usability. For example, test functions in a suite, it registers the tests typed inside the braces.

Remember what tools are for. They are for solving problems, not finding problems to solve

Re-imagining the browser with AngularJS by Igor Minar and Misko Hevery

– I attend this session in order to open my mind about HATEOAS applications. I so do hope it’s worth it! –
Browsers were really simple at one time. 16 years later, browsers offer many more features, going as far as allowing people to develop through a browser! For a user, this is a good, for a developer not so much because of complexity.

In a static page, you tell the browser what you want. In JavaScript, you have to take care of how browsers work and the difference between them (hello IE). As an example, remember rounded corner before CSS border-radius? In HTML or the DOM API, there doesn’t seem to be such improvement. Even if you use jQuery, you’re doing imperative programming. The solution to look for is declarative! Databinding helps with this situation. If your model changes, the view should be automatically updated. On the contrary, with two-ways data binding updates in the view should change the underlying model. Advanced templating can be used with collections. Actually, 90% of JavaScript is only to synchronize between the model and the view.

AngularJS takes care of this now, while a specification is under work for this should be part of how browser works. HTML can be verbose, tabs are a good example: they currently use div while they should be using tab. From a more general point-of-view, AngularJS brings such components to the table. Also, the Web Components is an attempt at standardization of this feature.
– here comes a demo –
AngularJS works by importing the needed js files, creating a module in JS and adding custom attribute to HTML tags.

Testacular is a JavaScript testing framework adapted to AngularJS.

Apache TomEE, JavaEE 6 Web Profile on Tomcat by David Blevins

Apache TomEE is trying to solve the problem in that you run with Tomcat unlike you hit the wall and go to another server. TomEE is a certified Web Profile stack, it is a Tomcat with the Apache libraries to bridge the gap. Core values include being small, being certified and stay Tomcat.

What is the Web Profile anyway? In order to solve the bloated image of JavaEE, v6 specs introduced kepping only about half the specs of the full JavaEE specs (12 on 24). Things that are not included:

  • CMP
  • CORBA
  • JAX-RPC
  • JAX-RS
  • JAX-WS
  • JMS
  • Connectors

The last 4 are especially lacking so TomEE comes into 3 flavors: Web Profile including JavaMail (certified), JAX-RS (certified) and Plus which includes JAX-WS, Connector and JMS (not certified). Note that in JavaEE 7 will certainly change those distributions, since profiles themselves will change. TomEE began in 2011 with 1.0.0.beta 1 and October saw the release of 1.5.0. TomEE is know for its performance but also its lack of extended support.
– here comes the TomEE demo, including its integration with Eclipse (which is exactly the same as for Tomcat bare –
TomcatEE is just Tomcat with a couple of JAR in both lib and endorsed, extra-configuration in conf and a launcher for each OS in bin. Also, there’s Java agent (– for some reason I couldn’t catch –).

Testing is done with Arquillian and runs within an hour. Certifications tests are executed on Amazon EC2 instances, each having 613 max memory (and runs in +hundred hours): note that TCK pass with default JVM memory. The result is quite interesting with a 27Mb download, with only 64Mb memory taken at runtime. It would be a bad idea to integrate all stacks yourself, not only because it’s just a waste of time, but also because there’s a gap between aggregating all components individually and the integration that was done in TomEE.

Important features of TomEE include integration of Arquillian and Maven. A key point of TomEE is that deployment errors are printed out all at the same time and do not force you to correct those errors one by one as it’s the case with other application servers.

Categories: Event Tags:

Devoxx 2012 – Day 3

November 14th, 2012 No comments

– the last evening was a little hard on me, shall we say. I begin the day at noon, which suits me just fine –

Securing the client-side by Mike West

If you want the largest possible audience, chances are you’ll use HTML, JavaScript and CSS. Native hasn’t (yet) the same traction as those traditional technologies. Business logic is slowly moving from those big backends toward browsers.

A prerequisite to anything is to send data securely, using HTTPS. There’s no reason today not to use HTTPS. In fact, getting to a server, you’ll probably hop from server to server until you get to your target. Now, if someone gets in the middle, you’re basically toast. Using HTTP, there’s no way to detect the data. On the contrary, using HTTPS, you can prevent reading and writing data until it gets to the server (and back to you).

However, servers should still listen to HTTP – because most users type HTTP, but then redirect to HTTPS. There a HTTP header Strict-Transport-Security. If you want to develop an application, got to https://startssl.com/ and get SSL certificates.

The most common security attack is script injection. Even you can write begign scripts, most scripts will have worse effects such as accessing cookies. Browsers normally disallow accessing cookies from other domains, but XSS exactly bypass that. The biggest problem is that users are not aware of this. A XSS cheatsheet is available at OWASP but in essence, in order to defend against XSS, you have to both verify each input and escape output.

Fortunately, both problems are resolved since ages. Developers just have to know about solutions considering the languages and frameworks they’re using. Unfortunately, developers may make mistakes, and browsers may contain bugs. Therefore:

Every program an every priviled user of the systle shoule operate using the least amount of privilege necessary to complete the job

HTML has a thing called Content Security Policy, which is a W3C specification (currently under draft). It takes the form of a HTTP header and specify valid external resources. Resources not specified won’t be loaded by the browser, as the latter would assume they were injected by scripts. Currently, CSP 1.0 is implemented in Chrome and Firefox. The main security hole in XSS are inline scripts: in order to prevent them, CSP disallow inline JavaScript, including declarations and event handlers. Note this could be seen as a downside, but both can be declared outside HTML, and remember it’s a security hole.

Alternatively, CSP also offers a Content-Security-Policy-Report-Only header, so as to send reporting to a URL. This lets you have a loose CSP policy and a stricter CSP policy report so you can test the scricter one before deploying it. HTML Rocks tutorial is a a good source of info to setup security policy.

Another way to protect against XSS is to use sandboxing. sandbox is an attribute to apply to iframe. It works by putting this iframe into a unique domain origin. This has some major consequences: no plugins, no scripts, no form submissions, not top-level navigation, no popups, no autoplay, no pointer lock and no seamless iframe. sandbox allows values to loosen some of these restrictions.

In concordance with the above ‘Least privilege policy’, a possible architecture is to design a page where each component is used to do a single thing. For example, we could use a sandbox to execute potentially dangerous scripts and wraps it under a simple message API to get its results from the main page.

Home automation for Geeks by Kai Kreuzer and Thomas Eichstädt-Engelen

– given my initial education as an architect, I’m due bridging the gap between it and my current job –
The talk if about fun stuff you can do with Java.

Home automation is not just electrifying things, because they’re already! Point is, none of these systems are integrated in anyway: home automation have to put emphasize on integration. There are 3 goals of home automation:

  • Comfort
  • Security (motion sensors, window contacts, etc.)
  • Energy saving (cut power off, etc.)

To create a smart home, you’ll need smart devices, to manage your home or to visualize data. Many of these features are already avaialble in the openHAB project. The project consists of two parts, the openHAB runtime, which is a headless Java running on the JVM and a designer which is used to configure the former. Both communicate either through a Samba share folde or Dropbox.

The runtime is composed of:

  • Java 7
  • OSGi (Equinox variety)
  • Declarative services
  • EMF
  • and an Embedded Jetty, with JAX-RS and Atmosphere

Core concepts around opneHAB include items: they are high-level abstractions that basically represent things you want to control. Note that devices aren’t the right level of abstraction. As an example, a radio device could be composed of a switch item, a dimmer item and a number item.

opneHAB’s architecture is a simple Event Bus, with bindings between commands (to the item) and status update (from the item) events and the real-life device. An item registry is plugged into the bus, so as to have both an UI and some automation rules, as well as a console and a persistence layer. There are a couple of UIs available: web of course, native for Windows Mobile, Android and iOS.

openHAB offers many different bindings over devices, but also integrates with XMPP, OSGi and Google Calendar. The latter is very important since you can create your triggers through Google Calendar. Other features include database services and export capabilities (Sense, Cosm).
– a real-life demon starts, taking a Koubachi plant device and a doorbell as examples –
A demo server is available at demo.openHAB.org

Behavior-Driven Development on the JVM, a state of the union by John Ferguson Smart

– if you’ve read Specifications by Example, this one is a must –
BDD has many defnite benefits such as less bugs and more productivity but was is BDD anyway? In essence, BDD is making Business, Business Analysts, Developers and Testers talk a common language. There are no tests in BDD, there are executable specifications. In TDD, you test what your code does, while in BDD, you test what your code should do in accordance to the specifications.

Effectively, executable specifications are not only specifications, but also tests and documentation. Givent that, every line you’re writing should be linked to a line in the speficications: this means you do not code at the micro-level, but at the macro-level. As a consequence, every line you code has a value to the business. In summary, using BDD will:

  • build only features that add real value
  • result in less wasted effort (through traceability)
  • improve communication
  • get a higher quality, better tested product

How to apply BDD to Business Analyst

A typical application will probably have some goals, each being decomposed into capabilities, features, stories down to examples, each one being translated into acceptance criteria. The main point of this is to be able to trace each criterion up.

Goal
The question to ask the customer what he really needs and keep asking why until you attain a high-level goal. Standard users tend to express requirements as implementations: we need to find the business need behind the suggested naive implementation.
Capability/feature
Those help reach the goals defined above. Which features do you do first? We should implement the minimum features required to deliver the business value defined in the goal. Traditionally, using the triplet “IN ORDER”, “AS A” and “I WANT”.
Story
The sotry takes the role of the user and describes, guess what, a user story as Agile defines it.

Tools to code

Available tools include JBehave, Cucumber, easyb and Thucydides, which does the job of reporting from the other tools.

Remember that TDD tools include JUnit, Mockito, Spock and Arquillian. While BDD is about acceptance tests, TDD is about developer tests (unit- and integration-).– I tend to think that high-level integration tests begin to approach user-acceptance tests –

The speaker uses Spock as a TDD testing framework, because it respects the Given/When/Then BDD structure. As a bonus, you can mock (and stub) needed dependencies very easily as well as code data-driven tests in a tabular way. Finally, Spock can be integrated with Arquillian. Alternatively, if you’re a Scala fan, BDD2 is the framework to use. Finally, for JavaScript, Jasmine is available.

In conclusion, it’s behavior all the way down.

Using Spring in Scala by Arjen Poutsma

The goal of the session is to see how to use Spring in the Scala language.
– Since the session is very based on demo, I encourage the reader to go directly to the Spring Scala project –

Effective Scala by Josh Suereth

  1. Basics
    • Use expressions instead of statements. Telling the computer what to do is a statement: create a variable, set its value and return the variable. In Scala, the last computed value is the returned value. Note that match, if and others return values. Likewise, Scala forexpressions let you just tell what you want, not how you want it!
    • Use the REPL (the Scala command-line evaluator). The good thing about REPL is that you’re not only shown the type of the variables you create, there’s auto-completion, so it’s a great way to find what exists. It’s a good practice to start with the REPL, then test, then code.
    • Just stay immutable. In Java, keeping references on mutable collections is a very dangerous thing to do, so you’ll have to copy them before storing/exposing them. In this case, you’ll endure a very big decrease in performance. In Scala, caseclasses with constructor “injection” make it simpler to implement immutability. Additionally, you’ll get automatic equality and hashcode for free.However, remember that local immutability is ok, that is it’s ok as long as you don’t expose your mutable variable.
    • Use Option. In Java, you’ll have to check whether every used parameter is null and you’d better not forget one! In Scala, for expressions can be used to easily perform checks and yield the desired Option result.
  2. Object-orientation
    • Use def for abstract members. Given the way memory allocation works on the JVM, we have to do either declare a block or use a def. Since the former is an advanced technique, use the latter. Addtionally, you can use the same def names in case classes.
    • Annotate API on non trivial return type. Since the compiler does type inference, you may run into complex cases where the type you infer is not the type the compiler does. Do document!
    • Favor composition over inheritance. – See the famed Cake pattern
  3. Implicits
    • Implicits are used when you need to provide a default parameter in a function but let the caller decide which default he wants to use (as opposed to default value in parameters).
    • Implicits are found when “in scope”. Scope computation follow a strict (– but complex –)search pattern. A good thing to do is to avoid imports, since it’s not explicit that importing will trigger the implicit. Thus, it’s best to put implicit in companion objects.
    • Implicit views. Don’t use implicit view, whatever they are, in favor in implicit classes.
  4. Types
    • Types traits. A best practice is to provide typed traits.
    • Type class. There’re several benefits to using type class, most importantly to monkey patch classes you don’t control.
  5. Misc
    • Scala isn’t Java (nor Haskell). Learn to write Scala, don’t write it as the language(s) you know.
    • Know your collections! Traversables are at the root of the collections hierarchy. Iterables comes next, etc. Indexed sequences are good to use when you need to access an element by their index, linear sequences when you need to access by head/tail. So, if coming from Java and wanting index access, use Vector instead of List.
    • Java integration. Prefer Java primitives in API
    • Learn patterns from FP.

Vaadin meetup with Joonas Lehtinen and the Vaadin team

Escalante, Scala-based Application Server

Escalante is an attempt of JBoss to create a Scala Application Server. Scaladin is a Scala wrapper around the Vaadin API, provided as a add-on. Lift, the Scala web framework, has very fat dependencies: Escalate prevents you from having to deploy the Lift JAR in your Lift-dependent webapps.

mGWT

GWT goes mobile, with mGWT and GWT Phonegap. mGWT supports major mobile browsers, including iOS, Android etc. The only major difference, is that you get it on the web instead of an app store. Web performance is an importante topic: compiling makes it easier, only you have to think about some things. For example, startup performance is composed of downloading, evaluating and executing. – see the talk about web performance –

JavaScript optimizations is hard to do by hand, and thus most websites don’t do them. On the contrary, the GWT compiler enforces those good practices (e.g. caching). Moreover, GWT let you do things really not possible otherwise, as componentizing parts of your application. Also, GWT can generate the offline manifest of HTML5 app cache. By comparison, GWT applications are much smaller, because of choices made by Google engineers: for example, there’s no image in the iOS theme, only pure CSS. Another axis of optimization is the minimal output (as for CSS). Finally, only the stuff actually used is in your final application: it’s called Dead Code Removal. In essence, only Android theme is downloaded by an application running on an Android device.

In the area of runtime performance, lets consider layout. Doing layout in JavaScript requires time! A rule is never to leave native code, in our case the CSS rendering engine. Well, mGWT uses CSS3 to layout the page. if we consider animations, looping in JavaScript is also a pretty bad idea. The solution is the same, just use CSS, which will use GPU acceleration.

SASS theming in Vaadin 7

SASS is a CSS compiler so think of it as a “GWT for CSS”. CSS should be easy, but it’s not for real-life applications and general-purpose themes. You want to do it the right way, because it’s gonna be there for a long-time. CSS support in IDE (Eclipse for example) is crap: there’s only auto-complication. Point is, CSS is hard to maintain with a team of more than one person (and even for a single person, it can be tedious – to say the least). On the other hand, a maintainable code structure – imports, kill startup performance because it will require an HTTP request per CSS. SASS can help in these areas.

For example, you can split your CSS file into smaller ones but SASS will concatenate them into a single one. Morevoer, URL paths for images in the CSS can be resolved automatically. – the rest of the presentation presents SASS features; you can have a look at an introduction there or check the full SASS documentation
The Vaadin team has developed its own SASS compiler in Java, since the original implentation is Ruby-based and we didn’t want any Ruby dependency in our apps, did we?

Besides SASS support, Vaadin 7 also offers a way to completely remove the default style name of a component through the setPrimaryStyleName() instead of being only able to add additional CSS class names. In combination to SASS mixins, this let you combine parts of different themes.

Important: SASS support is under active development, test and do provide feedback!

Usability in Vaadin

– Faros, a Vaadin solution partner created the Vaadin Belgian meetup group –
Usability is a huge topic, point is a good interface should be self-explanatory. Usability is defined as how it’s easy to accomplish a give task while accessibility is ensuring an equivalent user experience for people with disabilities (blindness, in most cases). Default Vaadin theming doesn’t render buttons as HTML buttons but as div, screenreaders cannot recognize them as such. Moreover, as Vaadin AJAX requests update the UI, it can really throw screenreaders off.

WAI-ARIA is a standard to define how to make RIA accessible. In order to achieve this, it defines standard roles such as navigation menus, or buttons, even though it’s not a real HTML button. WAI-ARIA is on the roadmap, but has been frozen due to time constraints.

Still, there’s already support for keyboard and shortcut keys.

Button button = new Button("OK");

button.setClickShortcut(KeyCode.ENTER);

A good practice is also to provide screen units in relative size, so that users can increase the font size on their browsers.

Vaadin 7 and what’s next

Vaadin 7 is in beta 8 status. The decision is to deliver when it’s ready, so as not to decrease quality: it should be RC in December. Version 7 is a real major release, with huge core rewrites. Goals included empowering developers, embracing extendability and cleaning up (web is the only supported platform).

After Vaadin 7, the roadmap is not decided yet… The biggest mistake is some things were announced with Vaadin 7, and too many couldn’t be released. Therefore, there’s no plan for Vaadin 8. However, version 7.1 is planned for February 2013 and will include:

  • Built-in push channel
  • New them that puts SASS to full use
  • Start adding client-side APIS to some (few) Vaadin widgets.

Version 7.2 will be more focused towards (client) widgets. For most of the cases, it’s wiser to use (server) components, but in some cases, there’s need for widgets (offline for example). Table and combo-boxes will be redesigned since support outside GWT will be dropped. The target will be summer 2013.

Other non-planned improvements include:

  • Declarative XML-based UIs, mix HTML in UI declarations and integrate with JPS, as well as taglibs, and mix-match them with XML
  • Better IDE tooling: add-ons support, full-theme support and declarative UI editing mode for Visual Editor
  • Book of Vaadin covering v7 is expected in January
  • WAI-ARIA
  • On the fly translations for UI
  • Comprehensive charting library, based on HighCharts and available with a Vaadin Pro Account
  • Java EE 6 CDI with Vaadin (support for injecting UIs and views), the work in progress is available on GitHub

– nighty night, folks, it’s more that 10 PM –

Categories: Event Tags:

Devoxx 2012 – Day 2

November 13th, 2012 3 comments

– after a few hours of sleep and a fresh mind to boot, I’m ready to start the day with some Scala. Inside, I’m wondering how much time I’ll be able to keep on understanding the talk –

Scala advanced concepts by Dick Wall and Bill Venners

Implicit are the first subject tackled. First, know that implicit is one of the reasons that the Scala compiler has such a hard time doing its job. In order to illustrate implicit, a Swing anonymous class snippet code is shown:

val button = new JButton

button.addActionListener(new ActionListener {

	def actionPerformed(event: ActionEvent) {

		println("pressed")

	}

})

button.addActionListener(_:ActionEvent) => println("pressed") // Compile time error with type mismatch

The implicit keyword is a way for the Scala compiler to try conversions from one type to another type until it compiles. The only thing that matters is the type signature, the name doesn’t. Since implicit functions are so powerful (thus dangerous), in Scala 2.10, you have to explicitlyimport language.implicitConversions. Moreover, other such features (e.g. macros) have to be imported in order to be used. If not imported, compiler issues warnings, but in future versions, it will be handled as errors.

Rules for using implicit are the following:

  • Marking rule: only defs marked as implicit are available
  • Scope rule: an implicit conversion must be in scopeas a single identifier (or be associated with either the source or target type of the conversion). This is something akin to inviting the implicit conversion. If the implicit is in the companion object, it will be automatically used. Another more rational alternative would be to put it in a trait, which may be in scope, or not.
  • One at a time rule: only a single implicit conversion must satisfy the compiler.
  • Explicit-first rule: if the code compiles without implicit, implicits won’t be used.
  • Finally, if Scala has more that one implicit conversion available, the compiler will choose the more specific one if it can. If it cannot, it will choose none and just throw an error.

Even though the compiler doesn’t care about the implicit def’s name, people reading your code do so that naming should be chosen for both explicit usage of the conversion and explicit importing into scope for the conversion. Note that some already available implicits are in the Predef object.

In Scala 2.10, implicit can set on classes, so that implicit are even safer. As a corollary, you don’t have to import the feature. Such classes must take exactly one parameter and must be defined where a method could be defined (so they cannot be top-level). Note that the generated method will have the same name as the class.

Implicit parameters can be used in curried defs and replace the last parameter.

def curried(a: Int)(implicit b: Int) = a + b

implicit val evilOne: Int = 1

curried(1)(2) // Returns 3 as expected

curried(1) // Returns 2 because evilOne is implicit and used in the def

Implicit parametes are best used as one of your own type you control: don’t use Int or String as implicit parameters. It also lets you set meaningful names on the types (such as Ordered in the Scala API).

By the way, in order to ease the use of you API, you can specify the @ImplicitNotFound annotation to provide a specific error message to the compiler. Also, when an implicit isn’t applied, the compiler doesn’t offer much debugging information. In this case, a good trick is to start explicitly using the parameter and see what happens. In conclusion, use implicits with care: ask yourself if there’s no other way (such as inheritance or trait).

Scala 2.10 also offers some way into duck typing through the applyDynamics(s:String) (which is also a feature that has to be imported in order to use). It looks much like reflection, but let compiler checks pass-through and fails at runtime instead at compile time (if the desired method is not present).

Now is time for some good Scala coding practices.

  • First, favor immutability: in particular, if you absolutely need to use a var, use the minimum scope possible. Also, mutable collections are particularly evil. If you need to use mutability in this context, it’s most important you do not expose it outside your method (in the signature).
  • For collections, consider map and flatMap above foreach: this tend to make you think in functional programming.
  • Then, don’t chain more than a few map/filter/… calls in order to make your code more readable. Instead, declare intermediate variables with meaningfulnames.
  • Prefer partial functions to bust out tuples:
    wl.zipWithIndex.map { case(w, i) => w.toList(i) } // better than
    wl.zipWithIndex.map { t => t._1.toList(t._2) } // not a good thing
  • Consider Eitheras alternative to exceptions since exceptions disturb the flow of your code.
  • The more if and foryou can eliminate from your code, the least you’ll have to take all cases in account. Thus, it’s better to favoir tail recursion and immutables over loops and mutables.
  • Have a look at Vector over List. The former is constant-time performance…
  • Know your collections!already began!
  • When developing publicmethods, always provide a return type. Not only it let you avoid runtime errors (it compiles because of type inference), it also nicely documents your code and makes it more readable.
  • Deeply think about case classes: you get already-defined methods (toString(), apply(), unapply(), etc.) for free.

– here my battery goes down–

Faster Websites: Crash Course on Frontend Performance by Iliya Grigorik

– Even though backend performance matters, I do tend to think, like the speaker, that in most “traditional” webapps, most of the time is spent on the frontend. I hope to increase my palette of tools available in case it happens –

The speaker is part of the Web Fast team inside of Google. The agenda is separated into three main parts: the problem, browser architecture and best practices, with context.

The problems

What’s the point of making fast websites? Google and Bing tried to measure the impact of delays, by injecting artificial delays at various points during the response. All metrics, including Revenue per User go down, and the more the delay, the sharper the decrease in a linear fashion. As a sidenote, once the delay is removed, metrics take time to go back to their original value. Another company did correlate the page load speed and the bounce rate. Mobile web is putting even more pressure on the load of the sites…

If you want to succeed with web-performance, don’t view it as a technical metric. Instead, measure and correlate its impact with business metrics

Delay User reaction
0-100ms Instant
100-300ms Feels sluggush
300-1000ms Machine is working
+1s Mental context switch
+10s I’ll come back later

Google Page Analytics show that on the desktop web, the median page load time is roughly 3ms while the mobile web, it’s 5ms (only taking in account those new Google phones). The problem is that an average page is made of 84 requests of is and weights about 1MB!

Let’s have a look at the life a HTTP request: DNS lookup, socket connect, request management itself and finally content download. Each phase in the lifecycle can be optimized.

  • For example, most DNS servers are pretty bad. That’s why Google provide free public DNS servers: 8.8.4.4 and 8.8.8.8. A project named namebench can help you find the best DNS servers, in part according to your browser history.
  • Socket connection can be reduced by moving the server closer. Alternatively, we could use CDN.

In the case of devoxx.com, there are only 67 requests but it’s a little less that 4Mb. Those requests are not only HTML, but also CSS, JavaScript and images. Requesting all these resources can be graphically represented as a waterfall. There are 2 ways to optimize the waterfall: making it shorter and thinner. Do not understimate the time taken in the frontend: on average, the frontend takes 86% of the time.

One could argue that improvements in network speed could save us. Most “advanced” countries have an average 5Mb/s speed, and is increasing. Unfortunately, bandwidth don’t matter that much, latency is much more important! Over 5Mb/s, increasing bandwidth won’t gonna help much Page Load Time (PLT) improvements, even though it will for downloading movies :-) The problem is that increasing bandwith is easy (just lay out more cables), while improving latency is expensive (read impossible) since it’s bound by the speed of light: it would require laying out shorter cables.

HTTP 1.1 is an improvement over HTTP 1.0 in that you can keep your connections alive: in the former version, one request required one connection, which is not the case in the later version. Now, this means that browsers open up to 6 TCP connections to request resources. This number is basically a limit, because of TCP slow start: it’s a feature of TCP that is meant to probe for network performance and not overload the network. In real-life, most HTTP traffic is composed of small, bursty, TCP flows because there are many requests, each of limited size (think JS files). A improvement would be to increase TCP initial congestion window, but it requires tweaking low-level system resources.

An ongoing work is HTTP 2.0! The good thing is that SPDY is already there and v2 is intented as a base for HTTP 2.0. HTTP 2.0 goals are making things better (of course), built on HTTP 1.1 and be extensible. Actual workarounds to increase PLT include concatenating files (CSS, JS), images spriting and domain sharding (in order to mislead the browser into making more than 6 connections because there are “fake” domains). SPDY is intended to remove these workarounds so that developers won’t have to sully their nails with dirty hacks anymore. You don’t need to modify your site to work with SPDY, but you can optimize it (stop spriting and sharding)

In essence, with SDPY, you open a single TCP connection and inside this single connection send multiple “streams”, which are multiplexed. SPDY also offers Server Push so that server can push resources to the client… so long as the client doesn’t cancel the stream. Note that inlining resources inside the main HTML page can be seen as a form of Server Push. Today, SPDY is supported across a number of browsers, including Chrome (of course), Firefox 13+ and Opera 12.10+. Additionnally, there are a number of servers that are compatible (Apache through the mod_spdy, nginx, Jetty, …).

The mobile web is exploding: mobile Internet users will reach desktop Internet users in 2 years. In countries like India, it’s already the case! There’s a good part of Internet users that are striclty accessing if from their mobile, the exact figure depends on the country, of course (25% in the US, 79% in Egypt). The heart of the matter is that though we don’t want to differenciate between desktop and mobile access, make to mistake: the physical layer is completely different. In fact, for mobile, the latency is much worse because exiting from idle state is time-consuming, both in 3G or in 4G (though numbers are reduced in the latter case).

There are some solutions to address those problems:

  • Think again SPDY
  • Re-use connections, pipeline
  • Download resources in bulk to avoid waking up the radio
  • Compress resources
  • Cache once you’ve got them!

Browser architecture

The good news is that the browser is trying to help you. Chrome, for example, has the following strategies: DNS prefetch (for links on the same page), TCP preconnect (same here), pooling and re-use TCP connections and finally caching (for resources already downloaded). More precisely, hovering the mouse over the link will likely fire DNS prefect and TCP preconnect. Many optimization parametes are available in Chrome, just go play with them (so long as you understand).

Google Analytics provides you with the means to get the above metrics when your site uses the associated script. An important feature is segmenting, to analyze the differences between users. As an example, the audience could be grouped into Japan, Singapore and US: it let us see that a peak PLT only for Singapore users was caused by the temporary unavailibility of a social widget, but only for the Singapore group.

Now, how does the browser render the bytes that were provided by the server? The HTML parser does a number of different things: create the DOM tree, as well as the CSS tree to create the Render tree. By the way, HTML5 specify how to parse the HTML, which was not the case in HTML 4: bytes are tokenized, tokens are built into the tree. However, JavaScript could change the DOM through doc.write so the browser must first manage scripts before doing anything. Consequence, sync scripts do block the browser; corollary, code async scripts, like so


(function()) {

	// do something here

})();

In HTML5, there are more simpler ways to async scripts, with script tag attributes:

  • regular, which is just standard script
  • defer tells the browser to download in the background and execute the script in order
  • async tells it to also download in the background but execute it when ready

– 3 hours talk are really too much, better take a pause and come back aware than sleep my way through –

In order to optimize a page, you can install PageSpeed extension in Chrome, which will basically audit the page (– note it’s also available as a Firebug plugin –). Checks include:

  • Image compression optimization: even better, the optimized content is computed by PageSpeed so you could take it directly and replace your own.
  • Resizing to 0x0 pixels (yes, it seems to happen and enough there’s a rule to check for it)
  • Looking for uncompressed resources. Web servers can easily be put in place to use gzip compression without updating the site itself

Alternatively, you can use the online PageSpeed Insights

Performance best practices

  • Reduce DNC lookups
  • Avoid redirect
  • Make fewer HTTP requests
  • Flush the document early so as to help the document parser discover external resources early
  • Uses a CDN to decrease latency: the content has to be put the closest to your users
  • Reduce the size of your pages
    • GZIP your text assets
    • Optimize images, pick optimal format
    • Add an Expiresheader
    • Add ETags
  • Optimize for fast first render, do not block the parser
    • Place stylesheets at the top
    • Load scripts asynchronously
    • Place scripts at the bottom
    • Minify and concatenate
  • Eliminate memory leaks
    • Build buttery smooth pages (scroll included)
    • Leverage hardware acceleration where possible
    • Remove JS and DOM memory leaks
    • Test on mobile devices
  • Use the right tool for the job (free!)
    • Learn about Developer Tools
    • PageSpeed Insights
    • WebPageTest.org
    • Test on mobile devices (again)

Reference: the entire slides are available there (be warned, there are more than 3)

Maven dependency puzzlers by Geoffrey de Smet

– seems like a nice change of pace, recreational and all… –

The first slide rejoins my experience: if you deploy a bad POM to a Maven repo, your users will feel the pain (– see last version of log4j for what not to do –)

– he, you can win beers! –

What happens when you add an artifact with different groupId?
Maven doesn’t know it’s the same artifact so it adds both. For users, this means you have to ban outdated artifacts, and exclude transitive dependencies. For providers, use relocation. In no case should you redistribute other project’s artifacts under your own!
What happens when you define a dependency’s version different from the dependency management’s version parent?
Maven overwrites the version. In order to use the parent’s version, don’t use any version in the project.
What happens when you define a version and you get a transitive dependency with a different version?
Maven correctly detects a conflict and uses the version from the POM with the least number of relations from the project. To fail fast, use the maven-enforcer-plugin and the enforce goal.
What happens when the number of relations is the same?
Maven chooses the version from the dependency declared first in the POM! In essence, this means dependencies are ordered; the version in itself plays no role, whereas a good strategy would be to retain the highest version.

– hopes were met, really fun! –

Junit rules by Jens Schauder

– though I’m more of a TestNG fanboy, it seems the concept of rules may tip the balance in favor of the latter. Worth a look at least –

Rules fixes a lot of problems you encounter when testing:

  • Failing tests due to missing tear down
  • Inheriting from multiple base test classes
  • Copy-paste code in tests
  • Running with different runners

When JUnit tries to run a test class, it finds all methods annotated with @Test and wraps them in “statement”. Rules are things that take a statement as a parameter and returns a statement. Creating a rule is as simple a creating a class inheriting from Rule and annotating it with @Rule.Thus, bundling setup and teardown code is as simple as creating a rule that does something before and after evaluating the statement.Rules can also be used to enforce a timeout, to run the test inside the Event-Dispatch thread, or even to change the environment. Additionally, rules can be used to create tests that are run only under specific conditions, for example to test Oracle-specific code in some environment and H2-specific code in other. Finally, rules may be used to provide services to test classes.

Once you start using rules, you’ll probably end up having more than one rule during in the test execution. As a standard, rules shouldn’t depend on each other, so order shouldn’t be a problem. However, real-life cases prove a need to have rules chaining: JUnit provides a RuleChain to achieve this.

Finally, if you need a rule to execute before a single run (something akin to @BeforeClass), you can use @RuleClass.

– IMHO, it seems rules are like TestNG listeners, which are available since ages. At least, I tried –

Categories: Event Tags:

Devoxx 2012 – Day 1

November 13th, 2012 No comments

The 2012 Devoxx Conference in Antwerp has begun. Here’s some sum-ups of the sessions I’ve been attending to. I’ll try to write one daily (unless I’m too lazy or tired, Devoxx has been known to have such consequences on geeks…).

–I followed no conference in the morning since I presented a hands-on lab on Vaadin 7

Modular JavaEE application in the cloud by B. Ertman and P. Bakker

Two current trends rule modern-day application: on one hand, applications tend to grow bigger, thus complex and agile development and refactoring become more common. Challenges that arise from these factors are manyfold, and include:

  • Dependency management
  • Versioning
  • Long-term maintenance
  • Deployment

When building applications for the cloud, there are a couple of non-functional requirements, such as 0-downtime and modular deployments. Once you got more customers, chances are you’ll end up having specific extensions for some, thus even more aggravating the previous challenges.

In university, we learned to promote high-cohesion and low-coupling systems for Object-Oriented systems. How can we lessen coupling in a system? The first step is to code by interface. Now, we need a mechanism to hide implementations, so as to prevent someone from accidentally using our implementtation classes. A module is a JAR extension component that provides such a feature (note that such feature is not available within current JVMs). Given a module, how can we instantiate a new hidden class? One possible answer can be dependency injection, another can be service registries.

Modularity is an architecture principle. You have to design with modularity in mind: it doesn’t come with a framework. You have to enforce separation of concerns in the code. At runtime, the module is the unit of modularity.

What we need is:

  • An architectural focus on modularity
  • High-level enterprise API
  • A runtime dynamic module framework: right now, OSGi is the de facto standard since Jigsaw is not available at the time of this presentation.

–Now begins the demo–

In order to develop applications, you’ll probably have to use the BndTools plugin under Eclipse. A good OSGi practice is to separate a stable API (composed of interfaces) in a module and implementations (of concrete classes) in other ones.

OSGi metadata is included in the JAR’s META-INF/MANIFEST file. In order to make the API module usable by other modules, we need to export the desired package. As for the implementation class, there’s no specific code, although we need a couple of steps:

  • Configure the modules available to the module implementation (just like traditional classpath management)
  • Create a binding between the interface and its implementation through an Activator. It can be done through Java code, through Java annotations or with XML.
  • Configure implementation and activator classes to be private

Finally, we just have to develop client code for the previous modules, and not forget to tell which OSGi container will be used at runtime.

In case of exceptions, we need a way to interact with the container. In order to achieve this, a console would be in order: the gogo console can be hot-deployed to the container since it’s an OSGi module by itself. Once done, we can start and stop specific modules.
–Back to the slides–
Are JavaEE and OSGi fated to be enemies? While it could have been the case before, nowadays they’re not exclusive. In essence, we have 3 options:

  1. Use OSGi core architecture and use JavaEE APIs as published services. It’s the approach taken by application servers just as GlassFish.
    • In this case, web applications become Web Application Bundles (the extension plays no part). Such modules are standard WAR files (JSF, JAX-RS, etc.) only for the Manifest. Extra-entries include the Web-Context-Path directive as well as the traditional OSGi Import-Packagedirective.
    • The goal of EJB bundles is to use EJB and JPA code as now: unfortunately, it’s not a standard yet. Manifest directive include Export-EJB.
    • For CDI, the target is to use CDI within bundles. It’s currently prototyped in Weld (CDI RI).

    All application servers use OSGi bundles under the cover, with supporting OSGi bundles as an objective in the near future. At this time, Glassfish and WebSphere have the highest level of support for that.

  2. When confronted with an existing JavaEE codebase but modularity is needed in a localized part of the application, it’s best to use an hybrid approach. In this case, separate your application into parts: for exameple, one ‘administrative’ part based on Java and the other ‘dynamic’ using OSGi glued together by an injection bridge. In the JavaEE part, the BundleContext is injected through a standard @Resource annotation. –Note that your code is now coupled to the OSGi API, which defeats the low-coupling approach professed by OSGi–.
  3. Finally, you could need modularity early in the development cycle and use OSGi ‘à la carte‘. This is a viable option if you don’t use many JavaEE APIs, for example only JAX-RS.

Note that for debugging purposes, it’s a good idea to deploy the Felix Dependency Manager module.

What happens when you have more than one implementation module of the same API module? There’s no way to be deterministic about that. However, there are ways to use it:

  • The first way is to use arbitrary properties on the API module, and only require such properties when querying the OSGi container for modules
  • Otherwise, you can rank your services so as to get the highest rank available. It’s very interesting to have fallback services in case the highest ranking is not available.
  • Finally, you can let you client code handle this so that it listens to service lifecycle events for available module implementations

It’s a good OSGi practice: keep modules pretty small in most cases, probably only a few classes. Thus, in a typical deployment, it’s not uncommon for the number of bundles to amount in hundreds.

Now comes the Cloud part (from the title): in order to deploy, we’ll use Apache ACE, a provisioning server. It defines features (modules), distributions (collections of modules) and targets (servers for distributions provisioning). By the way, ACE is not only built on top of OSGi but also uses Vaadin as its GUI (sorry, too good to miss). When a server registers on ACE (and is defined as a server), the latter will automatically deploy the desired distributions to it.

This method let us use auto-scaling: just use a script when a server is ready to register to the ACE server and it just works!

References:

Weld-OSGI in action by Mathieu Ancelin and Matthieu Clochard

Key annotations of CDI (Context and Dependency Injection) are @Inject to request injection, @Qualifier to specify precise injection and @Observes to listen to CDI events.

OSGi is a modular and dynamic platform for Java: it’s stable and powerful but provides old API. In OSGi, bundles are the basis of the modularity, those are enhanced JARs. OSGi proposes three main features:

  • bundles lifecycle management
  • dependency management
  • version management

Weld-OSGi is a CDI extension that tries to combine the best of both worlds. It’s a JBoss Weld project, and is developed by the SERLI company. What’s really interesting is that you don’t need to know about OSGi, it’s wrapped under CDI itself. Basically, each time a bundle is deployed in the OSGi container, a new CDI context is created. Also, CDI beans are exposed as OSGi services.

Weld-OSGI provides some annotations to achieve that, such as @OSGIService to tell CDI you need an OSGi service. Alternatively, you could require all services implementations through the @Service annotation so as to use the desired implementation. On both, you could refine the injected services through qualifier annotations, as in “true” CDI.

OSGi containers generate a lot of events, each time a bundle changes state (e.g. when a bundle starts or stops). Those events can be observed through the use of the traditional @Observes annotation, with the addition of some specific Weld-OSGI annotations. A specific API is also available for events happening between bundles.

– demo is very impressive, the UI being changed on the fly, synced with the OSGi container –

References:

Categories: Event Tags:

Scala cheatsheet part 1 – collections

November 11th, 2012 2 comments

As a follow-up of point 4 of my previous article, here’s a first little cheatsheet on the Scala collections API. As in Java, knowing API is a big step in creating code that is more relevant, productive and maintainable. Collections play such an important part in Scala that knowing the collections API is a big step toward better Scala knowledge.

Type inference

In Scala, collections are typed, which means you have to be extra-careful with elements type. Fortunaltey, constructors and companion objects factory have the ability to infer the type by themselves (most of the type). For example:

scala>val countries = List("France", "Switzerland", "Germany", "Spain", "Italy", "Finland")
countries: List1 = List(France, Switzerland, Germany, Spain, Italy, Finland)

Now, the countries value is of type List[String] since all elements of the collections are String.

As a corollary, if you don’t explicitly set the type if the collection is empty, you’ll have a collection typed with Nothing .

scala>val empty = List()
empty: List[Nothing] = List()

scala> 1 :: empty
res0: List[Int] = List(1)

scala> "1" :: empty
res1: List1 = List(1)

Adding a new element to the empty list will return a new list, typed according to the added element. This is also the case if a element of another type is added to a typed-collection.

scala> 1 :: countries
res2: List[Any] = List(1, France, Switzerland, Germany, Spain, Italy, Finland)

Default immutability

In Functional Programming, state is banished in favor of “pure” functions. Scala being both Object-Oriented and Functional in nature, it offers both mutable and immutable collections under the same name but under different packages: scala.collection.mutable and scala.collection.immutable. For example, Set and Map are found under both packages (interstingly enough, there’s a scala.collection.immutable.List but a scala.collection.mutable.MutableList). By default, collections that are imported in scope are those that are immutable in nature, through the scala.Predef companion object (which is imported implicitly).

The collections API

The heart of the matter lies in the API themselves. Beyond expected methods also found in Java (like size() and indexOf()), Scala brings to the table a unique functional approach to collections.

Filtering and partitioning

Scala collections can be filtered so that they return:

  • either a new collection that retain only elements that satisfy a predicate (filter())
  • or those that do not (filterNot())

Both take a function that takes the element as a parameter and return a boolean. The following example returns a collection which only retains countries whose name has more than 6 characters.

scala> countries.filter(_.length > 6)
res3: List1 = List(Switzerland, Germany, Finland)

Additionally, the same function type can be used to partition the original collection into a pair of two collections, one that satisfies the predicate and one that doesn’t.

scala> countries.partition(_.length > 6)
res4: (List1, List1) = (List(Switzerland, Germany, Finland),List(France, Spain, Italy))

Taking, droping and splitting

  • Taking a collection means returning a collection that keeps only the first n elements of the original one
    scala> countries.take(2)
    res5: List1 = List(France, Switzerland)
  • Droping a collection consists of returning a collection that keeps all elements but the first n elements of the original one.
    scala> countries.drop(2)
    res6: List1 = List(Germany, Spain, Italy, Finland)
  • Splitting a collection consists in returning a pair of two collections, the first one being the one before the specified index, the second one after.
    scala> countries.splitAt(2)
    res7: (List1, List1) = (List(France, Switzerland),List(Germany, Spain, Italy, Finland))

Scala also offers takeRight(Int) and dropRight(Int) variant methods that do the same but start with the end of the collection.

Additionally, there are takeWhile(f: A => Boolean) and dropWhile(f: A => Boolean) variant methods that respectively take and drop elements from the collection sequentially (starting from the left) while the predicate is satisfied.

Grouping

Scala collections elements can be grouped in key/value pairs according to a defined key. The following example groups countries by their name’s first character.

countries.groupBy(_(0))
res8: scala.collection.immutable.Map[Char,List1] = Map(F -> List(France, Finland), S -> List(Switzerland, Spain), G -> List(Germany), I -> List(Italy))

Set algebra

Three methods are available in the set algebra domain:

  • union (::: and union())
  • difference (diff())
  • intersection (intersect())

Those are pretty self-explanatory.

Map

The map(f: A => B) method returns a new collection, which length is the same as the original one, and whose elements have been applied a function.

For example, the following example returns a new collection whose names are reversed.

scala> countries.map(_.reverse)
res9: List[String] = List(ecnarF, dnalreztiwS, ynamreG, niapS, ylatI, dnalniF)

Folding

Folding is the operation of, starting from an initial value, applying a function to a tuple composed of an accumulator and the element under scrutiny. Considering that, it can be used as the above map if the accumulator is a collection, like so:

scala> countries.foldLeft(List[String]())((list, x) => x.reverse :: list)
res10: List[String] = List(dnalniF, ylatI, niapS, ynamreG, dnalreztiwS, ecnarF)

Alternatively, you can provide other types of accumulator, like a string, to get different results:

scala> countries.foldLeft("")((concat, x) => concat + x.reverse)
res11: java.lang.String = ecnarFdnalreztiwSynamreGniapSylatIdnalniF

Zipping

Zipping creates a list of pairs, from a list of single elements. There are two variants:

  • zipWithIndex() forms the pair with the index of the element and the element itself, like so:
    scala> countries.zipWithIndex
    res12: List[(java.lang.String, Int)] = List((France,0), (Switzerland,1), (Germany,2), (Spain,3), (Italy,4), (Finland,5))

    Note: zipping with index is very important when you want to use an iterator but still want to have a reference to the index. It keeps you from declaring a variable outside the iteration and incrementing the former inside the latter.

  • Additionally, you can also zip two lists together:
    scala> countries.zip(List("Paris", "Bern", "Berlin", "Madrid", "Rome", "Helsinki"))
    res13: List[(java.lang.String, java.lang.String)] = List((France,Paris), (Switzerland,Bern), (Germany,Berlin), (Spain,Madrid), (Italy,Rome), (Finland,Helsinki))

Note that the original collections don’t need to have the same size. The returned collection’s size will be the min of the sizes of the two original collections.

The reverse operation is also available, in the form of the unzip() method which returns two lists when provided with a list of pairs. The unzip3() does the same with a triple list.

Conclusion

I’ve written this article in the form of a simple fact-oriented cheat sheet, so you can use it as such. In the next months, I’ll try to add other such cheatsheets.

To go further:

I’ve found the following references around the web:

Categories: Development Tags: