Results
This page presents the results from our benchmark and about the ease of use of each solutions.
All performance tests were done with the JMeter application.
This graph below summarizes the results got from the different performance tests. We tested the connection with 1, 2 and 3 threads to see the impact of scalability. Each thread perform 50 connections.
These results don't show a lot of differencies between the implementations. Spring RESTful and Spring Remoting with HTTP Invoker give the best results. Jersey is a bit better than Spring with JAX-WS with only one thread but it seems to handle badly the scalability, as the number of requests doesn't improve a lot with 2 and 4 threads.
The second property that we compared between the APIs is the general ease of use, based on our experimentations.
Jersey is clearly the most simple API. To export a service, just add an annotation and it is almost done. The library is also a light dependency. As a negative point, one could say that there is no real parameter that can be tweaked in order to optimize the underlying behavior to export service.
This way of exposing services with Sprint is also simple to use. Comparing to Jersey, there are however more big dependencies in order to install the framework.
The setup for exposing services with Spring Remoting is a bit tedious. Spring beans are used to handle the service. We set the properties in a Spring XML configuration file. As other spring methods, this is a good candidate when used in an existing Spring-based application.
This required almost the same configuration as with the HTTP invoker. This is just the remoting technology that is different. It uses the standard JAX library to expose web services. Then, using it is quite comfortable.
To conclude, we can say that the best solution depends on the context. The perfect solution doesn't exist.
Jersey is perfect for a simple and fast solution to expose configuration services, as the setup is simplified at best. It is however a bit slower than other solutions with certain parameters.
Spring solutions fit at best on existing Spring-based applications. We have more control, e.g. to access low-level HTTP connection parameters. But this involves also more complexity when using them.
At this point, our benchmark tests also revealed that the Vaadin framework uses a lot of resources. Indeed, it seems to be the real bottleneck, involving several HTTP connections to show just one simple page.