Queue Patterns

Here is a list of all my Queue Patterns:

Rocket Anti-Pattern

Paratrooper Model

Marsupilami Pattern

Indian Train Anti-Pattern

Indiana Jones Pattern

Hamburger Pattern

Water Drop Anti-Pattern

Santa Claus Pattern

Godzilla Anti-Pattern

Opera Lift Pattern

Next time, I can move to my Thread Patterns.

Opera Lift Pattern

In my last part, I presented the Godzilla Anti-Pattern, where you would merge so many messages that a monster size packet would be sent and have negative effects.
 In this part, I would like to present the Opera Lift Pattern. From all the Patterns I presented in this serie, this one is the only one for which I did not come up with the name. I leave all credit for the pattern as well as for the name to our architect while I was working at MyCom, Hugues Bouclier.

The name of this pattern was inspired by the lift in the Opera metro station. If you’ve been there, I must point out that the name was not inspired because the lift is often not working, neither because of the smell. What characterize this lift is that it has a timer. When someone leaps in, the timer starts counting 30 seconds. That gives chance to more people to join. On one side, the lift has a maximum capacity, so you avoid the Godzilla Anti-Pattern. And on the other side, by waiting a bit, you avoid packets that are too small. Of course, if your lift is full, you might also choose to make it leave right away without waiting for the timer to end.
This is the last of my Queue Patterns. Next time, I can move over to the Thread Patterns.

Godzilla Anti-Pattern

Last time, I presented the Santa Claus Pattern, where you would send several messages wrapped up in the same packet to lower payload and increase performance.
However, if you create your packets without discrimination, sooner or later, you’ll meet Godzilla, the packet of monstrous size. Sending huge packets on your intranet can have quite a bad effects on all the other applications of your ecosystem. Some communication frameworks even forbid packets that are too big. For instance, I recently met Godzilla in a system using Weblogic, where maximum packet size is set to 50MB by default. Even more recently, I met a similar case with OmniOrb.

On a User Interface, big packets will make your windows seem to freeze. Although the processor is working at full speed to handle all your messages, the repaint will happen only after the whole packet is handled, and the user will not understand why nothing happens on the screen.

The way to handle Godzilla is of course to set a limit to the size of your packets. If we take the code from Santa Claus, a simple modification will help you avoid Godzilla:

private void sendData() {
  if (!queue.isEmpty()) {
    List<Message> packet = new ArrayList<>();
    queue.drainTo(packet, MAX_PACKET_SIZE);
    sendPacket(packet);
  }
}

As a way to fight both Godzilla and its opposite, next time I’ll show the Opera Lift Pattern.

Santa Claus Pattern

In a previous post, I talked about the Water Drop Anti-Pattern, where the ratio of payload to data was leading to poor performances. A way to alleviate this problem is to introduce the Santa Claus Pattern. Like Santa Claus, we will make packets. But instead of packing presents, we will pack several data messages into a bigger one. This way, the payload of sending this packet on the intranet will be insignificant compared to the useful data. Also, for a user interface, we will spare several repaints, leading to better performance.

There are of course several ways to implement this pattern. A possible solution is to use a timer:
 
 public void init() {
  executor = Executors.newSingleThreadScheduledExecutor();
    executor.scheduleAtFixedRate(this::sendData, DELAY, DELAY,
TimeUnit.SECONDS);
} 
private void sendData() { 
if (!queue.isEmpty()) {
List<Message> packet = new ArrayList<>();
queue.drainTo(packet);
sendPacket(packet);
}
}


But be careful with this pattern, as you might wake up the Godzilla Anti-Pattern!

My first steps with Java 9

Java 9 is finally out! So I decided, as a first approach, to try and start our main application with it, just like this, without any re-compilation.

A Dance of Flags

First things that need correcting are the start-up JVM flags. Some changed names, other simply disappeared. A good thing that the JVM helps us with explicit error messages.
In my case, I had to remove the -XX :+PrintGCDetails and -X :++PrintGCDateStamps flags, which do not exist anymore with the G1 garbage collector. I also had to replace the -Xloggc :gc.log with that one: -Xlog:gc:gc.log.

BootClassPath no more

My first surprise came from the disappearance of the BootClassPath. Of course, I heard about it, and I felt lucky that we migrated a couple of years ago from Orbix, a CORBA implementation (yes, this technology is still in use in 2017…) that installs its libraries in the BootClassPath, to JacOrb. But even without any hint of a BootClassPath in the command line, it still managed to bite me.
When we start our application, we display useful pieces of information about the environment in the log file. You can find there the command line, for instance, or the classpath. And of course, the famed BootClassPath. It turned out that the method we use for retrieving it, RuntimeMXBean.getBootClassPath(), is now throwing an UnsupportedOperationException.

What Java version again?

One try/catch later, I stumbled into the change of version number. We are moving from java version ‘1.8.0_111’ to version ‘9’. Once again, in spite of being well aware of this change, it still got me. I had commented out beforehand the part of our code that was forbidding our program to run with anything else than Java 8 (internal politics…), telling myself that I would go back to it later and try to use the all new java.lang.Runtime.Version class.
However, the problem came from an external library called Docking Frame. It tried for some reason to discover with which java version the application is running. I found myself facing an ArrayIndexOutOfBoundsException. Fortunately, someone else already solved this bug in the library's latest release. I only had to update its version.

The Forbidden Packages

Now, Java 9 forbids us to access any class coming from the sun.* or com.sun.* packages. And we are using 2 such classes in our application. The first case was easy to fix. I replaced this code:
if (UIManager.getLookAndFeel() instanceof WindowsClassicLookAndFeel)
With that one :
if ("Windows Classic".equals(UIManager.getLookAndFeel().getName()))

The second case was harder to fix. We are using the ToolkitImage class for its quite practical getBufferedImage() method. I found a replacement in the javax.imageio package.

Modules put to rest

All this was just a warm up. My next hurdle came in the shape of a NoClassDefFoundError for a class that is actually inside the JDK, more precisely in the java.corba package. As it turned out, some Modules are never loaded at start-up, probably because they are deemed as rare, or too “JEE”. And CORBA falls into that category. To solve that problem, I needed to add a ‘--add-modules java.corba’ flag to the command line.
In fact, we are using other such Modules, such as java.xml.bind or java.activation. So I decided, against all good practice described in the Java 9 Migration Guide (https://docs.oracle.com/javase/9/migrate/toc.htm), to use the ‘--add-modules ALL-SYSTEM’ flag. Of course, this is only a temporary solution…

Module Patching

Great! We have now CORBA Modules loaded, but JacOrb doesn’t seem happy. It looks for several classes in the omg.corba package, which belongs to Java, but those classes are nowhere in sight. The explanation is to be found in the omg-jacorb.jar library, bundled with JacOrb release, and that declares some of its classes in the Java package. Of course, now, this is all forbidden. You cannot declare anymore classes in a package that does not belong to you.
But you can patch a Module. This is easy, you just add a new flag to the runtime: --patch-module java.corba=omg-jacorb.jar.

Invisible Packages

Unfortunately, patching a Module is not quite enough. Indeed, many packages are not exported in the java.corba Module, because they are for internal use only. But accessing internal classes is actually the reason why JacOrb reverts to declaring its classes in the omg.corba package. So I have to declare, one by one, all the sub-packages to which it need access.
That is quite a long and painful process, where I’d add a flag, start the application, check the next exception, then add a flag again… At the end, I had to declare around ten packages with flags similar to this one: --add-exports java.corba/org.omg.GIOP=ALL-UNNAMED. It translates to: « Dear Java, could you kindly make the org.omb.GIOP package from the java.corba module visible to all the classes loaded by the classpath? »

Java is 64 bits

One last hurdle and I’ll be finished. Java is released in 64 bits version only. And our application is loading a 32 bits DLL. That makes our application crash. As luck would have it, this DLL was not needed anymore thanks to a recent migration to a full java version of an external library. We just never came around to remove the dependency.

Done!

That’s it! I did it. I have a Java 9 compatible application. It was not trivial, I had to add several flags to the command line. But most of the hard work was due to libraries that were not ready. My next step: a java 9 compilation. But that’s for another time.

Water Drop Anti-Pattern

In my previous entries, I presented two patterns that allow you to limit the number of messages sent over a queue: the Indiana Jones Pattern and the Hamburger Pattern. The first one filters data while the second merges it.

Once all your data is filtered and merged, messages still arrives individually, one after the other, on your queues. In some cases, it might introduce the Water Drop Anti-Pattern, where your data arrives like from a tap from which drops are falling one after the other.

Consider an application where you highest layer sends data through a network. Each individual message comes with a payload: it has a header which allows it to arrive to the correct destination. The smaller the message, the highest the payload.

If your highest layer is a user interface, your message will also have a payload: the repaint method which needs to be called after each update. Usually, this is the most expensive operation in your interface.

Next time, I will show a way to solve this problem: the Santa Claus Pattern. Wait for it!

Hamburger Pattern

Last time, I introduced the Indiana Jones Pattern, where you would filter your messages in order to send less data on your overcrowded queue.

Another way to reduce the number of messages is to use the Hamburger Pattern. Instead of separately sending the beef, the salad, the onion and the lettuce, why not send a complete hamburger at once? Imagine you are sending coordinates through the network. If you have two messages in your queue, one updating X, and another updating Y, you can merge both messages into one sending both coordinate changes at the same time. In the same way, if you have two messages, both updating X, you can also decide to merge them, sending only the second update.

This is more difficult than filtering, since you need to have access to your message queue and be able to remove messages from the middle. The default implementation of Thread Pools in Java uses queues that are optimized for adding at the end and removing from the front. Here, you will clearly need a different implementation.

public class Display {

    public void run() {

        while (true) {

            Data data = queue.getNextMergeValue();

            show(data);

 }
    }

}

After you filtered and merged all your data, you might end up running into the Water Drop Anti-Pattern. Next time!

Indiana Jones Pattern

In my previous entry, I presented the Indian Train Anti-Pattern, where we add without discrimination data to handle on our queues, leading to overcrowded queues.

One possibility to alleviate the problem is to use the Indiana Jones Pattern. In his Last Crusade, Indiana Jones had to go through three trials in order to prove himself worth of the Graal. We can imagine data going through a similar process to prove themselves worth of the access to the queue. We are in fact introducing a filter.

For instance, we do not wish to add to our queue an event telling us that data was not modified. Another example would be to send an event to our UI to notify of a data change for an item that is not even visible on the screen. We can imagine a large table with 10000 lines, but only 100 of them are displayed at a time.

public class Display {

    public void show(Data data) {

        Data prevData = getPrevData(data);

        if (data.equals(prevData)) return;

        if (!isDisplayed(data)) return;

        SwingUtilities.invokeLater(() -> table.updateRow(data));

    }

}

This is one way to reduce the size of the queue: drop some of the events. Next time we’ll see another way: the Hamburger Pattern. Stay tuned!

Indian Train Anti-Pattern

Last time, I talked about the Marsupilami Pattern, where we introduce queues to transport data between application layers.

Now you’ve been using these queues for some time, conveying all sorts of data, and you realize that they are getting quite crowded, like those Indian Trains we see sometimes on TV. So crowded that people are standing on the roof. You might think that this is not a problem. Those are internal queues, so you can handle all this data internally, adding some more memory. Data might lag a little, but you are not doing “real time”, some seconds do not hurt.

But what if the Indian Train enters into the UI Thread? What if your user clicks on the menu bar, and the mouse events ends up at the end of the train. Would he want to use program where menus take several seconds to appear?

Or what if the Indian Train gets sent on the network? It might impact all the other applications sharing the network with yours, and they might not be happy playing the role of those sacred cows watching the Indian Train pass.

But what is to be done? Your Display class, for instance, is quite straightforward:

public class Display {

    public void show(Data data) {

        SwingUtilities.invokeLater(() -> table.addRow(data));
    }

}

Data is received, data is displayed. So simple. But sometimes, simplicity is too naive. One possible solution is to use the Indiana Jones Pattern. Wait for it!

Marsupilami Pattern

In my previous entry, I described the Paratrooper Model, where each application layer would have its own thread pool for handling data. And the way to implement that is to use the Marsupilami Pattern.

It might be that you haven’t heard of the Marsupilami. You have to know that there are three countries in the world where comic strips are akin to a cult: the US with Marvel and DC Comics super heroes, Japan with their mangas, and France. Growing up in France, my childhood was populated with comics characters such as Asterix, Tintin, Lucky Luke, or the Smurfs. Or the Marsupilami, a creature half way between the jaguar and the monkey, living in the Amazonian forest and who solves all his problems using a very long tail.

That makes for a nice pun in French, since we use the same word for tail and for queue, and here, we will solve our Paratrooper Model problem by introducing a queue. Each application layer is separated from the next one by a queue. When data is handled, we just push it to a queue, from which next layer’s threads will draw, while we can concentrate on our next data.

In Java, this is really easy. If we go back to our Calculator class from last time:

public class Calculator {

    private final Executor executor = Executors.newFixedThreadPool(5);

    public void push(Data data) {

        executor.execute(() -> calculatePrice(data));

    }

    public void calculatePrice(Data data) {

        data.price = data.unitPrice * data.quantity;

        Display.show(data);

    }

}

The important line here is the one where we create a new Executor. In Java, with this one line, you create two things: a thread pool, and a queue (the Marsupilami’s tail).

But beware, if you do not pay enough attention, you might end up with the Indian Train Anti-Pattern. Stay tuned!

Paratrooper Model

In my previous entry, I was describing the Rocket Anti-Pattern, where one thread would handle too much work across several application layers, leading to communication buffer overflow and message loss.

 To solve this problem, you can use the Paratrooper Model. When paratroopers jump from a plane, they are not waiting for the previous guy who jumped to land safely on the ground before jumping themselves. They are jumping as soon as they can after the guy that precedes them left the place. It should be the same with your data. The thread that receives and transcodes the messages should only do that and nothing else. It should work as fast as possible, and let another thread go on with the work in the next application layer, while concentrating on the next message. This way, we are building a sort of pipeline, with one or several worker threads in each application layer.

Here is an equivalent of the application shown in the previous entry, using our new model:

public class Comm {

    public void onMessage(Message m) {

        Data data = transcode(m);

 calculator.push(data);
    }

}

public class Calculator {

    public void push(Data data) {

executor.execute(() -> calculatePrice(data));
    }

    public void calculatePrice(Data data) {

        data.price = data.unitPrice * data.quantity;

 Display.show(data);
    }

}

public class Display {

    public void show(Data data) {

SwingUtilities.invokeLater(() -> table.addRow(data));
    }

}

Each class represents a layer, and propagates the data to the next layer, which will in turn ask the layer’s dedicated thread to handle it.

To implement this model, you will have to use the Marsupilami Pattern. Wait for it!

Rocket Anti-Pattern

At my presentation at Devoxx Paris this year, The Thread Awakens, I introduced several Patterns related to multithreading. Since it was a quickie, I had only 15 minutes to talk about 15 Patterns. That got me to think: maybe some people would like me to develop a bit more on them. Or maybe some other people would prefer to read about them than to watch me on video (I, for instance, would not watch myself on video…). Or, maybe more surprisingly, some people do not speak french (it might happen).

So for all those people, I will introduce my Patterns in my blog, one at a time. And I’ll start with the Rocket Anti-Pattern.

Data in an application travels like a rocket. It crosses several application layers in the same way that rockets cross several layers of atmosphere before reaching space. Data might enter through a database layer, or a communication layer, then cross one or several business layers, before leaving the application through another communication layer, or a user interface. But sometimes, the analogy goes a bit too far

When you are watching a rocket leaving, you follow it from the moment it ignites to the moment it disappear into the sky. The Java equivalent would be:

public class Comm {

    public void onMessage(Message m) {

        Data data = transcode(m);

        calculator.calculatePrice(data);

        Display.show(data);

    }

}

public class Calculator {

    public void calculatePrice(Data data) {

        data.price = data.unitPrice * data;quantity;

    }

}

public class Display {

    public void show(Data data) {

        SwingUtilities.invokeAndWait(table::addRow);

    }

}

What you see here, is a Message that is transcoded into a Data class, then a price is calculated, and finally, Data is displayed on the user interface. You’ll notice the use of SwingUtilities.invokeAndWait(), which might be too much, but you’ll get the point: the same thread is doing almost everything, and won’t return before data has crossed all the layers.

The drawback of this method appears if you get many messages. In a usual communication layers, messages will fill a buffer, where they’ll wait to be handled. If the handling thread is too slow (like when it waits for data to cross all the layers), buffer will get overflod, and you’ll start to loose messages. And that’s not what you want.

If you wish to avoid the Rocket Anti-Pattern, you’ll need to use the Paratrooper Model, which I’ll describe in my next entry. Stay tuned!