Poor Man's Message Prioritization

Imagine that you have a distributed enterprise system held together with messaging middleware (JMS, to be specific). Imagine that it's processing a steady load of work in bulk, but not too often there is some time sensitive operation and your system needs to expedite a message or two to meet its time constraints.

If you're lucky, your chosen JMS provider supports message prioritization. Turns out that unfortunately many do not - they simply ignore the priority parameter in MessageProducer.send(). The one I happen to be using also does not support prioritization. Switching to a different provider turned out not to be an option unfortunately - after spending a nontrivial ammount of time (about a month) testing few other, both open source and commercial offerings, none had both the performance and high-availability of the one we originally used.

What I did to overcome the lack of prioritization was "inventing" (quote marks as quite possibly other people had the same idea already) what I'm referring to as "poor man's message prioritization" - instead of using a queue FOO, I now use two queues FOO and FOO_H (for "H"igh priority) and split the JMS sessions used to process the FOO queue into two separate sets of dedicated processors for FOO and FOO_H, typically with only few sessions servicing the "_H" queue, i.e. a 95%/5% split.

What's important to realize is that what's implemented here ain't a message prioritization. It's just separation of traffic. Logically, the queues FOO and FOO_H are treated from the system's point of view as a single queue, and I refer to the individual "physical" queues as "lanes". Sometimes I refer to this technique as "laned JMS".

It does however create the illusion of prioritization as long as the ratio of prioritized traffic to nonprioritized traffic is lower than the ratio of threads processing prioritized traffic to threads processing nonprioritized traffic. It's a workaround for an annoying lack of a feature in the JMS provider. I won't pretend I'm happy with this - it works, and the implementation I came up turned out to be quite clean, but it's still more work to do for me and more complexity in my code; it should've been taken off my back by the JMS provider vendor.

As for technical implementation, it might sound tedious to implement this across the whole codebase, but is actually not that awful:

• On the message producing side, it's sufficient to write a wrapper implementation of JMS Connection interface that is mostly pass-through, except it creates wrapper Sessions, that are themselves mostly pass-through, except they create wrapper MessageProducers, that are again mostly pass-through, but implement the send method with a priority parameter so that if priority is less than 6, the message goes to the base queue, otherwise to the "_H" queue. This way, client code that sends messages just keeps using the Connection.createSession(), Session.createProducer(), and MessageProducer.send() API with priority parameters and is just blissfully unaware of laning in the background.


• On the message consuming side, it'd be tricky to implement this across the board, but luckily all my JMS based message consumer services build on a common codebase that I modelled after the Servlet container/Servlet API model - basically, I have a "JmsServiceContainer" (analogous to Servlet container) that deals with session management and other JMS plumbing, and a "JmsService" interface (analogous to the Servlet interface) whose implementations are more-or-less factories for session-specific MessageListeners that do the actual work on a single message. So, I only had to write the enabling code for laned processing in the "JmsServiceContainer" - basically assign the same JmsService to both lanes of a queue. This way, the "JmsService" implementations also remain blissfully unaware - they're just creating message listeners, and don't care that the messages for those listeners now come from two different queues.