Terracotta is a JMS killer

I often feel bad (or maybe it's guilt) at the decision to not use Spring in our software platform. I don't think it would have helped our project, and I certainly don't miss the XML "sit-ups" we would be stuck with now. But, when one of our team leaves (even if that's me), it's going to be one major item missing from their resume.

So, now we come to our current dilemma. Awesome (the code name for our platform) currently has a queue for asynchronously processing tasks. It's not a bad design and is throughly tested. But it's also not the cleanest design. We've encountered another place where asynchronous processing is required. So the time has come to refactor the current queue logic so these, and all future queues, are coded consistently.

My experience tells me that this new queue especially, should be consumed on a remote machine. Warnings of millions of events, each requiring significant processing power, occurring repeatedly for sustained periods of time, are common. Of course, there's no way to currently verify the loads we'll have to support, or the amount of power required to process each event, but my general sense is, we could cripple most machines.

So, my first reaction was to use JMS to manage our queues. This way, we could move them off of our main server easily, and let some other machine churn away on the events without effecting anything else.

But then I realized, at least for the time being, we are developing for a single JVM. Of course, we're planning on using Terracotta to make it a really big JVM, but it's still acting as a single machine. We're also working from a single database and file system. Being able to code for a single machine definitely reduces the complexity of distributed processing, but at some point, this model will not be optimal. I'm pretty convinced that we'll eventually want to move some of the processing off to a separate environment.

So, to use JMS or not? It would certainly look good on everyone's resume. And while we have a persistent queue already coded up and working in production, JMS would handle the persistence for us, but that's probably a "push". JMS would allow us to use a subscription scheme where we currently only support one consumer per message, should we ever need it. But introducing JMS would add more complexity to the overall architecture. The only way to really know which route is best would be to research, test and benchmark. Since our timeline is so compressed and management views QA as a secondary concern, guesswork is all I've got.

All our team can do is build the features we're being told to build (if this sounds bitter, it's not - we have a huge feature list and a very tight deadline). I'm certainly not turning a blind eye to the potential for performance problems down the road, but I don't have the time or resources to make an informed decision. So, for now, we hope that one big JVM can handle the load we're going to encounter.

I hope when shit hits the fan, the switch is set to low.