The Duct Tape Programmer and surviving the "death trap"

I really like Joel Spolsky's blog, but I have a lot of problems with one of his recent posts about the Duct Tape Programmer.

He is the guy you want on your team building go-carts, because he has two favorite tools: duct tape and WD-40. And he will wield them elegantly even as your go-cart is careening down the hill at a mile a minute. This will happen while other programmers are still at the starting line arguing over whether to use titanium or some kind of space-age composite material that Boeing is using in the 787 Dreamliner. When you are done, you might have a messy go-cart, but it’ll sure as hell fly.

Seriously? When you're done, you're lucky to still be alive! If you make it down the hill in one piece!

Of course, overengineering can be just as bad as fly-by-the-seat-of-your-pants programming. But, just because you win the race doesn't mean you're going to win. Go ahead and win the first heat. I'll gladly take your spot in the second heat, while you're trying to figure out how to keep your front wheels from falling off.

Remember, before you freak out, that Zawinski was at Netscape when they were changing the world. They thought that they only had a few months before someone else came along and ate their lunch. A lot of important code is like that.

Netscape did win the first heat. Great for them! I'll bet it was high-times while all this was going on. Of course, since that time, Netscape hasn't had their go-cart in any races. I'll bet after the initial thrill of winning, life wasn't so happy for the employees and developers at Netscape.

No, I think I'll stick with taking a little extra time to make sure my shit works. If you don't have time to do unit tests, you don't have time to write the code. There's nothing about testing that requires you to use "templates, multiple inheritance, multithreading", etc., approaches.

In fact, I do agree that it's best to go for the simple approach. But, most duct tape programmers I've met aren't going for simple - they're going for "works when I try it" without any regard for readability. And since they don't use unit tests, if it doesn't work for you, the only way you can refactor their code is by crossing your fingers and feeling like you're flying down a hill at a mile a minute on a bucket of bolts held together by duct tape and wd-40! No thank you!

I'm such an asshole (and I really think it's going to pay off big)!

The company I work for moved into new office space awhile back. We simply outgrew the old space (well, that, and the boss went in with some other investors to buy a building in China town).

One of the worst parts of the old space was that the developers sat in "the back room". To get to the restrooms, you either went out the front door, or through the back door (which was in the back room). This resulted in numerous "drive bys".

A "drive by" can range from a random thought to "the next big thing". It usually starts in the mind of a salesperson or marketing type. The process can start anywhere, but the urinal seems to be a treasure trove of drive by opportunities. As soon as the thought forms (but hopefully not before hand washing is complete), the thinker looks for the first developer they can find. Then, the thinker unloads his wonderous wisdom upon the developer who is supposed to stand in awe, absorbing all of the ingenious and innovate ideas spurting forth. Once the stunned feeling subsides, the developer is naturally going to figure out some way to fit the new feature into the product as soon as is humanly possible (or faster).

Needless to say, drive bys suck! These are ideas that, if fed, will be spread to clients and throughout your entire organizations sales engine. Before you know it, what you thought was an innocent head nod has come back to bite you. If you're lucky, you find that this great feature (which has no specification, definition or clear vision) is already overdue. If you're not so lucky, you get a bug report from a client asking why they just paid your company money for a product when they can't get that feature to work (because, of course, that's the one thing the client bought your software instead of your competitors).

At our new office, we're no longer in the "bathroom path". Drive bys have been reduced exponentially! I owe a big thank you to Cody, who was the dev team manager when we moved into the new space - he worked hard to make sure drive bys wouldn't be an issue, including getting us a door - that locks!!!!

But yesterday, while trying to troubleshoot a failing unit test, here comes Mr. Salesman with "the next big thing". To be honest, as he stood over my desk and asked whether our puny brains could fathom how to realize his incredible concept in software, I didn't even listen. All I heard was "will the new product have blah blah blah" and "no one else does that". And, instead of politely nodding, I shook my head and said "I don't know. You'll have to ask Jeff and Sheryl what's going into the new product". When this didn't seem to satisfy him, I added "I don't know what's going to be in there, but I don't think we'll have that."

First of all, I have since found out, we will, indeed, have this innovative concept in our new product. In fact, it's so innovative that it's already been thought of and defined. Apparently, even though I still don't know what the feature is, it's something we've already built.

Secondly, I suppose I could have saved Mr. Salesman a few minutes by actually listening to him. I could have listened and realized that "yes, we've already built that". I could have even taken some of the credit and made myself look smart by explaining how well we wrote that particular branch of code. I should have probably flushed the error I was trying to track down from my mind, realizing that however long it took to get my head back in the code, it was time well spent to make sure that this saleperson could turn his attention to even grander innovations.

But, my reaction was to basically tell him he was talking to the wrong person. I was polite enough to not be offensive, but steadfast enough to make sure there was no question that I would not play the "define a feature" game. If you want to talk about whether a feature is technically possible, then by all means, let's chat! If you want to understand how our architecture will impact future growth, integration possibilities or performance, then I'm all in! But, if you want to tell me about a rough draft of an idea that occurred to you in the shower, then fuck off! I have better things to do!

So, as bad as I feel about turning Mr. Salesman away with the understanding that I will not take part in inflating his ego, I think it will pay off! I'm pretty sure I've helped cure someone of "drive by disease".

Zend_JSON_Decode barfs on funky characters

Zend works fairly well with json, but we've found many problems with funky characters. The latest character was an "end of text" - \u0003. No idea how it got input into the system, but now that it's there, we need Zend to be able to deal with it.

Thanks to Katie for helping us figure out what characters are causing problems!

Is Java dead? Are you kidding?

Posts that contemplate whether or not java is dead are amusing at best and annoying at worst. Regardless of which company bought who, or which scripting/functional/newfangled language is so much better, or how many people are "jumping ship", Java will continue to thrive for years!

This is no different than people proclaiming that Linux would spell the demise of Windows (I must admit, I was among those fools). True, Windows has much less market share than it once did, but it is by no means dead.

Heck, if COBOL continues to run "almost three quarters of the world’s business applications", then Java will surely survive 2009! And if Java is currently the most popular language, saying that Java is doomed seems pretty ignorant.

Of course, if by dead you mean something you won't use anymore, then sure . . . move along! Those of us who view languages as tools will continue to use it (and claim the money that companies are still willing to hand out). You go ahead and make decisions based on what language "feels" better and which one you think will get you the most respect. In the meantime, I've got work to do, and that my friend, involves writing heaps and heaps of Java code!

Creating an Easy Mock Custom Matcher

This blog isn't really supposed to contain "how-to" entries (it's really just a place for me to vent). However, in the past I've found certain tasks worth putting somewhere online in case I need them in the future. I've used planet-source-code for snippets, sourceforge for projects, and maybe a few other sites here and there. Custom matchers in Easy Mock are just hard enough to want a reliable tutorial. While there may be better posts about this same subject elsewhere, there's something about reading code you've written to remind you how to write more code!

So, here's my "CollectionMatcher" class:

package project.test.common;

import java.util.Collection;

import org.easymock.IArgumentMatcher;
import org.easymock.classextension.EasyMock;

public class CollectionMatcher implements IArgumentMatcher {

 private Collection collection;
 private String notFound;
 
 public CollectionMatcher(Collection collection) {
  this.collection = collection;
  notFound = "";
 }

 public static Collection collectionEq (Collection collection) {
  EasyMock.reportMatcher(new CollectionMatcher(collection));
  return null;
 }
 
 public void appendTo(StringBuffer buffer) {
  buffer.append(notFound).append(" not found in {");

  String comma = "";
  for (Object o : collection) {
   buffer.append(comma).append(o.toString());
   comma = ",";   
  }
  
  buffer.append("}");
 }

 public boolean matches(Object otherCollection) {
  if (!(otherCollection instanceof Collection)) {
   return false;
  }

  Collection otherText = (Collection)otherCollection;
  
  for (Object o : otherText) {
   if (!collection.contains(o)) {
    notFound = o.toString();
    return false;
   }
  }
  
  return true;
  
 }

}

So, now we can do things like:

package com.project.api.delivery;

import static project.test.common.CollectionMatcher.*;
import static org.easymock.classextension.EasyMock.*;

//other imports

public class PreviewCommandTest extends TestCase {

  init(new PreviewCommand(preview));
  
  {
   ContentReplacementManager manager = managerContext.getContentReplacementManager();
   expect(manager.replaceValues(content, null)).andReturn(content);
   expect(manager.replaceTags(eq(content), (Preview)eq(null), (Collection)collectionEq(entity.getPreviews()))).andReturn(content);
   managerContext.doReplay();
  }
  
  command.invoke(webContext);
  
  managerContext.doVerify();
  
 }
 
}

Testing the internet (or how to mock a web service)

Part of the architecture of project Awesome involves an engine that processes a high number of tasks, each requiring a significant amount of work. While I can't explain what the engines main responsibility is, this is related to the persistent queue I mentioned in a previous post.

At the end of each unit of work, a message is sent to an external system for additional work to be done. However, because the external system is essentially a non-blocking remote queue, we get no feedback or information about the state of the work being done. To solve this, there is yet another service which returns feedback on the process of each unit of work as it executes on the remote system. We reconcile the feedback with each unit of work sent for processing.

Because there are so many steps involved, testing this entire process is a bitch. Testing this entire process in Fitnesse is even harder. The engine spans threads to periodically check the incoming queue and poll the feedback service. Since Fitnesse runs in a single thread, we will need to test some of the threading issues with JMeter (for example, if two callers attempt to place the same item in the queue, only one should be accepted). However, we don't have the resources to do JMeter, so for now, we'll have to hope that our unit tests are enough.

So, the external system gets mocked in Fitnesse! To accomplish this, we create an actual HTTP server on the local system which we can control within test tables. Fitnesse starts up this mockable server, and then starts the engine, telling it to call the address of our mocked service. With no changes to it's existing code, our engine is now processing records that we can include directly from our wiki.

While this doesn't test the entire engine, it does allow us to test one critical piece. We can now verify our engine performs correctly, regardless of the state of actions of the external service (in fact, we can even make it do things it shouldn't ever do - just in case).