Development vs. Manufacturing

A colleague pointed me to the this link which is lecture notes from an MIT class. It points out some common fallacies (derived from a study by Yuko Fujigaki) in running software development and cites the classic mistake as running software development as you would run manufacturing.

What is pointed out is that the time management techniques used in running manufacturing cannot be associated with software development without modifications. What are those techniques, then, and what modifications are required?

Taking from another article on a blog that I enjoy by Jeff Atwood, there are some parallels between software development and manufacturing, but they're often misapplied.

In manufacturing, you spend a whole bunch of time up-front doing design work and end up with a blueprint for what you want to build. You (loosely speaking) feed that blueprint into a bunch of machines who churn out your end product The traditional software-as-manufacturing metaphor maps "creating blueprints" to "writing a spec", "bunch of machines" to "developers", and "end product" to the code that those developers produce.

The revised metaphor instead maps "creating blueprints" to "writing code", "bunch of machines" to "compiler", and your end product is the binaries that you produce.

Using this metaphor better emphazises the challenges and efficiencies of software development as compared to manufacturing.

First, it casts the coding process as the creative design step that it is, rather than suggesting that developers are a bunch of automatons that take specs in at one end and churn out software at the other.

Second, it acknowledges the need for a detailed spec, but casts the code itself in that role. After all, what more detailed spec of how software works is there than the code itself? And unlike real manufacturing where screwing up the blueprints and starting to manufacture something incorrectly is extremely costly, actually "manufacturing" something from code is essentially free. Appreciating this frees you up to try different things with your blueprints to see which one produces the best end product.

So what does that mean? It simply means that the idea that the actual process of writing code is often misconstrued as being the easier part once specs are developed. The truth is, once a real spec (and what specs actually are is a topic in and of itself) is produced, the fun begins. With a spec in hand, a developer has a roadmap, but faces all of the challenges of working within the current codebase (or architecting if it's turnkey work), stumbling on functions that weren't written with scope and expansion in mind, data models, interface design, etc...

What this boils down to is the notion of time management in the software development process. The analogy I often use when discussing time management is asking a mechanic how long it will take to fix your car when the only description of the problem he/she has is "it's broken." You can narrow it down for them a bit with "it won't run" or even further with "it will start but won't move" but even with that detail you're not going to get a straight answer. It's not until the mechanic has had a chance to look at the vehicle that you're going to get anything resembling a time and cost to fix the problem.

For some reason, that's usually not acceptable in software development. Hard dates are demanded, and the developer has to live up to that date. And of course, when that date turns out to be short sighted, what happens is a stressed developer hacks in bad code at the last minute if they don't fight back on the deadline. That process leads to a common cycle in development where bad code is deployed, bugs are discovered, a client becomes unhappy or unhappier, more deadlines are set for the bug fixes, rinse/repeat.

What can be done to solve these problems? First, management of software development need to be aware that the development process is more art than science. There is no one way to solve any problem, and it's the thought process that gets to any of the solutions that takes the bulk of the time on the project. Without intimate knowledge of the codebase and the requirements, a hard date is impossible to define. It becomes less impossible when those two are known entities, but stumbling blocks, interruptions, scope creep, and any number of things can push that date out.

WWPD (What Would Paul (Graham) Do)

For those who don't know, Paul Graham has accomplished quite a lot in my industry, including cofounding Viaweb (now Yahoo Stores), and contributing to Lisp projects. More importantly to me, he offers tremendous insight into development through his website and blog.

I had yet another wonderful moment here at work last week. Late Thursday, as my team was trying to wrap up the unanalyzed, under-resourced project that affected all of our clients' ability to bill, another issue came up.

The way client issues are handled by this company are thus. A client calls, says they have an immense problem (which may or may not be the case), that it's a problem we've created (which may or may not be the case), and our senior management makes it the highest priority. Immediately all hands are on deck scrambling to figure out what the issue actually is, and all other work suffers.

This happened last Thursday. As it turns out, we fixed a problem with patient statements a few months ago. The previous development team did not tie patient statements to the account ledger, but instead used the source tables which tend not to know about adjustments, write-offs, etc... the usual information found in an account ledger. We completely rewrote them and they now work beautifully.

This particular client didn't use best practices for charge writeoffs and apparently didn't read the release notes nor bother to watch the training video we provided on the subjects. For a couple of months it seems they were sending patient statements to patients whose debt had already been written off. This apparently inundated the practice's call center with questions from these patients.

The MO here is usually that development, business analysis, training, and client services was wrong because the client said so.

After a few hours of analysis we determined what the problem was (our legacy versions' business analyst is unbelievably good at what she does) and reported it back up the chain and to the client. You can still hear the crickets chirping.

Meanwhile, the secret code was uttered from senior management to the product manageer. "Tell Jeff 'Nobody goes home until the EMC work is done.'" That was the project that was behind and pushed aside by this new important issue. And that is the code that senior management used four days in a row while they were at a conference and couldn't bear to tell clients we had a process for analyzing issues and would look into their particular "pink font" issue. That is the code that makes me want to quit my job and flip burgers for a living.

My response this time was "feel free to tell them I'll go home when it gets late, and if the work isn't done, I'll consider myself fired and never come back." In a fraction of a second I remembered a company policy regarding timely notice and lack of payout of accrued vacation pay. So I ammended the statement to "Wait.. you guys owe me an assload of vacation since I can never take one, so I'll come back with my letter of resignation."

Three people have stopped by my desk since to make sure I still worked here. I don't know if I really intended to quit on the spot, but the moral of the story is a developer/team lead/manager can only take so much of fireman mode. That mode causes the real work to suffer, and on the back end of that suffering are more fires to put out because the work was tied to a deadline and hurried or interrupted so often that the abstract vision was difficult to maintain.

The conclusion I came to over the weekend was that this incident only accelerates my desire to find work with some other company that mistreats its developers... only closer to home. And for better money. And maybe with a candy machine in the break room.