The Paradoxes of Software Development

When you've been working in Software Development for a while, you eventually wise up to the three important factors that drive any project: Time, Money, and Scope. (Sometimes people replace that third option with "Quality", but I think that the three of them together make up the three sides of a "Quality" triangle.)

They are certainly the three things that drive any Project Manager, although you sometimes hear them talk about it using different words:

1. Schedules, Milestones and Deadlines
2. Budgets, Resources, People, Equipment, Tools, Training, etc.
3. Features, Requirements, Work Breakdown Structure

When you start any project, you always desire lots of each of these. In reality, you are constrained in each. You can't have all the time in the world. You can't have all the people, tools, training, or equipment that you'd like. And someone is always trying to sneak in more features, but there are limits to what you can put into any software release if you ever want to get it out the door and make some money from it.

Consultants use these three factors (Time, Cost, and Scope) during contract negotiations. The experienced consultant will be able to take any two fixed factors and give you a practical estimate for the third.

So what are the paradoxes? These three factors are, but not for the reasons listed above so far.

Time

You never have enough time. Never. Live with it.

It's more than that though. When you are at work, your time is always interrupted. This is unfortunately unavoidable. It is the nature of Software Development that we miraculously create a complex masterpiece through a process of continuous interruption.

Interruptions come in different forms. Everybody expects to take bathroom and mealtime breaks. But what about the interruptions by email? What about when someone calls you on the telephone or shows up at your desk? Bam! There goes your train of thought!

Of course, it's not polite to yell at people to go away and leave you alone. (Although it may be very tempting with some people.) The reality is that, as a complex human activity, software development requires that a lot of people get the information they need to do their part of the project. Everybody has a part in creating information, so everyone has a responsibility to make sure that the information they know is passed onto the people who need it.

You could be on the verge of solving a complex problem that has eluded you for days, when all of a sudden a Project Manager shows up at your desk to ask you your opinion on the risks involved in making a change somewhere. Or it could be an impromptu meeting. Or a technical writer asking you to review some documentation that you have some familiarity with.

I knew a programmer once who said that he was far more productive when he worked at home for two reasons: (1) he didn't have to get up and leave the building whenever he needed to smoke (he could just smoke at his desk at home), and (2) no interruptions.

The problem of course is that when he does get back into the office, he will eventually be inundated with questions about the work that he did -- in order to pass that information onto the next person who needs it. He also eventually needed to collaborate with other programmers to understand the integration of his work with theirs.

Try as you might, you cannot make software in isolation forever. You may need to get away from others for a while to help you focus on a problem or goal, but you cannot stay away for long. You just can't.

Money

This is a funny one. Not everyone is immediately aware of the importance of the money available for a project until they try to get something and realise they can't. For example, it would be really cool if we could get this "super analyzer metrics-reporting database-query problem-solving automatic-tester all-in-one XML-friendly scriptable-API user-friendly reliable coffee-making appointment-remembering time-saving dry-cleaning fully-customizable easy-to-use tool" ... until they see the price tag.

Or worse, you realise that you can get some of the tools you need cheaply, but you don't have the time to spend figuring out how it can be integrated into your project or environment. Is there any way that we can hire a contract person (e.g. a high school or university student) to figure it out for us so that we can reap the benefits of the tool without actually spending any of that time figuring it out for ourselves?

What? No money in the budget for extra hires? But how can we meet the deadlines if we don't have the help/tools/equipment/training/whatever that we (think we) need?!

Start-up companies generally have a harder time with budgets and available finances than larger, more established, global organisations, but big companies have their share of money problems too.

The funny thing about needing money is that often you get it - but at a price! You will end up "owing" someone, in one way or another. Nothing comes free, and money is at the top of the list.

Many years ago, I had a terrible experience working for one company when we had a person leave our test team. I approached the Director of Development about when I could start hiring her replacement, but then he started giving me the run-around, talking about the "return on the value" of our test team, and that perhaps the funds allocated to the test team salary budget could be better reallocated to hire another programmer instead. After all, we would certainly be more productive with another programmer and one less tester, wouldn't we?

I quit a few months later. I have been extremely happy with my decision and never regretted it. That company no longer exists since it lost all of its customers a short time after I left.

Money is a tricky thing. It should never be placed in the hands of fools, because they will always do foolish things with it.

It's about more than just the Economics of the business though. "Time to Market" can have a BIG impact on the price of the Shares in your company (if it is a Publicly traded organisation). You don't meet the release deadline for a particular Quarter >> share prices drop >> No bonuses this year! In fact.. there might even be "restructuring" in the near future.

To do a great job, you may need more people, tools, training or equipment, but those things all cost money. You may or may not have money "in the budget" for these things. Getting those things may come at the expense of a lower salary or absence of financial bonuses. Not having those things may mean that you will miss the important deadline and cause your business to miss its Sales goals... which in turn mean lower salaries, no financial bonuses, or even layoffs.

Money. How do you want it?

Scope

It all starts innocently enough with the customer. This "customer" may take different forms depending on the industry you are working in.

You start with a list of features. These features are deemed to be the critical success indicators for the software if it is to be bought and used.

Software Development is a complex and iterative process whereby some really smart people take that feature list and break them down into hardware and software components from a particular technology. Suddenly that list of ten or so items, is now broken down into hundreds of smaller pieces that all need to work together in harmony to make up the features that the customers want.

Think of it this way: In Ancient Egypt, a Pharaoh once said: "Build me a Pyramid! I want a tomb of magnificent proportions that will leave a legacy for eons to come!" The Architects designed it, and then the Engineers built it.

Have you ever looked closely at a Pyramid? It's made up of thousands of stone cubes! Where did these stones come from? How did they get there? How did they put them together to make the finished product? What about the workers? How did they feed and clean up after thousands of slaves?

Thinking about some of these questions will begin to give you an idea of the orchestra at work in a software company to produce a product that the customer needs and will pay for.

But then a funny thing happens. Someone looks at the software. It might be a Product Manager, a Software Tester, or even the customer. And they discover something that isn't quite right. Sure, perhaps the big, important features are there, but the little details between them might be clunky, incomplete, or too complicated to make sense.

Uh oh. We need to make changes to the software.

And we keep "testing". And we keep discovering more "little things". And before you know it, that "critical list of ten features" that was broken down into hundreds of components is now tied together with thousands of little details!

It takes a keen mind to be able to keep all of these details in perspective of "the big picture". That is, does it make sense to make a particular change now, or should we put it off until the next release or version of the software?

I've met many testers who believe that every bug they find should be fixed right away. That's a foolish and narrow-minded perspective, in my opinion. I believe that if the problem is worth reporting, it should be worth fixing (whenever feasible), but each problem needs to be assessed for the risks involved and the priority or relevance for that particular release.

For example, I once tested a client-server application and found a bug that crashed both the client and server at the same time. (This is unusual because usually you only find bugs that affect one or the other.) In the Project Status Update meeting that week, I argued that we should NOT bother fixing the bug for that release. That is, even though it was a serious crash, I knew that no customer had ever reported the problem in the five years that the software was on the market. This led me to conclude that no one was in fact actually using that feature. So, I could only conclude that it would be a waste of time to fix a feature that no one was actually using.

Being able to keep things in perspective like this is what helps you to avoid one of the strongest Failure-factors in the business: "Scope Creep." Changing Requirements.

It's just a game sometimes. I've seen it many times. It goes something like this..

You ask the customer what they need to do their jobs. They tell you and you make the list. You come up with the requirements and double-check them with the customer to make sure it looks right. You make the software, work out the bugs, and release it to the customer for approval.

Now that the customer has the working software in front of them, the bar has been raised. "Oh well, it almost does what we need. It just needs these many more features, and then it will be perfect."

The smile falls from your face, but you go back and work to get these new features into the product. When you bring the software back to the customer, the bar has been raised again! "Well, it's almost perfect, but it still needs these features for it to actually work for us with the way we do things."

Just for the record, these are the kinds of customers that cause software companies to go out of business. More importantly, the Project Managers who don't recognise this situation and allow these scenarios to play out late in the development cycle are really the ones who cause the software companies to fail.

"Scope Creep" or "Feature Creep" simply refers to the changing of the requirements while the product is still in development. What may seem like a straightforward list of requirements at the outset may suddenly become conflicting requirements by the time you expect to release the software.

Changes may be subtle and sneak up on you in inconspicuous ways, or they may just hit you in the face like a closing door when you are looking the wrong way.

Getting a handle on Scope requires you to understand: (1) the Development Models (Life Cycles) used to provide the guidance in the way you build the software, (2) the customer and their appropriateness to help you achieve your desired results, and (3) the Requirements gathering process and the iterative nature required to truly flesh out the useful features from all the unnecessary "Bells and Whistles."

Scope. Feature list. You think you know what you need to do. You will be wrong every time. Software Development is a process whereby you usually engage on a journey with nothing more than a general sense of direction but no clear understanding of the final destination. That's the way it is.

Go to the ocean and learn to surf. Things will become clearer.