I've been thinking a lot about this phrase lately: "Your thoughts shape your reality." I think it's from a Buddhist teaching but I'm not sure because I read it as a second-hand quote somewhere that I can't recall right now.
I'm also familiar with the tester's lament: "hey, I didn't put the bug in the code, I just report it."
But what about the phrase: "if a tree falls in the forest and no one is around to hear it, does it make a sound?" That one has always annoyed me. The way I see it, the answer to the question depends mostly on how you define what a sound is. If a sound requires a listener to be real, then, no, no sound was made.
What about a bug? If there is a bug in the software but no one encounters it, is there really a bug in the software?
Now, I report bugs. A lot of bugs. I employ many creative and imaginative tricks to find them. They love me. They come to me even when I don't expect it. One of the tricks I use is to change my perspective on how I look at software. I also exploit vague areas, or areas of doubt within the development team, development processes or methods of communication. For example, if it looks like there might be some vague statement in a specification that could be confusing or interpreted in different ways, I'm pretty confident there will be lots of bugs there.
But hold on. What if I don't go looking for the bugs? What if I don't report them and no customer ever sees the problem or complains about it? Does the bug really exist?
Or does the mere act of identifying this discrepancy between intention and execution create the entity that I call a bug? Did I create it or was it already there?
Scenario: A spec-writer wrote a document that was interpreted in some way by one or more programmers. According to their interpretation, the implementation (the software) is working as intended. They check that their code measures up to their understanding (unit testing?) and then build the software into a deliverable product.
Then along comes a tester. A tester looks at it in a completely different way and suddenly the software is full of bugs. But wait a minute! The bugs weren't there until someone looked and said they were there.
This reminds me of Schrödinger's cat. Is the cat alive? Is it dead? Is it both alive and dead, or neither? You have to look to find out, but once you've looked you've changed everything. What a psych trip!
So, what if finding bugs falls into the same category? In that case, the mere act of observing has changed the reality of what's in the box. By looking inside and asking the question, we have shaped the reality by our thoughts.
Doesn't that mean that we have therefore created the bugs that we report? The programmers didn't put the bugs there. They interpreted the specs and design documents according to their thoughts and shaped the software according to their reality. They built no bugs.
The bugs were called into existence when we, the testers, said they were there with our mind tricks and clever tools.
The bug report, therefore, is just a record of how we have reshaped reality with our thoughts in a way that is different from how reality looked before we started testing. We created those bugs in the bug reports.
One might argue, then, that if you make the bug reports go away you make the bugs go away. If no one (customer) sees the bug, does it really exist?
That's a change in perspective from what I was taught about testing. I don't report bugs. I create them!
Hi Paul. How have you been? Ok? Great.
ReplyDeleteI was reading your interesting text about testers creating bugs. First of all I think we should clear that tree falling in the forest issue. According to the wikipedia a sound "is vibration transmitted through an elastic solid, liquid or gas, composed of frequencies capable of being detected by organs of hearing.". This means that, unless the laws of physics change, if the tree falls there will be sound.
Ok, so now that we cleared that one, we can move on to the "testers creating bugs" issue. There is no doubt in my mind that testers absolutely do not create bugs. Unless they sabotage the code or the requirements, or they are acting in bad faith effectively calling something a defect when it is not. Or maybe the tester plain and simply understood it wrong. That happens too.
In your example you say that the programmer interpreted the requirements in some way. And apparently that was not the correct way. There are several reasons for this.
1. The requirement was ambiguous or incomplete. If this is the case, then there is a defect in the requirement. It should be removed from the requirement and the code corrected accordingly.
2. The requirement is fine, and the programmer misread it. Tough, there's a defect in the code. Correct the code.
3. The business specialist didn't explain himself correctly or the analyst didn't understand him. Defect in the requirements, follow recipe in 1.
Can't really see how was it that suddenly the tester created a bug.
Cheers
Paulo
Hello Paulo, thank you for your reply. Part of me agrees with almost everything you've said.
ReplyDeletePart of me is still thinking about the main idea I was trying to get across in my original post, though.
First: the tree falling. I studied Physics in school and my initial thought is to agree with both you and Wikipedia. However, that only represents one, Scientific, perspective, and not necessarily the same perspective that everyone else will have. The purpose of this example is because (I believe) it is a non-software example of the same idea I'm trying to get across in the rest of the post.
To illustrate my point here, don't debate with me in a blog about it (because that won't probably be as much fun as the alternative..). Ask your friends or family that question the next time you see them and see what they think. Don't give your answer first. Just listen to their opinions. Instead of correcting anyone's opinions, just ask them why they think that way. The question can always be answered easily when you only have one opinion to consider -- your own. Not everyone will necessarily agree with Wikipedia either.
Next, regarding your 3 possible answers to the remaining issue of who creates a software bug. I will divide your answers into 2 categories: (a) the programmer made a mistake of some kind, and (b) there was a misinterpretation in or of the requirements somewhere.
The first one, (a), is trivial I think. I agree with you that it is the programmer's bug and you are simply helping them realise it. Out of curiosity, do you know roughly what percentage of your overall bug count this represents? (BTW, I don't know the answer. I think the answer might change for every project, although it might be an interesting question to try and answer.) Based on my experience, for the projects that I have worked on over the last decade, these bugs don't represent > 50% of all the bugs reported on a project. So they are not the majority of the bugs found on the projects that I have participated in.
As for the (b) responses, I think all requirements are vague, ambiguous or incomplete in some way to some person. That's just the nature of requirements. I don't believe it's possible, or economically prudent to invest the time and energy, to make them 'perfect' (i.e. correct, unmistakable and clear to all readers).
So if requirements are always vague in some way, it comes down to interpretation by the reader. As a tester, we change our interpretations by wearing different 'hats'. This is otherwise known as applying different test techniques.
Here's the important part: the requirements don't change; the code doesn't change; the bugs appear because your perspective on them changes. You change your frame of reference and you see things in a different way. Sometimes this leads to more bugs; sometimes it doesn't.
This is an important part of what we do as Software Testers. We change our perspectives. We change our frame of references. And to get bugs fixed, we try to change the opinions/minds of those who matter to see these new perspectives as important enough to be worth spending time, money and effort to 'fix' them.
One might argue that these 'bugs' already exist in the 'solution' or 'product' (by which I am referring to more than just the software -- include the whole system and requirements too). So, in that case, we don't create the bugs.. they're already present.
Perhaps. That's one perspective.
We change our perspectives because we believe that the perspectives are valid when we find a bug. Are all bugs valid though? That is, do they always represent a valid perspective of someone who matters to the development of the project?
Are they still bugs if the perspectives are not correct for a given project?
I leave it to you to think about it.
Thanks again for your terrific reply!
Ciao! Paulo. =)
Hmmm, ... I don't know, I won't say you are wrong, it's just your way of looking at things. To me it seems more like the case of the person who is used to drive through a certain course and one day he just seats on the passenger seat and is driven instead. He then goes "oh my, that's such a lovely scenery. I had never noticed it before". But the scenery was there all along.
ReplyDeleteI do agree with you when you say that requirements will never be perfect.
Anyway, it was fine chatting with you. I'll probably visit more often. Have fun!
"if a tree falls in the forest and no one is around to hear it, does it make a sound?"
ReplyDeleteOh, it did make sound. I was there but maybe no one could see me from far enough distance or through imaging systems because it was a dense forest.
It was so dense that I heard a "THUD" near me and when I turned, I saw a tree lying. I conjecture from what I see and heard that it could be a tree that caused the "THUD".
I see a bug and create a story of the bug. I am a character in that story. I came in when too many things were happening. As a human, I was unable to notice all things that were happening so I conjecture from what I see, hear and feel that this is what could have happened.
One might argue, then, that if you make the bug reports go away you make the bugs go away. If no one (customer) sees the bug, does it really exist?
If no one to our knowledge see a bug and report them or talk about it near our ears, we don't hear about the bug. Hence we spend our time on things that we have heard ( because there might be too many things we heard or suspect ).
Not many people might click "Send Report" whenever a Microsoft Application or OS crashes. So, if I were sitting in the team that receives all this report and I see only three people having accepted to send a report for MS Word crash think "It just crashed thrice out of millions of customers"?
We ( as customers ) ignore bugs sometimes ( unless it bothers us too much ). We ( as testers ) are bothered about customers being bothered.
We ( testers ) don't know or have little information about what might bother a customer and try flushing out as many bugs as possible.
This forms a vital information about the quality of the product.
I think your first question you should ask yourself is "what is a bug"? Ask, your co-workers what their definition of a "Bug" is, and just listen. You have your opinion of what a "Bug" is, you can even surf the web for many opinions on the definition of a "Bug". I think this is the key question that needs to be asked before any arguments can be valid for either side. I know my definition of a "Bug", and from that this is my opinion, in fact I do not call them "Bugs", I call them issues, because I am reporting what I observe. It may, or may not be a "bug" in the software, my observed result may be what was supposed to happen. It may not even be the projects software that is causing the issue, it could be settings or a specific setup I have on my test machine.
ReplyDeleteTesters most definitely do create bugs though , but not always. In fact both points have truth. But a piece of software always has a scope. It is always developed to some general standards, sometimes very strict standards. The question the tester asks is, "is this issue within the scope of the project?". There is far to much hardware in the world to make a piece of software run perfectly on everything. Do you happen to possess the correct setup to see an issue that 99.99% of other users may not. Sure you can change your perspective on how to use the software not intended by the developers, which would obviously create issues. In this regard yes, testers do create bugs, this is especially seen in game testing. Using the tools given in a game environment, testers will always do things not even though of by the developers.
On the flip side, developers also make bugs. Of course they are not intentional, its not as though they woke up in the morning and said, ok I am going to place 9 bugs in my snippet of code today. We are only human, we make mistakes. No matter how good the developer thinks they are, they still make mistakes. The larger and more complex the code has to be, the more accidental issues are likely to be introduced. Then there is the merging of snippets of code. No two people code the same, so when multiple people merge code it is up to a few coders to get all of those snippets of code working together. This can be just as hard as getting people to work together.
Instead of asking yourself "If a tree fell in the forest, does it make a sound?", ask "What caused the tree to fall in the first place?".
Hello ibeta. Thanks for your comment.
ReplyDeleteibeta wrote:
"Instead of asking yourself "If a tree fell in the forest, does it make a sound?", ask "What caused the tree to fall in the first place?""
I can't answer that! That's a "Quality Assurance" question, not a "Software Tester" one! ;)
I take a passing interest in what the cause is, but that wasn't the point of my original post. I think following that line of thinking will lead to a whole new set of ideas... :)
Cheers! P.