Something Interesting about Reporting Bugs

I just happened by this link just now: http://cwe.mitre.org/ for the "Common Weakness Enumeration" (CWE) project.

Here's the blurb:
"International in scope and free for public use, CWE™ provides a unified, measurable set of software weaknesses that will enable more effective discussion, description, selection, and use of software security tools and services that can find these weaknesses in source code."

I'll have to look into this later. Don't know anything about it yet.

To be a Good Tester you must think like a Scientist

It's funny how many times this particular analogy keeps coming up. The comparison between Testers and Scientists, and the similarity between testing and the Scientific Method.

Most recently it occurred to me while reading one of my son's chapter books. FYI: "chapter" books are short books broken into chapters for kids just beginning to read on their own. Usually for kids ~ 7-8 years old. These aren't Harry Potter books.. most of them barely reach 80 pages.

The book that caught my attention is called "Jigsaw Jones #9: The Case of the Stinky Science Project" by James Preller. The main characters are in Grade 2 and in this particular story their teacher was giving a Science lesson:
"The world is full of mystery. Scientists try to discover the truth. They ask questions. They investigate. They try to learn facts. Scientists do this by using the scientific method."

The teacher then handed out sheets of paper which read:

THE SCIENTIFIC METHOD

1. Identify the problem. What do you want to know?
2. Gather information. What do you already know?
3. Make a prediction. What do you think will happen?
4. Test the prediction. Experiment!
5. Draw a conclusion based on what you learned. Why did the experiment work out the way it did?

Back when I used to teach High School Physics, I recall giving a set of steps very much like this one. I might have used the word "inferences" instead of "conclusion" but otherwise it's a pretty good list.

When you think about testing software, generally you run through the same process and set of questions. If you don't think about each of these questions, then you're probably not doing something right.

For example, here are some questions that come to mind when I think of the Scientific Method applied to testing software:

1. Identify the problem.
  • What are the risks?
  • What is the particular feature of interest?
  • What is it you want/need to 'test' and 'why'?
2. Gather information.
  • What references are around to tell you how something should work? (e.g. Online Help, manuals, specifications, requirements, standards, etc.)
  • What inferences can you deduce (or guess) about how something should work? (i.e. based on your experiences testing similar apps, or other parts of the same system, etc.)
  • What can you determine by asking other people? (e.g. customers, programmers, subject-matter experts, etc.)
3. Make a prediction.
  • Design your tests.
  • What is your hypothesis?
  • What are the expected results?
  • Think about any assumptions or biases that might influence what you observe. How can you compensate for these?
4. Test the prediction.
  • Set up the environment
  • Execute the tests
  • Be creative! Make as many observations as you can.
  • Collect data
5. Draw a conclusion based on what you learned.
  • Did you observe the expected result? Does this mean the test passed? Are you sure?
  • If the test didn't turn up the predicted result, does this mean the test failed? Are you sure?
  • Revise the test design and any assumptions based on what you observe.
  • Do you have a better understanding of the risks that drove the test in the first place?
  • Do you have any new questions or ideas of risks as a result of this test?
  • If you collect a lot of data, summarise it in a chart that can help demonstrate the trend or pattern of interest.
  • Write a few words to describe what these results mean to you. (You might not have all the information, but don't worry about that. Just say what you think it means.)

In general, I find the Scientific Method to be a very good guideline for both beginners and experienced testers alike. Wikipedia has some entries on the Scientific Method as well as a Portal. I think it's a good read. I'd recommend those pages to anyone serious about becoming a good tester.

If there are things on those pages that you aren't sure about, look them up! You might just learn something new about how to think about things that will help you do your job better.

Happy Learning!

Ubi Dubium, Ibi Occasio (Opportunitas).

Where there is doubt, there is opportunity.

That's my new motto as a Software Tester. =)

It came to me when I read a comic that I borrowed from a friend recently. You see, I'm a fan of the writer J.M. Straczynski, so my friend told me about a comic that JMS had written a few years ago called "Supreme Power" (Max Comics). If you've ever read a comic, you'll know that each issue or story usually has a separate title. Issue #8 of Supreme Power has the episode title "Ubi Dubium, Ibi Libertas" which he translates for you on the last page as: "Where there is doubt, there is freedom."

That title made sense in the context of the story, however I couldn't stop thinking about that phrase for several days afterwards. There was something about it that I liked, and yet it didn't completely fit for what I feel I do as a tester.

When there is doubt, I go to work. I have fun. I explore. I discuss and test. Most of software testing is about working in the space between the vagueness of specs or requirements and their ever-changing interpretation into working software code. So really, doubt is everywhere. Doubt is the whole thing! Where there's doubt, there's opportunity.

Over the years, I have often contemplated different analogies and ways of describing what I do as a software tester so that I could explain it to others who don't really understand the role. (For some reason, if you're not a programmer and if you're not doing Support or Sales, then most people don't really understand what else is there.)

So now I feel that I'm really close to a good analogy. Doubt is the space where I work and play in. I'm a Doubt Management Specialist or Facilitator, if you will. Someone writes up some specifications based upon what they think the customer wants to the best of their knowledge and understanding. (There's doubt.) Someone else interprets those requirements and transforms them into mathematical algorithms that perform some function on a computer. (There's more doubt. Is that like Doubt-squared? ;-) )

Enter the Software Tester, the go-between. We see the doubt in the specs and come up with some ideas (i.e. tests) to explore the meanings and possible interpretations. We see the doubt in the software as features are incomplete, don't perform as expected, are insecure in some way, unusable or not robust enough according to our interpretations and experiences as users of the technology.

If there was no doubt in the whole process, I don't think we'd have anything to do. We'd totally be out of jobs. Maybe we could be Project Managers or Programmers I suppose. ;-)

So you see, where there is doubt, there is opportunity for us. Opportunity to explore, to test, to ask questions, to find bugs, to strengthen understanding, to clarify, to add value.

If you don't see the doubt, I don't believe you are adding any value. At the end of the day, I believe the best testers are the ones who add value by reducing the doubt in the development project.

It's another way of looking at the problem. I kind of like it. What do you think?

Bonitatem et disciplinam et scientiam doce me (et gaudium ostende me)

There are a few things that I recall from my high school days -- the school motto was one of them: "teach me goodness, discipline and knowledge." It served me well as a guide over the years but I always felt like there was something missing in that motto alone.

Years later, when I was working as a high school teacher, the three main things that I focussed on getting across to my students were: Attitudes, Skills and Knowledge. That was pretty similar to my old high school motto, so it was easy for me to remember and apply. I thought those three things were a good guide for my classes, but again I felt like there was something a little too dry about it that I couldn't quite put my finger on.

One day, during a contract placement teaching one of my favourite topics - Physics - I got some interesting advice from a seasoned teacher on my teaching approach. He said that I did well in the class but that he lived by a different motto: "any class that goes by without some humour and laughter is an hour wasted of the students' lives." And of course he said it with a smile. =)

It's interesting because I have always had this joie de vivre that everyone who gets to know me notices. I like to smile. I love to make jokes. I love to stop and smell the roses and listen to the wind and the trees. I have a penchant for puns and when the going gets tough, I get silly. =)

Yes, I suppose there are times when seriousness is called for. However, for the most part, life is too wonderous and entertaining not to be silly and enjoy and to make other people smile and lighten up too.

Up until that moment when I got that feedback from that teacher, I had been working on a different model: one where you are serious in your professional life and save the humour for your personal life.

After that moment, I ditched the old "professional = serious" model and decided for myself that every hour worth living was worth enjoying, regardless of whether I was at work or at play.

Needless to say, after that advice I began to share my passion for the subjects I taught in the funnest ways I could think of while still adhering to my teaching goals and objectives. A decade later, if you have ever attended one of my QA or Software Testing workshops, you would also know that it is always with a certain amount of levity that I share my experiences and knowledge. After all, if you can't laugh at yourself, who can you laugh at, right?

At work I'm the same. Always the professional, and always with the smile on my face or joke to add. Don't get me wrong.. I'm not a clown every hour, but not a day goes by that I don't try to do or say something to add a little levity or have some fun.

Here's where it gets interesting. 8-)

Not everyone sees it the same way. Some people are simply "no fun at all." In fact, I even worked with someone once with the nickname "the Director of No Fun."

(Aside: Hmm, I wonder if it is a particular trait of middle-management to have the least amount of a sense of humour? Maybe it's because they realise that their jobs are the most expendible, so they tend to try to look and act important all the time?)

Oh, but it's not just people at work.. there are many people out there in the world who have simply forgotten what it is like to have fun when interacting with other human beings. To these people, adding a smile to a thought or reply suddenly takes on a different meaning. Now you are being facetious or patronizing or sarcastic. Your words are not only not meant to be taken seriously, but you are insulting as well. Nice.

Interesting. Personally, I find that these words tell me more about the listener than they do about the speaker. In fact, it tells me that the listener is not really trying to listen at all because they are too busy imposing their own negativity on the world around them thereby tainting everything that they hear.

I feel very sad for these people. It really is a shame that negativity appears to be more predominant in modern society than positivity. Acting and speaking in ways that emphasize goodness, joy and happiness generally makes you the odd one out.

(Aside: It's fun to meet other 'positive' people at parties and elsewhere. It really causes an exponential increase in positive and silly energy that just invigorates and excites me. I don't think that most people can handle having more than one of us around at any one time. =D )

Which brings me back to my old alma mater's motto: "Bonitatem et disciplinam et scientiam doce me." That's a good start, however, I'm also adding: "et gaudium ostende me." My Latin might be a bit rusty, but I'm pretty sure that translates as "and show me joy/happiness/fun."

It's not enough to teach and be taught the right attitude, the right information or the right skills. I want to see the fun that other people have in their work and I want others to see the fun that I have in mine.

If you happen to see me with a smile when you meet me, please feel free to smile back. =)

The Three Physical Requirements of a Good Software Tester

There are three physical elements that I find a good software tester must have:
  1. Good working senses
  2. Brain - ability to think
  3. Heart - someone who cares

Your senses (sight, sound, touch, etc.) give you the information that you need to process.

Your head helps you process the information and form them into ideas and models to work with.

Your heart gives the information meaning. Someone with heart is someone who cares about others and about the quality of their work. Without this, you are little more than a computer.

Interestingly enough, there are some people with reasonably good-working senses who are still unable to see. I think that it is likely an impediment from their head or their heart that prevents them from seeing. Can this be fixed? Perhaps -- if the person genuinely wants to see. Not everyone wants to see.

The problem is no longer a mechanical one but rather a psychological one. That's tricky.

All written communication is fundamentally flawed. It tries to capture some of the above 3 elements, but usually fails to really grasp the element of 'heart'. (And Poetry is likely the exact opposite - more heart than anything else.) I don't believe that any useful communication can take place without being physically present with the other person. There are many things said and understood between people that may be poorly or incorrectly inferred through written communication alone.

A software tester is an Information specialist. Do you have what it takes to be really good at it? Do you really care?