One of my favourite television series was called Babylon 5. Good stuff. From time to time I recall particularly well-written scenes and moments in that series that seem to apply well to some situation happening in my life. That happened again today.
I had a meeting today that changed everything. I've spent the last decade searching for answers that apparently weren't there. Today I found a well, no a sea, of good answers. It's going to take me a while to assimilate everything and find a way to communicate it back to people. It's not the right piece of information, it's the right way of looking at things.
I said to someone this afternoon that this is the first time in 9 years that I've really learned something NEW. Something that makes me think. Something that makes me think in a new way.
For the first time in almost a decade, the phrase that comes to mind is "I am the arrow." I know where I'm going and what I need to do.
The context for that phrase is from an episode in Babylon 5 - Season 3, episode 61: "War Without End (Part Two)" There was this scene where Sinclair is absolutely certain he knows what is going to happen (in his future which is really everyone else's past - Time paradox stuff) and what he needs to do. That scene has always haunted me. It's not every day that you know with such certainty exactly what you need to do.
In all fairness, that has happened to me once before. It was the day that I started the relationship with my wife -- that was almost 20 years ago. I had no doubt. I was absolutely certain.
Unfortunately, moments like that don't come around very often. I needed to put a little note here to mark the occasion that it happened to me again today. It feels good for a change - to know the solution to a decade-long problem.
Happy St. Patrick's Day! =)
Ross Collard's Tea-Test (Technique)
I was reviewing the test notes for one of my testers today when I came across an interesting note. I decided to look up the problem report in the bug tracking system to read the details. The bug report said that if you go to particular new page in a web app (currently in dev't) and enter some information, wait 35 minutes, and then press a button to continue, you get an error. If you wait less than 30 minutes there is no error, and if you wait over 60 minutes then the application will timeout (as expected), so you have to wait just the right amount of time.
Suddenly I started laughing out loud saying: "Hey, it's the Tea Test!"
I called over the tester whose notes I was reviewing to thank him for his good work in finding, isolating and reporting the bug and to tell him this story. You see, there are many kinds of test techniques out there. At the very least, most programmers and testers have heard about BVA and Equivalence Classes, and some testers who take an interest in their profession learn about other techniques as well. In our test team, we can rattle off at least a dozen techniques at any given time and we are usually selecting from among a pool of 30 or more techniques on any given project, but that's not important right now.
What was important at this moment was that the technique that I could apply to the bug found wasn't on any list I had ever seen, but it was a technique that I knew about.
Back in the summer of 2003, I drove down to Virginia to attend a special 5-day "Black-Box Software Testing" workshop offered by both Cem Kaner and James Bach. It was a great opportunity and I didn't want to miss it. Much to my surprise and delight, Ross Collard had also come to attend the course. Ross had taught me my first courses in Test Case Design and Test Management some 5 years prior, and it is information I still use to this day.
One day during the BBST workshop Cem and James asked the participants to name some test techniques. Ross offered two that I hadn't heard before. The first was the "shoe test". That's where you take off your shoe and put it on the keyboard. Then you wait to see how the app handles the non-stop input.
I've seen this technique happen in real life. I've seen someone lean back against their desk while talking to others not realising that they were leaning on the keyboard. Another time, I saw someone put a magazine on their desk which accidentally landed partly on the keyboard and proceeded to cause a beeping noise from the computer as the keyboard input cache filled up.
This can be an interesting technique as you ponder which key on the keyboard to place your 'shoe' for maximum effect in the App Under Test. For example, how well do you think your web app can handle you pressing [F5] to refresh the page non-stop? Can you find a web page that makes several database calls and then try [F5] repeatedly again? Think you can bring down a database server by doing this? [evil grin]
The second technique Ross mentioned was the "tea test". I hadn't heard of that one before, so I asked if the "T-test" was related to the Statistics t-test. He said "no, it wasn't." He said that what he would do here is enter some input in an app, get up from the desk, walk over to make a cup of tea, have the tea, walk back and enter the next input. And he punctuated this by saying that since he doesn't walk very fast these days, this process could take anywhere from 30-40 minutes. Ha! That was funny. I hadn't heard of any test technique like that before.
Fast forward to today. That was exactly the amount of time (30-40 mins) that my tester had to wait for the bug to appear! The tester told me how he had spent over an hour trying to reproduce that bug in a background VMWare session, so that he could continue with other testing while waiting for the right amount of time to pass. We both laughed at how the "Tea test" applied here.
The developer assigned to fix the bug (who sits on the other side of the desk partition from me) must have overheard me telling this story. He piped up and said: "I hate that bug! I have to wait a long time to try and reproduce it!" We laughed harder. =D
This was the first time I've seen Ross' "Tea test" actually work -- i.e. actually find a bug. I thought it was just a joke at the time. I now know there's truth in that technique. It's not that I really doubted Ross, it's just that he's a funny guy and sometimes you can't tell when he's pulling your leg. =)
Ross, you really are the Test Master. Next time I see you, the tea is on me. Cheers!
Suddenly I started laughing out loud saying: "Hey, it's the Tea Test!"
I called over the tester whose notes I was reviewing to thank him for his good work in finding, isolating and reporting the bug and to tell him this story. You see, there are many kinds of test techniques out there. At the very least, most programmers and testers have heard about BVA and Equivalence Classes, and some testers who take an interest in their profession learn about other techniques as well. In our test team, we can rattle off at least a dozen techniques at any given time and we are usually selecting from among a pool of 30 or more techniques on any given project, but that's not important right now.
What was important at this moment was that the technique that I could apply to the bug found wasn't on any list I had ever seen, but it was a technique that I knew about.
Back in the summer of 2003, I drove down to Virginia to attend a special 5-day "Black-Box Software Testing" workshop offered by both Cem Kaner and James Bach. It was a great opportunity and I didn't want to miss it. Much to my surprise and delight, Ross Collard had also come to attend the course. Ross had taught me my first courses in Test Case Design and Test Management some 5 years prior, and it is information I still use to this day.
One day during the BBST workshop Cem and James asked the participants to name some test techniques. Ross offered two that I hadn't heard before. The first was the "shoe test". That's where you take off your shoe and put it on the keyboard. Then you wait to see how the app handles the non-stop input.
I've seen this technique happen in real life. I've seen someone lean back against their desk while talking to others not realising that they were leaning on the keyboard. Another time, I saw someone put a magazine on their desk which accidentally landed partly on the keyboard and proceeded to cause a beeping noise from the computer as the keyboard input cache filled up.
This can be an interesting technique as you ponder which key on the keyboard to place your 'shoe' for maximum effect in the App Under Test. For example, how well do you think your web app can handle you pressing [F5] to refresh the page non-stop? Can you find a web page that makes several database calls and then try [F5] repeatedly again? Think you can bring down a database server by doing this? [evil grin]
The second technique Ross mentioned was the "tea test". I hadn't heard of that one before, so I asked if the "T-test" was related to the Statistics t-test. He said "no, it wasn't." He said that what he would do here is enter some input in an app, get up from the desk, walk over to make a cup of tea, have the tea, walk back and enter the next input. And he punctuated this by saying that since he doesn't walk very fast these days, this process could take anywhere from 30-40 minutes. Ha! That was funny. I hadn't heard of any test technique like that before.
Fast forward to today. That was exactly the amount of time (30-40 mins) that my tester had to wait for the bug to appear! The tester told me how he had spent over an hour trying to reproduce that bug in a background VMWare session, so that he could continue with other testing while waiting for the right amount of time to pass. We both laughed at how the "Tea test" applied here.
The developer assigned to fix the bug (who sits on the other side of the desk partition from me) must have overheard me telling this story. He piped up and said: "I hate that bug! I have to wait a long time to try and reproduce it!" We laughed harder. =D
This was the first time I've seen Ross' "Tea test" actually work -- i.e. actually find a bug. I thought it was just a joke at the time. I now know there's truth in that technique. It's not that I really doubted Ross, it's just that he's a funny guy and sometimes you can't tell when he's pulling your leg. =)
Ross, you really are the Test Master. Next time I see you, the tea is on me. Cheers!
Subscribe to:
Posts (Atom)