tag:blogger.com,1999:blog-9421494.post1833729622626896615..comments2024-03-27T06:18:38.214-04:00Comments on Lessons Learned by a Software Tester: Test Management is WrongPaulhttp://www.blogger.com/profile/16826575269870573990noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-9421494.post-29814022895193077662016-03-04T23:41:06.308-05:002016-03-04T23:41:06.308-05:00Perhaps. I believe that people and their mindset m...Perhaps. I believe that people and their mindset matter more than processes and tools. We can come up with the right processes and tools we need if we have the right people collaborating with the right mindset. You can build high quality things without a single documented process. And I'm not talking about a Cowboy/Hero culture either.<br /><br />The vast majority of current IT/Development cultures are broken. Test processes exist as a kludge/band-aid solution to the broken relationships and culture. If we fix the culture we shouldn't need those processes anymore and our tools will be different.Paulhttps://www.blogger.com/profile/16826575269870573990noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-18271648349822284382014-02-10T00:11:51.796-05:002014-02-10T00:11:51.796-05:00A very interesting article on testing management. ...A very interesting article on testing management. Thank you for posting. Zora Ferrelhttp://www.cloudstaff.com/noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-43395377655635367282013-04-22T08:37:13.718-04:002013-04-22T08:37:13.718-04:00Handling the actions of evaluators operating like ...Handling the actions of evaluators operating like that has very little to do with examining. The supervisors in such organizations often don't really get examining. They befuddle the procedure with the truth, seeing examining as a procedure that must be followed, and checked off on the venture strategy before the program goes stay. Project supervisors don't want analyze supervisors to handle examining, they want them to handle the examining procedure so that the venture can be monitored nicely through the MS Project strategy.<br /><br /><br /><a href="http://arcmail.com/" rel="nofollow">Email compliance</a><br /><br />Anonymoushttps://www.blogger.com/profile/04852462588609018388noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-39895520803405607002013-03-12T14:13:28.068-04:002013-03-12T14:13:28.068-04:00I believe it's all about balance. What Mohinde...I believe it's all about balance. What Mohinder says about measurements being the illusion of control and knowledge is an interesting point. Measurements are part of the picture. Feelings, instinct and intuition are other parts of the picture that no ALM tool will capture. But to be truly knowledgeable and in control you have to take and analyse data from many different sources. <br /> <br />I like the analogy of the car. You could be one of the best drivers in the world. Yet in different conditions, different roads and even different cars you're still going to look at the dashboard to check the instruments and confirm that you're driving within the speed limit. Yet conversely if you're always looking at the dashboard and monitoring your speed to make sure you're bang on the speed limit you could well be ignoring other obvious factors like the road being wet (which isn't measured on the dashboard). <br /> <br />Ultimately as a test manager you want to be taking information from many sources. One of those sources may well be a test management tool. Yet you better not focus on that completely at the expense of indicators like instinct and intuition. Some of the best developers I've worked with talk about code not feeling right. And some of the best testers I've worked with have the feel for identifying bugs. All part of getting the right balance to ensure a successful project. William Echlinhttps://www.blogger.com/profile/09205065711550894955noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-25333779393968446872013-02-28T05:01:24.339-05:002013-02-28T05:01:24.339-05:00So we're all pretty much in agreement that tes...So we're all pretty much in agreement that testing and quality assurance are not the same thing and that test management isn't counting test run and defects raised. Tom du Préhttp://www.hoinick.comnoreply@blogger.comtag:blogger.com,1999:blog-9421494.post-37912163402553204672013-02-26T09:50:21.269-05:002013-02-26T09:50:21.269-05:00Thanks for the follow up Paul. I think we're p...Thanks for the follow up Paul. I think we're pretty much in agreement. James Christiehttp://clarotesting.wordpress.com/noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-82700811397144679652013-02-25T18:58:12.734-05:002013-02-25T18:58:12.734-05:00James & James,
I am neither attacking Testin...James & James, <br /><br />I am neither attacking Testing, nor management in general. Management is an important role/function within a company as it (done well) helps people to work together better. This is important if the team is to deliver something of value to their customers.<br /><br />The main idea I am trying to get across here is the false 'best practice' of <b>using testing as an indicator of customer value.</b> This is about more than invalid/misleading metrics like test case counts. Sometimes stakeholders (or even customers) may demand to see test cases as a demonstration of good quality - more false indicators. (I wrote about such a time in my Tea Time with Testers article last April.)<br /><br />Let's look at the big picture and what everyone on the team contributes to Quality. There is no Test Management system that can provide all this info, although it should contribute to part of the picture (assuming the testing is done well and also contributes something of value).<br /><br />This is a synthesis problem. I have a thought from Russell Ackoff in my head that may express it better, but I currently lack the exact words.<br /><br />So, no, I am not against teams using some system/tool to help communicate and collaborate within their team. An ALM tool vendor that packages a Test Case Management tool as a "Quality" Management piece is doing it wrong. They are doing their clients and our industry a disservice.<br /><br />Saying that you will assure Quality through [only] Test Management is wrong.Paulhttps://www.blogger.com/profile/16826575269870573990noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-19094643702532334932013-02-25T05:06:38.315-05:002013-02-25T05:06:38.315-05:00I prefer to use the term Test not QA as the latter...I prefer to use the term Test not QA as the latter indicates that test are there to verify a products quality, which we most definitely are not. We verify that the product works for the tests we have performed and that is all.<br /><br />Some of this post I agree with, but you still need to have a process that is managed, but managed in away that doesn't hinder the process, no processes for processes sake.<br /><br />Using Test Case Management Tools are essential in my opinion. Without them how can you know what's been tested and what hasn't, especially when you have a large team all testing the same product, how do you prioritise what needs to be tested first.<br /><br />Yes I think its very true that this shouldn't be used to give metrics on Quality or coverage, but do need to be used to efficiently manage the testing phase.<br /><br />I've seen mind maps used, but with the products we test these really wouldn't fit the bill due to the massive amounts of functionality that has to be tested.<br /><br />Stephen Blowerhttp://www.stephenblower.co.uknoreply@blogger.comtag:blogger.com,1999:blog-9421494.post-83407505614958401012013-02-25T04:53:50.908-05:002013-02-25T04:53:50.908-05:00I think I agree with what you're saying, but I...I think I agree with what you're saying, but I'm not sure I agree with the way you've expressed it. <br /><br />Management per se is not bad. It can be done well or badly, tightly or loosely; managers can inspire or demoralise. There always needs to be some form of management to see the big picture; to coach, direct and lead testing by personal example; to report, co-ordinate and assign work; to deal with other managers and stakeholders; to protect the testers from crap so that they can be productive. None of that is bad, and it is all test management.<br /><br />We need to be clear about the distinction between managing testing and managing the testing process. It's a vital distinction that applies equally to other disciplines in software development. The difference is that it's possible to blur the difference where testing is concerned. Managers can spend all their time managing the process but doing nothing that could honestly be called testing. That's maybe because testing is poorly defined and understood. Sure, the thought leaders of testing are crystal clear about what they are doing, but do they influence a majority of testers? I'm not sure. They certainly influence those who are listening and whose minds are open. But there are huge regiments of testers working away in traditional environments, drawing up detailed plans and scripts, then plodding their way through them in test execution.<br /><br />Managing the activities of testers working like that has very little to do with testing. The managers in such companies often don't really get testing. They confuse the process with the reality, seeing testing as a process that must be followed, and ticked off on the project plan before the application goes live. Project managers don't want test managers to manage testing, they want them to manage the testing process so that the project can be tracked neatly through the MS Project plan.<br /><br />Sadly testing is an activity which it is possible to totally fake (as James Bach memorably put it). Test managers can run through the process entirely plausibly, without any <b>real</b> testing. I find it hard to imagine that happening with other IT disciplines. The folly would be immediately apparent. That's why we need to be clear about what test management is, and angry when managing the testing is confused with managing the process.<br /><br />Phil Kirkham is right. I too have known that feeling of futility and despair when I realised that I was adding nothing to the real testing even though I was the test manager. I was spending most of my time producing reports for people who didn't really need them, didn't understand them and certainly didn't understand the reality. If the reports had been scrapped all that would have been missed was the chance to tick the box "report produced".<br /><br />I had a great discussion with Fiona Charles about this a couple of years ago at the Test Management Summit in London. Fiona worked it up into a marvellous talk for STPCon. http://www.quality-intelligence.com/documents/Are%20You%20Managing%20Testing%20or%20The%20Test%20Process_Fiona%20Charles.pdfJames Christiehttp://clarotesting.wordpress.com/noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-21645661556351932812013-02-24T16:06:11.238-05:002013-02-24T16:06:11.238-05:00This is, BTW, essentially the same thing I argued ...This is, BTW, essentially the same thing I argued in my keynote at STPCon in October 2011, "Are you managing testing - or The Test Process". (also at KWSQA just before)<br /><br />The more of us who get out there and say this -- in blogs like this, in mainstream publications and at mainstream conferences -- the greater the possibility that we can begin to make a difference and discredit these pernicious practices.Fiona Charleshttps://www.blogger.com/profile/05261957091656214838noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-20050135730245061452013-02-24T15:50:47.243-05:002013-02-24T15:50:47.243-05:00Interesting post, Paul. I'm not sure your titl...Interesting post, Paul. I'm not sure your title really reflects your argument, though. There's a difference between "Test Management" and managing testing (at least there should be!) and that's not clear in your title.<br /><br />"Test Management" as you define it here is all about so-called processes and systems, human or otherwise, that go under the guise of test management. I agree with you that these are superfluous and actually detrimental to delivering value. So, I would argue, is a lot of what is called Project Management -- a bunch of PMI-propagated systems that are more about controlling cost than delivering value, which often gets lost in the morass of reports and control mechanisms. (A couple of good articles about this are Tom DeMarco's "Software Engineering:<br />An Idea Whose Time Has Come and Gone?", and Sylvain Lenfle/Christopher Loch's "Lost Roots:<br />How Project Management Came to Emphasize<br />Control Over Flexibility and Novelty".) <br /><br />I do believe, though, that there is a place for managing testing, as there is for managing projects.Fiona Charleshttps://www.blogger.com/profile/05261957091656214838noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-68320374795436774952013-02-24T15:03:39.323-05:002013-02-24T15:03:39.323-05:00I read your post and I'm left with little idea...I read your post and I'm left with little idea what you mean by test management. In part of your post you seem to be attacking the very idea of testing, in another part you seem to be attacking the very idea of managing.<br /><br />The word "management" means many things. One of the problems I keep seeing in Agile projects is the lack of management-- by which I mean time spent getting the infrastructure and agreements together by which we can get the work done, then maintaining the status of those things over time. This is not a huge deal on tiny projects. But for testing, on a project of any significant size, I keep seeing testers with no coherent test strategy under too much pressure to think about tomorrow.<br /><br />Shallow testing does not require foresight and preparation. But deep testing does, and that's where you will find it necessary to have activities recognizable as management.James Marcus Bachhttps://www.blogger.com/profile/09985950531079499844noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-87359246991193206382013-02-24T11:41:31.391-05:002013-02-24T11:41:31.391-05:00I must say you hit the nail on the head and expose...I must say you hit the nail on the head and exposed the truth about TM. Still it does not hide the fact that many managers wrongly believe ie ‘what can’t be measured can’t be controlled’. Measurements are the illusion of control and knowledge. If a tool is lying then where is the truth? If you cannot trust the people on the ground then a tool changes nothing. TM or ALM tools put you in straight jacket that works against the way we do testing or perform any other activity within a project. It cannot capture human interaction with the software which has many facets such as bias, emotions and behaviour that effect our testing. There is no room to record activities outside testing that testers are dragged into. If it is a tool to measure progress then it fails in providing feedback and improvement, comparing results release by release, links to improvements by activities and vital intelligence to offer better way of doing things. It is no brainer at the end of the day that relies heavily on human intelligence to attach validity to results. Software development is a social science that is outside the scope of TM/ALM toolset and dependency on them doomed to failure.<br />We are all exposed to quality syndrome to some extent and made responsible for it. Don Reinertsen (http://www.amazon.co.uk/Principles-Product-Development-Flow-Generation/dp/1935401009/ref=sr_1_1?s=books&ie=UTF8&qid=1361719696&sr=1-1) said, ‘Watch the product work not worker work’. As Mary Poppendieck said in one of her quotes, ‘quality’ is not the responsibility of a tester or a developer but of the system. A system thinker may be one who is responsibility for value-mapping with better view of the system and value it should deliver. In system’s thinking, a system consists of everyone that interacts with the system directly or indirectly, human or non-human. Surely, we as tester hardly ever know the full list of stakeholders and value software deliver to each. Even if we know then how a test management tool would record those attributes. Impact Mapping has changed the concept from stakeholder to actors/players for European benefits who have problem using stakeholders. How would TM tool handle that change?<br />Jerry Weinberger (http://secretsofconsulting.blogspot.ca/2012/09/agile-and-definition-of-quality.html) definition of quality is as good a definition as any other you can find that has stood the test of time. How do you, through your testing, break down the value for each stakeholder and record it for everyone to see? A Kanban board or post-it notes provides better visibility and status than a TM tool would ever do. One twitter message recently said it all ‘Jira does not speak for itself that post-it note would do. Low tech dashboard (http://www.satisfice.com/presentations/dashboard.pdf) is more effective in delivering visibility than cumbersome TM tools from major vendors would do. <br />Unfortunately, I plan to attend two one-day ALM events this and next month from HP and other vendors to find out more what’s on offer and how can they improve SDLC.<br />Mohinderhttps://www.blogger.com/profile/03621573389986589766noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-40847859634567709502013-02-24T10:21:11.182-05:002013-02-24T10:21:11.182-05:00but but but how will we know the project is on tra...but but but how will we know the project is on track if we dont know what %ge of test cases have been written, run, passed, failed ? How can we move through the Quality Gate to the next phase if we don't know that we have 100% requirement coverage ? Do I still need to produce a RACI matrix ? How are we going to decide how many test cycles to run ?<br /><br />I've worked on projects where the 'test managers' never used the product, had no idea what the customers would do with it - and had no time to train their testers as they were too busy running reports on how testing was going<br /><br />Like Lisa, I'm also reading Impact Mapping and working towards getting a better understanding of what the customers want and value<br /><br />great post<br /><br />Phil Kirkhamhttps://www.blogger.com/profile/18008391232663423821noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-62861763518326154362013-02-24T09:56:39.778-05:002013-02-24T09:56:39.778-05:00A long time ago, in the 1970's I believe, some...A long time ago, in the 1970's I believe, someone decided to define testing as an emotionless, mechanical exercise. It was also decided at some point that testing should be called 'quality assurance' because it seemed related to the QA processes in manufacturing.<br /><br />We as testers have been living with the consequences of this ever since. It is very easy for upper management to look at a department labeled 'quality' and assume they provide it and are responsible for it, hence the existence of all of the 'quality management' suites.<br /><br />Anyone in testing, and especially anyone who has a testing group reporting up ton them should read this.Erikhttp://testingthoughts.com/erikdavis/noreply@blogger.comtag:blogger.com,1999:blog-9421494.post-17754825711935611592013-02-24T04:41:03.095-05:002013-02-24T04:41:03.095-05:00This fits in so well IMO with Gojko Adzic's ne...<br />This fits in so well IMO with Gojko Adzic's new book on Impact Mapping. Too many business experts ask us for software features without really knowing what value those will bring for the biz. We should help them understand the "why" first. What brings biz value may or may not be a s/w feature. Lisahttps://www.blogger.com/profile/10230090963033880060noreply@blogger.com