What’s wrong with software development? The Testing Mole
Disclaimer: I will be talking a lot about QA resources and testers. In my Agile team, I support the notion of engineers: people who can test or develop in any given iteration. If it sounds like I am supporting the notion of one person being a tester, it is only because the example story involves a team in which the supervisor believes in a sharp division between testers and developers. Also, I use QA as a broad umbrella to contain both QA and QC activities, even though more appropriately I’d take the time to use the terms specifically as applicable to their respective activities.
The subject of this entry, called The Testing Mole, is an issue I ran up against at least once in my 20 years of experiences.
The question is: the goal of testing is to prevent, rather than catch, errors, right?
I’ll ask it again a different way: would you rather your testers catch errors when code is submitted to them, or would you rather your testers get involved early, helping to identify possible points of failure as requirements are discussed?
My team lived at the painful intersection of a work culture which rewards QA reporting defects and Agile. When I first joined the team, I was, in part, hired thanks to my interest in being collaborative and in promoting collaboration. I’m fairly obsessive about both. I’ll comment on collaboration versus competitive engineering groups in another post.
I quickly noticed a colleague who prided himself on having been promoted to QA from development. When we were assigned the same team of two teams deployed to address a common requirements backlog, I felt uncomfortable already. He was the guy in early training sessions who would assert he knew the answer to a team question, even though he was wrong, and would stick with that assertion, causing the team to side-track and clarify to him with whiteboard discussions how he misunderstood. He was short on listening and big on being the authority.
As our fledgeling Agile team began taking on user stories [waterfall: read as tasks and requirements], I noticed something else: he’d disappear after the stories were assigned. He’d work with resources outside of our team on automation strategies to test the stories. Then, once the stories were ready to test, he’d deploy code which I knew he didn’t have the capacity to have written. My uncomfortable feeling returned. When I asked him to explain the testing automation, he refused, saying it was his job as tester and that he wasn’t ready to demo it. When pressed, he showed me code used to verify our stories which provided false positives. I offered to help him, but he told me his job was separate from mine. He told me flat out that testers should not work with developers.
One thing lead to another, and he let it slip that the boss backed his strategy. When I escalated the issue, the boss told me a developer working with a tester would cause the tester not to look at the program objectively, thereby causing the tester to approve all programs, even with errors. Meanwhile the tester said his job would not be needed if he didn’t catch errors, and he said that he needed to file defects to keep his job. In my experience as tester and as developer, the collaboration between tester and developer helped prevent errors, and that it encouraged a team effort to catch errors, even if those errors were caught after code was released to the testers for a full testing session. Heck, sometimes I’d even go: wait, I forgot this edge case. Can you test it? I trust a tester to be able to work with me as a developer to help hone user and unit test cases as a team, and also to remain objective when testing during the final battery of cases.
To me, the boss’s approval of this secretive method of developing test cases indicated we had a problem, and so I asked our company’s Agile guidance team. Two sets of Agile coaches and the product owner independently shot down the argument that testers should take a story and develop test cases in secret. I thought the problem was solved. I realized the tester was still not including developers in his testing, but I felt comfortable he understood admonitions not to be secretive about test case development meant that he’d pitch in when needed.
We were in our sixth iteration when a story struck me as being suspiciously too easy. I called the tester and shared my screen, asking him to review the specs with me. Note: I was not asking him to complete testing on the story or to perform any testing at all. I just wanted him to talk through some of the things he’d be looking for in testing the story.
Initially, he said that would cause “duplication of effort”. I asked him how that would be duplication of effort? He couldn’t explain. Ultimately, I shared my screen and we reviewed the specs. It turned out he had questions about the specs I hadn’t noticed. Both of the questions were handled by assumptions I made, but another observation he had about the specs was not. The 15 minutes we spent on the phone stopped me from submitting code which would have been thrown back for a rewrite, including all the version control and release tracking associated with it.
I wanted him to get credit for what he noticed, even if it wasn’t and official defect against my code, so I asked him to email the product owner. He did, and the specifications were clarified. The product owner recognized he needed to be clearer in future specifications, and anyone else working on the same specifications knew to watch out for the same issue. His total investment was probably 20 minutes, including our call.
The next day, the boss called a meeting of teams. He said that QA members only review code when code is marked as QA ready, so that QA members can mark defects. He said there is to be no communication between developers and testers in the Agile teams after stories are assigned, until a defect is found and code is rejected. He said he hates Agile and wished we were using waterfall. When I supported the concept of developers and testers collaborating, he said: when a developer works with a tester, and the tester is convinced the code is good. He approves it, and it gets rejected in UAT — who is to blame? I said: the team? He also said that a tester needs to find defects. I said: the focus does not need to be on individual team members but on the work products produced by a sprint. I furthermore reminded him of the Agile leadership and other leadership at our company who asked us to work collaboratively, not secretly, during iterations. He said: I don’t care what Agile or the company says, I want you to do it my way. The conversation was less interactive and more authoritarian than I would have hoped for. In the end, I agreed to do it his way, as now my job seemed to be on the line.
However, for a broader audience, I’d like to fork two possible outcomes which the boss would not listen to: one is the teammate approach, and one is the testing mole approach.
In the testing mole approach, the team receives a set of tasks to perform with requirements. The testing mole goes off and works with outside resources to automate test cases for the tasks while the developers complete their work. As work gets completed by development it is released to the testing mole, who then goes: gotcha! you missed this case! Go back and rewrite it! Developers talk with tester to understand and possibly contest the defect. They do a rewrite, going through the hassle of updating versions and maybe even dependencies, to correct the defect. Once again, the code gets thrown over the wall to the black box which is QA. Gotcha! Process is repeated. The 2 weeks sprint completes with a slower velocity, but the tester gets complimented for finding so many defects.
In the teammate approach, the team receives a set of tasks to perform with requirements. The teammates work together to understand the requirements and to develop test plans for the task, if not drafts of test scripts. As development continues, the tester, an experienced and wise QA resource, comes up with more and more cases, keeping the team in the loop. Some cases are discovered by the developers, who share with the QA person — the developers, knowing their code, also know internal edge cases which might not be visible to an outside user. If there are gotchas because test cases are arrived at late in the game, everybody sighs a collective sigh of relief because the team was spared releasing defective code. The QA person, meanwhile, has more time, rather than less, to develop advanced test cases because the collaborative nature of testing has allowed that person to learn from the team and to develop additional cases which that person has the time and focus to create.
So, waterfall or Agile, I assert the following: QA and testing should be trusted to be independent-minded enough to question a developer’s concept of how to test code, but should not ignore a developer’s input or questions and should be willing to divulge test cases in advance of final code approval.
In other words, especially in Agile, don’t be a testing mole. A mole is an enemy, not a colleague, in your ranks who springs up at the worst possible moment to leak info to destroy progress.
You don’t want to be that person.
I’ll explore team dynamics of competition versus collaboration in another article.
Thanks for reading! When I see applause for a piece of mine, I want to write more pieces! Please applaud if you appreciate people writing about topics like this.
If you enjoyed this piece, you might consider reading some of the other stories I wrote for this series. Thanks!
Second Disclaimer: I am a senior software professional with experience as project lead, team lead, and lead engineer in the greater Boston area. I am actively seeking a job while currently fully employed. The world of software development can be an ugly place, and I hope to expose some of the more bewildering and uncomfortable corners in the hopes of improving the industry and the craft. Out of respect for current and former employers, I will keep the names of employers and/or clients scrubbed and anonymized as best as possible in these reports.