Agile Testing and Process Thoughts with Janet Gregory

Graphic Recording of Awareness Session at Agile Testing Days

2011-11-28T14:53:00.000-07:00

I attended Rob Lambert’s session “Do agile teams have wider awareness fields” at Agile Testing Days in Potsdam, Germany in November this year. The ‘doodle’ I have included in this blog is from that session. I learned about this medium from ImageThink who recorded all the keynotes at the Oredev conference in Malmo Sweden, and gave a session on how to do it. I don’t hope to compare my scribbles to theirs, but I did find it an interesting practice. My test was to leave my book alone for 2 weeks and then go back and see what I could understand. Here’s my takeaways from Rob’s session.

To broaden one’s awareness, we need to widen our choices, and surround ourselves with different types of thinkers. Too often, we concentrate or focus on an idea too narrowly and then are constantly surprised by the outcome. Rob suggested creating a learning time line – what do you want to learn, and then set yourself some ideas about how to get there. That way you can slowly move items from the “Can’t Do” list to the “Can Do” list. The first step is awareness. Only when you can see choices, can you begin to accept. Start with yourself, and then start raising awareness to the team, and so on. In my classes, I talk about scope of control, and Rob pointed out when you think about concentric circles in your scope of control, you see that when you widen your awareness, you can widen your influence. Of course, there can be a downside of too much awareness – information overload.

I think I need to keep practicing with this medium, but I found I remembered most of it. If you follow the Rob's link, you can find the slides there.

Testing the Big Picture

2011-10-21T13:55:00.000-06:00

One of the top seven success factors that Lisa Crispin and I talk about in the summary chapter of our book Agile Testing: A Practical Guide for Testers and Agile Teams is "Look at the Big Picture". We talk about using the agile testing quadrants to help think about all types of testing, and how to keep the customers point of view.

This means understanding what problem the customer is trying to solve with each new feature. One practice I have been coaching teams to try, is defining acceptance tests at the feature level (or theme / epic if you are practicing Scrum). ATDD (Acceptance Test Driven Development), BDD (Behavioural Driven Development) or Specification by Example all talk about defining tests at the story level. We often forget about the feature level – the bigger picture. How do we know when the feature is “DONE”?

In specification workshops, we start with a feature, decompose it into stories, and often lose or forget to keep that feature in mind when testing. Now, if we created acceptance tests – both desired and undesired behaviour – at the feature level, then have something to test.

When we decompose the feature into stories, I suggest creating a story to “Test Feature A”, with the corresponding acceptance tests. I have found the idea of creating a “Feature DONE” definition to be very powerful. There are many tests that do not make sense to do at the story level, but apply to the feature. Many of the Quadrant 4 (technical facing tests that critique the product) fall into this category. Some examples are load tests, usability, and browser compatibility.

The ‘Test Feature’ story is prioritized after all the individual stories are DONE. Tasks for this story might include: Test browser compatibility, Perform load test, Automate GUI tests, Create user documentation, and Perform UAT (User Acceptance Test). Testing at the feature level enables us to run these tests when it makes the most sense. End users can be given the chance to test the complete feature and give feedback as soon as it is complete.

Adding acceptance tests to the feature, defining Feature DONE to include the tasks and tests that make sense at this level helps teams to consider the big picture and the business problem that they are trying to solve

Choosing Functional Test Tools for Large Organizations

2011-10-07T08:23:00.000-06:00

In one of my tutorials at StarWest 2011, someone asked the question about choosing tools – specifically functional automation tools. I do believe teams should chose the tools that work best for their application, but sometimes we have to think beyond our immediate project team.

As large organizations adopt agile, many are running into a common issue where each team wants to choose their own tool – after all, they are supposed to be self-organizing? I think this might be a bit short sighted.

One of the premises of eXtreme Programming is shared code ownership, and if you ask any programmers working on one product, they will all be using the same tool for their unit tests. For example, jUnit if they are coding Java. They may use different IDEs (editors) but the unit testing framework remains the same. We need to apply some of those ideas to choosing our functional test tools.

If we think about an extended team, say one that included the support or operational folks, would we still want each individual project team selecting a different tool? If multiple teams are working on the same product, can we really support the automated regression suite mixed and matched arbitrarily?

One suggestion I have for organizations facing these problems is to define a class of tools for teams to select from. For example, for API level tests, you can choose from FitNesse or Cucumber or for tests through the GUI, Selenium or Ruby and Watir. One way to determine these selections is to have product teams experiment with several tools, and then compare pros and cons, and select one for each type of testing they want to perform.

It is important to understand who will using them, and the teams they impact.

I am a member of the Community of Thinkers

2011-10-06T07:12:00.000-06:00

Jean Tabaka, Liz Keogh and others devised this statement and to me, it expresses the power behind a community of like thinkers. Patrick Wilson-Welsh mentions the idea of Network Weaving on his blog, and I like that idea as well; that also resonatated with me.

Thank you Jean, Liz, Eric, and others for sharing.

--------------------------------------------------------------------------------

A Community of Thinkers

I am a member of a community of thinkers.

I believe that communities exist as homes for professionals to learn, teach, and reflect on their work.

I challenge each community in the software industry to:

reflect and honor the practitioners who make its existence possible;
provide an excellent experience for its members;
support the excellent experience its members provide for their clients and colleagues in all aspects of their professional interactions;
exemplify, as a body, the professional and humane behavior of its members;
engage and collaborate within and across communities through respectful exploration of diverse and divergent insights;
embrace newcomers to the community openly and to celebrate ongoing journeys; and
thrive on the sustained health of the community and its members through continual reflection and improvement.

I believe that leaders in each community have a responsibility to exhibit these behaviors, and that people who exhibit these behaviors will become leaders.

I am a member of a community of thinkers. If I should happen to be a catalyst more than others, I consider that a tribute to those who have inspired me.

--------------------------------------------------------------------------------

”A Community of Thinkers” by Liz Keogh, Jean Tabaka and Eric Willeke is licensed under a Creative Commons Attribution-Share Alike 3.0 License. Please attribute to the distributor of your copy or derivative.

Jigsaw Puzzles & Small Chunks - The Ending

2011-09-14T14:28:00.000-06:00

I was not able to finish the puzzle I described in the previous blog during the week I was at the cabin, but I decided to finish the story. One of the reasons iterative & incremental development is so powerful is that we can deliver the highest priority items first. Although I didn’t finish the puzzle, the pieces that I did get put together were my highest value sections. I decided to show you as much as I completed, along with a copy of the picture as it would have looked. You may not agree with my decisions on what was the most important part of the puzzle, I was happy with what I had completed within my allocated time frame.

Jigsaw Puzzles & Small Chunks

2011-07-14T21:37:00.000-06:00

One of the hardest things I find to get people to understand is how to think in small chunks when it comes to breaking up features and stories. One company I was recently talking to is still struggling on defining a release scope. They haven’t gotten the idea of thin slices, delivering small pieces and adding more each time until they get something that meets their needs.

Last week I was at our cabin and pulled out a jigsaw puzzle. I love doing them, but the only time I seem to tackle them is when I am there. This time, I was all alone when I dumped the fresh puzzle on to the board. I looked at the picture – my final goal, the release acceptance test if you like, and tried to decide how to sort the pieces. There is no one correct way to do this. Everyone has their own ideas. The only consistency that I have seen is everyone tries to make sure they get all the outside pieces separated.

As I began sorting, I realized that it was easier than normal even though there wasn’t a lot of definition difference between pieces. I realized it was because I was making all the decisions and didn’t have to explain my thinking, why was I putting that piece in that pile, etc. ... and that got me thinking. I started off with 4 piles – the outside pieces, the ones that were mostly white, the beige coloured ones, and the shaded browns. As I sorted, I added piles because I started to see differences that I hadn’t before – the curtains, and shaded gray pieces. I started to think of this process as if I was a Product Champion. What was going to be in this release – my initial thoughts, and then a bit more detail as I considered options. My piles of pieces were pretty generic with little understanding of what the details were. I believe this is why I found it helpful to do it alone – I had no real idea of what I was doing, I was only starting to understand the bigger picture.

Once I got all the pieces into my initial 6 piles, I was able was ready to start collaborating with people, and get more ideas in play. Was my thinking correct? Did they understand my ideas and why I sorted them into those piles? Could I explain my vision so that others could help me put it together?

Since I was alone, I started by framing the puzzle, putting together the outside pieces. Of course, it did not work the first time. I had extra pieces that didn’t quite fit, and others that were missing pieces. I desperately wanted someone to challenge my thinking – where did I go wrong. Instead, I got up, walked away and came back a bit later to see where my issues were. All the pieces were there, just not quite in the right places. This is what I like to see in release planning sessions – looking at the big picture, where are the holes, what is missing.

Once the frame was done, I looked at the other five piles trying to decide what the most important thing was. To me, the most important part of a puzzle is the “core”, what makes the picture. If I am doing a landscape, the buildings or highlights are the most critical. If I don’t finish the sky, I don’t think my puzzle is incomplete – the challenge for me is not trying to fit like colour pieces. It is tedious work that is a ‘try it’, ‘does it fit’ kind of effort. To me, that is low business value and I will happily put the puzzle away without completing a sky.

In this puzzle, I chose a simple well-defined area to start with. These pieces were in the beige coloured pile (my feature). I divided this large pile into 4 smaller piles and started with the simplest “story”, the mirror. It was then easy to continue and add complexity, the rest of the beige pieces which belonged to a couple of picture frames, and the background wall. There were some that didn’t belong to this feature, so I put them into a ‘backlog’ to be grabbed later when I found something useful they belonged with.

As I was going through this process, it dawned on me that this might be an easy way to think about creating stories. Don’t try to look at the whole picture at once, but tackle each section a bit at a time. Keep breaking it up smaller and smaller. It becomes much easier to deal with. Eventually, it gets put together and you have your big picture. It is important to keep the big picture in view at all times, but work on small chunks ... one piece at a time.

The Power of Personas in Exploratory Testing

2011-05-08T09:06:00.000-06:00

There are many great articles and books on exploratory testing and I’m not going to try to explain what other people do better. What I do want concentrate on is the power of approaching your exploratory testing from different persona’s point of view. It occurred to me that I haven’t heard people talk much about it lately and decided to raise the awareness of this easy but powerful tool.

Exploratory testing belongs in Quadrant 3 of the Agile Testing Quadrants – Business facing tests that critique the product. You can see more about the quadrants in Brian Marick’s blog “My Agile Testing Project,” http://www.exampler.com/old-blog/2003/08/21/ or in our book “Agile Testing: A Practical Guide for Testers and Agile Teams”.

I guess the first time I heard about it, was at the Canadian Agile Networks workshop in 2005. At that time, I was very interested in the cultural aspect of agile. Jonathan Kohl’s position paper was Exploratory Testing using personas, which can be seen here http://www.kohl.ca/blog/archives/000095.html. Since then, other people have touched on it as well. For example, Elisabeth Hendrickson in her presentation http://www.slideshare.net/codecentric/exploratory-testing-inagileoverviewmeettheexpertselisabethhendrickson

I understood personas much better after listening to people like Jeff Patton and David Hussman. Read up on Jeff Patton’s website- http://www.agileproductdesign.com/blog/agile_product_development.html

Using personas is a great way to help identify requirements but they can also help us test better. When I use personas to help identify test data and test scenarios that I (as a tester) had not thought about before, it helps me get better test coverage. To do this, I need to understand our customers – both internal and external. The marketing folks are often the ones who create the external personas, but sometimes overlook internal customers like administrators.

In agile projects, we encourage testers to help define the requirements by questioning assumptions. Personas are a great way to think differently. In a recent workshop at Quest conference in Boston, Ellen Gottesdiener demonstrated how testers can truly add value by thinking about different dimensions – one being the people. If we carry those thoughts after the code is built, and approach some of our exploratory testing from that perspective, we look at the product differently. For example, if am testing a shopping website as a “hurried working mom” as well as a “senior citizen”, I will be looking at completely different perspectives and may find different types of bugs.

There are many different ways to approach exploratory testing, and I encourage everyone to try lots of different styles.

Programmers as Testers?

2011-03-11T19:31:00.001-07:00

There was an interesting and bit controversial thread on the agile testing yahoo group over the last couple of days. Some people thought that all agile testers should have programming skills. Others thought that programmers cannot possibly be good testers because they don’t consider overall quality.

It raised a lot of thoughts in my mind, so I thought I’d put them down here. I started my career as a programmer and developed much of my attitude towards testing in my first job. My supervisor was diligent about making me walk through the tests I wrote, and my test results. We did all our own functional type testing, and the operations group did the installation and system testing.

Programmers can make excellent testers, and on agile teams that practice TDD, when asked, they say they spend about half their time testing. Teams that truly practice the ‘whole team’ approach and believe everyone on the team is responsible for quality will work together as a team to make sure all the necessary testing is done.

That said, I probably wouldn’t hire someone who’s main passion was programming and saw the tester role as a stepping stone to being a programmer – mostly because they wouldn’t be committed to the craft of testing. However I do prefer testers with a technical background because it makes it easier for them to collaborate with the programmers. That does not mean I wouldn’t hire testers who do not have programming or some kind of technical background. Depending on the tools the team uses, it is not an absolute necessity... as long as they are willing to learn. I look for attitude.

There are many testers who are “ex-programmers” – I am one, so I wouldn’t let the fact they are programmers stop me from hiring them as testers if that is what they want to do.

Testers do add value to an agile team – with or without programming experience. They can see the big picture and consider impacts to the system that programmers often miss. When programmers are in a team, they are programming at the code level. It is hard for them to switch contexts and be concerned about the big picture impacts. I think we sometimes do teams harm when we ask them to switch contexts often. I’ve seen “programmer only” teams be successful, but they do have to make a conscious effort to think big picture and often have one team member play the role.

To quote Lisa Crispin, “Diversity of skills and viewpoints is critical to a successful project. We need people with different backgrounds, different types of experience, different abilities working together.”

I do not believe an agile team can succeed in the long term without automation of some kind. We need to consider the agile testing quadrants, as well as the pyramid. Our regression suite includes automated unit tests, as well as service layer tests and yes, some GUI and workflow tests. We do need to remember that automation does the checking – it doesn’t find new bugs but exposes unexpected changes. Automation is necessary, but not sufficient, so the team needs the skill set to support all automation tasks.

The team also needs skills to perform exploratory testing, and most often those come from someone with a testing background. I have seen several teams be successful where “manual” testers create/design the tests using Fit, or a Fit-like type of interface, while the programmers create the fixtures in the code itself. This has been very successful, and when it was done using Ruby instead of Java, within 1 ½ years, those manual testers were supporting the whole automation framework. They learned how to code in Ruby, but still collaborated with the programmers on new ideas.

I don’t really care who does the automation – unit tests, API or service layer, GUI / scenario tests, load automation, etc..... , as long as it gets done in a timely way, and provides the feedback necessary.

Book Review – Jonathan Rasmusson “The Agile Samurai”

2010-12-21T21:33:00.000-07:00

I first met Jonathan about ten years ago when he was a young programmer eager to prove to the world that he was the best he could be. At that time I was impressed with his willingness to be open to new ideas – even the idea that a tester could be valuable on an agile project that was practicing eXtreme programming (XP).

As I read “The Agile Samurai”, I was amazed once again with his simple way of expressing some of the most difficult issues we face in agile projects. He introduces us to the Inception Deck – a great place to start if you are thinking about transitioning to agile, or if you are having problems and think expectations need to be reset. One of the most powerful ideas he talks about which I see consistently lacking in many teams, is the idea of ‘Meet your neighbours’. Who do we need to talk to? Teams forget there is more than just the Product Owner or single customer that we have to please or may have information that can help us deliver the right product.

Jonathan uses examples to back up each principle he introduces. The conversations between the Master Sensei and the aspiring student at the end of each chapter are better than any thought provoking questions I have seen in some books. They make you stop and think; one of my favourite conversations is the “The Incomplete Story”. What do you do when you don’t finish a story within an iteration - something I see in many new agile teams.

After reading the book, I have adopted Jonathan’s ideas of sliders to show the trade-offs between quality, scope, budget, and time. These forces are in conflict, yet consistently clients want ‘all’. The sliders give us a great way to demonstrate the trade-offs between these forces – it is such a simple idea, yet so powerful.

One of the things I really appreciated was the acknowledgment of more than one “right” way to do things. He echoes my feelings - If we adhere to basic agile principles and understand them, we are ahead of the game and dramatically increase our chance of success. Agile is not a silver bullet, but takes work to achieve success. Vocabulary is one of the issues I struggle with when I go from client to client. Jonathan has done a suburb job of making his vocabulary generic using phrases like Master Story List.

One of the things I thought was missing was how ATDD feeds into the TDD cycle. The last few chapters are very developer focused, but I would have like to have seen how tightly coupled customer tests are to the developer tests. However, I am happy that he talked about the importance of building quality into the system using the practices of TDD and refactoring.

After I read the book, I realized how much I would really like to work with Jonathan again.

ATDD vs. BDD vs. Specification by Example vs ....

2010-08-31T20:38:00.018-06:00

At Agile 2010, there were about 20 of us at the AA-FTT (Agile Alliance Functional Test Tools) workshop. The emphasis was on “the state of the practice” of Acceptance Test Driven Development (ATDD). In one of the breakout sessions, we had an interesting discussion on what we actually meant by ATDD, which made me think a lot about the practice itself. Although we all knew that acceptance tests defined by the customer / business expert is what drives the TDD (test driven development) process, there were discrepancies in how each of us thought of full process.

JB Rainsberger had considered ATDD as two concentric circles with TDD, the developer practice of Test Driven Development, to be the center, with ATDD surrounding it. I felt his view of the practice was from a developer perspective – give me an example, and I will start coding. Many of us at the table thought it was more than that; it included the collaboration necessary to get the examples. Based on that discussion, the next question asked by JB was, “Then, what’s the difference between ATDD and BDD?”

Part of the problem is that we don’t all use the same vocabulary relating to the practice. I started by writing down all the words I know are used in conjunction with the practice. The following words and phrases are some of the ones I could think of:

Examples, scenarios, acceptance tests, customer tests, behaviour, business-facing tests, story tests, functional tests, acceptance criteria, conditions of satisfaction, business rules, executable specification

There have been many explanations about ATDD and I’ve included a couple here to help explain.

Jennitta Andrea wrote a short article for Iterations about what ATDD is, and why teams should be doing it. She described ATDD as:

“

the practice of expressing functional story requirements as concrete examples or expectations prior to story development. During story development, a collaborative workflow occurs in which: examples written and then automated; granular automated unit tests are developed; and the system code is written and integrated with the rest of the running, tested software. The story is "done"—deemed ready for exploratory and other testing—when these scope-defining automated checks pass”

The full article is at: http://www.stickyminds.com/Media/eNewsLetters/Iterations

A couple of years ago, Elisabeth Hendrickson used an example to describe ATDD in her blog: http://testobsessed.com/2008/12/08/acceptance-test-driven-development-atdd-an-overview/

As a result of the AA-FTT workshop, Declan Whelan, Gojko Adzic and a few others came up with a diagram trying to get a common understanding around Specification by Example. It is on Google Docs - http://tinyurl.com/3x52ap9

Dan North’s description of Behaviour Driven Development (BDD) http://blog.dannorth.net/introducing-bdd/ is a bit more complicated, but I interpret it as using the word ‘behaviour’ rather than ‘test’ , and use the “Given, When, Then” guidelines to describe the precondition, trigger and post-condition. In a recent agile testing yahoo post, Liz Keogh posted this:

“BDD's focus is on the discovery of stuff we didn't know about, particularly around the contexts in which scenarios or examples take place. This is where using words like "should" and "behaviour" comes in, rather than "test" - because for most people "test" presupposes that we know what the behaviour ought to be. "Should" lets us ask, "Should it? Really? Is there a context which we're missing in which it behaves differently?"

Where we choose to call it BDD or ATDD or Specification by Example, we want the same result – a shared common understanding of what is to be built to try to build the ‘thing’ right the first time. We know it never will be, but the less rework, the better.

Using business rules, use cases, business expertise (customers, Business Analysts, domain experts, etc), the development team’s (programmers, testers, DB guys, etc.) expertise, and any other supporting information, we can describe what we expect the software to do in the format of examples which are just a specialized test. In workflow type applications, describing behaviour may be applicable. If it is calculation heavy, then a spreadsheet type test format with specific examples might be more appropriate.

Do these tests have to be automated first? I encourage new teams that are struggling to find the right tool, to focus on the purpose of ATDD which is to drive development to deliver the right ‘thing’. If we can do that with a sentence, let’s do that first. We do have to automate them at sometime during the iteration, but until a team has their rhythm or cadence, the automation may not come first. Once you have the automation, you then have documentation that is up to date and describes the system behaviour.

The picture I use to describe ATDD for new teams is:

I think the story isn’t ‘Done’ until the customer has actually validated the actual completed story which includes pairing with the developer after he’s completed coding for a first look, and the exploratory testing.
I will continue to use the phrase Acceptance Test Driven Development (ATDD) unless the industry decides on a common vocabulary, because I find that business stakeholders not only understand how to give examples, but also understand when I talk about acceptance tests that prove the intent of the story or feature. Once we have that, the team knows enough about the scope to start coding and testing. That's what's important.

About Learning 2

2010-05-10T12:28:00.000-06:00

We know about how people learn – auditory, visual, kinesthetically. But what about those things we can’t know – like people’s attitude, or how our brain perceives messages. This post comes about because of my reaction to a specific teaching style I experienced lately.

Over the years, I have been exposed to ‘learning games’, the kind where you throw two dice and have to guess the total. The leader knows the rules and you have to figure them out based on what you guess is the total. The leader says yes or no to each guess until you figure out the ‘trick’. Until a couple of weeks ago, I’ve been able to skirt around being the person doing the guessing, but finally got approached one evening at StarEast to be an active participant. I tried to decline several times, but the leader insisted and finally asked if I considered myself a tester and issued the challenge. Looking back, I knew I always avoided the games, but didn’t realize how strong my reaction would be. I actually felt myself have a physical reaction to the invitation. I became very agitated, and finally mumbled something about how I didn’t want to because of ‘childhood issues’ with games of these sorts. I then ran away feeling very stupid and joined another group of people.

I’ve done a lot of soul searching since that evening, and what I have come to realize, is that I enjoy puzzles of the variety I can work out such as logic problems, Suduko, etc., but really don’t like the variety of puzzles that have ‘tricks’ to them. My brain doesn’t work that way, and there are reasons why those types of puzzles upset me – I won’t go into details.

So, what does this have to do with learning? Well, I know I’m mainly an auditory learner so I listen. People who work with me, often think I’m visual because I use a whiteboard or draw in the air. I do that because it helps me explain an idea I’m trying to get across, or my concerns. It is articulating the problem that helps me to reinforce the ideas in my brain. I’m also a people person. Most problems are people problems, and I like to solve issues by talking and listening. I guess that is why I usually have jobs that are not purely testing, but involve coaching or working with teams to adapt new processes.

But, given all that, I also recognize I have blind spots that may prevent me from learning. Jerry Weinberg, through his PSL (Problem Solving Leadership) course, defines problem-solving leadership as the ability to enhance the environment so that everyone is empowered to contribute creatively to solving the problem. As a leader, that means I need to recognize that other people many have similar blind spots and learn to recognize them. When I am in a teacher role, I need to understand that people may not be taking in the message if they don’t like the delivery format. I need to open my mind to the possibilities and observe the feedback I am receiving from participants.

How many of us teachers / trainers have learned this skill? Or even recognize that it is a skill to learn.

About Learning

2010-02-17T08:39:00.000-07:00

There have been a number of discussions about certifications lately on some of the on-line groups. People seem to be either ‘for’ or ‘against’ testing certification. I am neither, but I am against advertising certification as a way to teach people how to test. The testing certification programs I am familiar with, only show that people were exposed to a certain amount of information, and memorized enough to pass a test.

I have been questioning Carol Vaage1, a grade one teacher, about learning skills. Research says pushing kids into reading by memorizing in the early years isn’t the best way to teach them; getting them to think is much more effective.

To that end, she encourages her class to explore, create, play and invent. This method teaches the children problem-solving skills as well as finding the wonder in learning. They begin to believe they are capable of figuring out the world and the wonderful problems and opportunities. She has an open-ended program where her class can engage in projects that interest them. Brain research shows that constructivism helps their brain the most – i.e. building their own sense of systems. For more on constructivism, see http://tip.psychology.org/bruner.html .

She compared her class to some of the very structured classes where instructors teach children reading and phonics using preprogrammed series and worksheets. Carol says that if children start young in a confined structure like this, it is really hard to break them out of the habit of accepting the “authority” (eg. the teacher or the phonics program) as knowing the most. In Carol’s program, children grapple with constructing the language of print as they try to research and seek out the answers to questions they’ve created. They also learn sounds for letters or combination of letters, but in a way that is constructivist; they choose to learn how to read.

She used a math example to help me understand what she meant. There are two ways to teach how to solve it.

• teach a strategy and kids can memorize it
• give them a problem situation and they invent their own strategies

Guess which one can be generalized? You got it, the ones they can figure out themselves. Her example:

I gave my kids a sheet of paper and asked them to prove to me that they know how much 100 is. Some of them wrote down 25-50-75-100 and smugly passed in their paper. I said, you’ve shown me 4 numbers. That doesn’t prove you know how much 100 is. Show me more. By the time they had worked with their partners for 2 days, they had some amazing work. Some had drawn groups of 10 dinosaurs, colour coded, others had written out every single number by ones, others had counted by 5’s and drawn a hand with 5 fingers by each number, etc.

This same type of problem on a worksheet would look like this. It has 100 numbers written on it, and 8 are missing. Fill in the blanks.

That’s the difference in thinking that you’re looking for. Something that engages the mind, not limits it to memorization. That’s the lowest on the thinking skills. What you want is the highest – problem solving, synthesizing and creating.

Testing certifications as they stand today, do not test skills or problem solving. They test memorization. As long as testers and organizations that hire testers recognize that is what certification gives you, it is not an issue.

I think we should encourage learning in any form. I would rather testers look to develop their skills, but one of the problems we face, is the lack of courses or training programs. One is Rapid Testing (http://www.developsense.com) . Michael Bolton teaches this with the intent for participants to learn testing skills through exploration in experiential exercises. Lisa Crispin and I have been giving a 3 day course on testing in agile projects for the past year based on our book. We give some theory, but also have interactive exercises to demonstrate how to collaborate, or break down stories into testable chunks, etc., as well as a full iteration simulation. We use different tools to encourage participants to think. We have been asked to turn it into a certification course but have refused as we don’t think it is possible to do so with any degree of confidence.

Carol wrote me an email one day just before school was out for the summer relating an experience she had that day. Her story: “My kids still want to keep coming to school next month - everyone else around us is ready to quit. We still had so much we still wanted to learn. They were finishing up their front pages of their growth portfolios - they needed to write "My Learning Book" and draw or write at least 3 things we learned about this year. They started talking about all the things they learned, and kept drawing and writing - I had a hard time stopping them. One child said – ‘This is more fun than fun day’. That meant I reached my teaching goal for this year. They love to learn! Next year, we'll be learning again - just don't know what - the kids will have a voice in it.”

To summarize, I don’t necessarily think the actual certifications are the problem. The real problem is the artificial need generated by organizations requesting the certificate, and the certification community advertising them as the solution to all testing education.

How can we, as a testing community get the testers in the world want to learn and challenge the status quo – How can we get them to want to learn – anyway they can?

Footnote

Carol Vaage holds a Masters in Early Childhood, with B Ed, Ed Dip, M Ed from U of A

For some of the amazing projects Carol has completed with her classes, take a look at: http://carolvaage.net/ She shares one of her projects and how she approached the learning at: ttp://www.2learn.ca/projects/together/STORIES/vaage.html

Quality – What is it, and what’s the cost?

2010-02-02T13:06:00.000-07:00

Last night (Feb 1rsts) at the Scotsman’s pub in Oslo after the XP Meetup, I had an interesting philosophical discussion with Tom Gilb, Geir Amdal and Rune Sørensen.

My definition of quality is pretty simple – A product has a sufficient level of quality if it meets or exceeds customer’s expectations. I know it’s pretty vague, but so are most of the customer’s expectations.

Tom presented some of his ideas about defining attributes of quality with the values varying depending on the defined parameters given by the customer. For example, the quality attribute of availability might vary between 0% and 100%, but the acceptable level for the customer might be 99.5%.

Geir suggested that the most common interpretation of the term in an everyday context (as opposed to the requirements and value discussion) is 'how well a thing is made'. He then talked about how we often have one conception of quality when we speak of quality in general, and another when we're talking about specifics such as Tom outlined. What is its relationship to the cost-time-scope triangle of project management?

We discussed pros and cons of all three ideas and really didn’t solve the riddle. However, about 3:00 am this morning (I was suffering from a bit of jetlag), I realized that I could use a simple example to explain my position.

Let’s take 3 makes of cars. The Mini Cooper, Toyota Corolla, and Kia Rio, all small, all imported into Canada.

Base Kia Rio - $14,000
Base Corolla - $16,000
Base Mini - $24,000

I bought a Mini Cooper for lots of different reasons. Ron Jeffries claims it is because of its cuteness factor and that may be part of it, but I also took into account its reputation (how well it is made). Another deciding factor was its acceleration and handling capabilities. I chose the upgraded Clubman S model so paid more than the base price because I wanted the extra room and the added acceleration.

I drove my daughter’s Corolla the other day, and found it lacking in so many ways. Basics I take for granted never even crossed her mind when she bought her car. I missed my heated seats, and automatic door locks – forget the other reasons. However, she bought it because of its resale value. She has a lease and wants to buy it out and to be able to recoup some of her money at the end of it all. She chose the Corolla, even if it was a bit more money at the time than something like the Kia Rio because of its reputation, but couldn’t afford any of the extras like automatic door locks.

Quality has a cost. If we make all cars to the standards of the Mini or a Mercedes, we eliminate a whole market of buyers. So, although I agree with both Tom and Geir, I still think quality is meeting or exceeding customer’s expectations. In my example, my expectations are much higher than my daughters, but I’m still not willing to pay for the perceived higher quality of a high end BMW or Mercedes. And I really didn't care if it accelerated from 0 to 60 in less than xx seconds. I only cared that it went fast enough for me.

In agile projects, when the customer understands what the critical success factors are up front, it helps the team work towards those delivering to them. If quality is defined as one of those factors, and the customer(s) can articulate it and how much they are willing to pay (a constraint), we can work towards that level to meet their expectation.

Reviews in Agile

2010-01-30T15:29:00.000-07:00

A few weeks ago on the agile testing list, a developer in a tester role asked about reviews – Where do they fit in the agile process? Lisa (www.lisacrispin.com) and I didn’t directly answer this concern in our book, so here are my thoughts about this subject.

Organizations handle reviews differently, but I like to see the developers take responsibility for their code and believe they should take accountability for code reviews. If the developers create standards for the team to follow themselves, they are more likely to follow them. There are tools that can be run during compilation to catch easy errors and can be used to supplement more extensive reviews of the code.

My personal preference is for developers to do pair programming. This not only ensures a better design (two perspectives on the code), but also gives a ‘real time’ code review.

My second choice is peer reviews, with the coder walking through what they have done with another developer immediately after finishing coding (or if they are having trouble). This has a few advantages. First, it is a few direct method of getting quick feedback. Also, articulating out loud makes people think differently, (http://c2.com/cgi/wiki?RubberDucking) and developers will catch many errors themselves. The added benefit of peer reviews (or pair programming) is the cross-training that happens so shared code ownership is easier to adopt. Another advantage that I can think of immediately is its relatively low cost compared to formal code reviews that take several people’s time. Personally, I think it is more effective as well.

When I test, I like to include reviews by developers and other testers for my own tests that I create. I find the more people that have input into what I test, the better the tests are. Developers give me many great ideas for exploratory testing as well, since they know where the weak spots of the code exist.

Many teams include reviews in their definition of ‘Done’. If you are having problems putting peer reviews into practice, one idea I have found useful when starting new practices, is to either write a separate task card, or use the testing column of your story board to mean reviews for programming or testing tasks.

Resolutions for 2010

2010-01-10T16:17:00.000-07:00

Last year was a bit of a blur for me. The book, Agile Testing: A Practical Guide for Testers and Agile Teams, that Lisa Crispin and I wrote, far exceeded expectations in sales. The need for courses on the subject also came unexpectedly. I spent the year travelling, giving the course and consulting. It was tiring, but very rewarding when I got evaluations back that said things like “Probably all the information we learned will be immediately useful” and “The topics covered in class match problems we are facing”.

However, it left me with very little time to do other things like building a sustainable business. This year I have more information and a better idea of what type of opportunities exist, so I am going to approach this year more pro-actively.

One of my resolutions includes getting my thoughts down, which means more blogging. Look for a new blog every couple of weeks; this is my first of 2010. My focus has been on testing on agile projects but over the past year, I have learned that testing problems extend into the whole agile process. This means many of the problems I encounter on teams are more than “just” testing issues, so many of my blogs will be on expanded topics.

I am going to spend more time and energy writing articles as well. My first article (co-authored with Lisa Crispin) of the year, entiled "Agile Projects: 6 Ways to Avoid the "Mini-Waterfall", came out in the January edition of STP Collaborative magazine http://www.stpcollaborative.com/ which features “Women of Influence” in the testing profession.

Also, because I have been travelling so much (made super-elite), I have neglected my immediate surroundings. This year, I will work on building and maintaining more of a Canadian and North American presence, and try to give back to the community. My first self-sponsored course is to be held in Calgary, Canada on January 25 – 27, 2010. See my website www.janetgregory.ca/training for more details, or register at: http://www.regonline.ca/agile_testing_with_janet_gregory

Happy New Year to all

What is the real problem?

2009-09-15T20:11:00.003-06:00

I’ve been seeing a pattern lately with some of the teams I’ve been training or coaching. Some have been doing agile for a couple of years but have gotten stuck in a rut and don’t seem to be able to find a way out of it. The same problems keep cropping up in retrospective after retrospective. They read the literature but it doesn’t seem to help. Let me relate one example that demonstrates what I mean.

The team size is about 14 at this time and has varied a bit but not a lot. The team members have been together for over three years, and have been struggling with making their stand-ups effective. They read that the appropriate team size is seven plus or minus two, so they’ve tried dividing up into two teams but that created a different communication issue between the teams.

They united the team as one again because the ineffective daily stand-ups were better than the lack of communication. The meetings went too long and no one seemed interested in what anyone else had to say, and were boring. The sessions became useless as a communication tool.
As I started asking more questions to the team to understand the underlying problem, I realized there were several contributing issues. The team size perhaps was one of them, but I have worked on teams that size that had very successful stand-ups so I didn't think that wasn’t the main factor.

After some digging (mostly asking questions and listening), one problem that I saw was the developers had become very specialized, each one working on their own component. There was no interaction between what one developer was doing and another. The stories were divided up by component – horizontal versus vertical through the layers. One side-affect of this was the iteration became a mini-waterfall. The testers could not test the whole feature until all the components were put together – at the end of the iteration. This meant the testers had no real interest in what was happening either until the end of the iteration, and then they were so busy they didn’t have time to attend. The problem was amplified because of the 30 day iteration; there was a lot of code to test.

Looking at the real problem rather than the symptoms, gives us a very different problem to solve. By dividing the stories into smaller vertical slices, developers need to work together and communicate on a daily basis and will learn more about what the others do. The testers get the stories a lot earlier to test, and they become interested in the issues and progress communicated at the daily stand-ups.

With this team, I believe that changing the way they look at stories will solve multiple problems. Sometimes having someone from outside the team looking at the problem can save months, or even years of time.

Questions on Key Factors for Agile Testing Success

2009-07-22T15:48:00.002-06:00

Last week I gave a webinar on the Seven Key Factors for Agile Testing Success. I answered many of the questions on line during the webinar but there were quite a few that I didn’t get to, so I answered them by email. I thought I’d share them as they are questions that have been asked again and again.
========
1. How do you address the whole team approach when there are multiple (>5) teams contributing to a product?

MyAnswer: There a couple of things that can help here. I think there should be an overall Release Manager who owns the whole product release and can make decisions about the 'whole'. In Scrum, you should be having Scrum of Scrums (or a daily stand-up) with all of the ScrumMasters, or a representative from each team who can talk about the team's activities. I also believe that the teams should all be working on the same code base and have seen that work very positively. You get a constant integration happening. During demos at the end of every iteration, invite all the teams so that everyone can see what is being developed so there are no overlaps, or gaps in the product development. There is a new book out "Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum" by Craig Larmen and Bas Vodde (I haven't read it yet, but it is on my ‘to read’ list) that might help.
=========
2. Have you applied the whole team approach to an operating system (including its additional levels of testing such as system and solution testing) or just at a unit or application level? This refers both to a 3rd party operating system (like Windows and Linux) where we provide a product that includes the 3rd party operating system. It also refers to a proprietary operating system we are responsible the entire hardware and software stack.

My Answer: The system or solution verification testing tends to be more of a QA responsibility, but I like to get input from the whole team when planning it. Developers often have useful tips for testing early - before the final testing needs to happen which can greatly reduce the time spent at the end. I also really encourage early system testing (if possible) and developers need to be aware of this. The product owners, or customers often have great ideas about testing the solution as a whole, so yes, include them. There are definitely some unique issues that many organizations don't have, since you are working on the hardware and software. I have worked with embedded systems where we had the same issue (on a smaller scale), and I found it even more important to include all the stakeholders so we all knew what to expect.
==========
3. How useful is monitoring branch-flow code coverage for automated tests?

My Answer: I like to monitor code coverage in some fashion and branch flow is one way. However, I don't usually go in to depth as the power of the coverage is in pointing out gaps. Code coverage tools do not tell you how good the tests are.
============
4. How are the cases handled where automation has limitations?

My Answer: I'm not sure what limitations you mean, but there are areas that sometimes cannot be automated because it doesn't make sense. I think there are tools out there for most code bases, and sometimes we just need to explore those options. Manual testing sometimes makes sense if the automation takes a lot longer than running the test manually. There is a trade off though - if the test is core to the system, you may want to spend the time to automate even it is hard to do.
===========
5. How are we going to automate the unit test when the modules are under development? Most of the developers are reluctant to write stub code. What is your suggestion?

My Answer: It sounds like the tests you are talking about are more low level integration tests than unit tests. I recommend reading Gerard Meszaro's book xUnit Test Patterns (it's big), but pretty good. Unit tests should be small - verifying the behaviour of some small part of the system the developer is working on. It might be as small as a single object or method. Agile cannot succeed with out good unit tests, and it sounds like the developers may not understand their part in the building the quality into the system.
===========
6. If the iteration is for 6 weeks with development activities for 4 weeks, what is the best approach to test the iteration for short time i.e. for 2 weeks?

My Answer: I believe 6 weeks is too long for an iteration. Testing and coding should go hand in hand all the way through. In a two week iteration, I like my first story out of development by day 4 at the latest (preferably earlier) so I can start testing it. It sounds like you are doing a mini-waterfall... code, then test. The faster the feedback to the developer on a story, the easier it is to fix. I encourage small stories (and this might be hard at first), that can be tested right away.
===========
7. What is your definition of defect density? Would you use all test phases in the test pyramid to measure defect density?

My Answer: I actually haven't found it useful to track defect density on an agile project. If everyone is trying to produce quality code and we are fixing bugs as soon as they are found, the tracking becomes moot. We want to focus on prevention, not detection. However, if you have a legacy system I would track defect density by component so we could determine where a new story might be needed to address the technical debt.
===========
8. How do you handle inter-dependant modules? We cannot release the modules to the customer though they are on high priority, if they are dependable on some other module.

My Answer: High level release planning should take care of dependencies like that. Understanding the risks associated, which includes dependencies is critical. I answered more in Question 1
===========
9. Will agile work well in a team environment where team members work on multiple concurrent projects?

My Answer: Simple answer - No. I think that task switching is very detrimental to the success of an agile project. Now, sometimes a team member has 2 jobs. For example, testers often need to be caregivers for the test environments which is not a full time job by itself. When those testers are on an agile team, their availability is just lower. They need to be aware of their other responsibilities when taking a testing task and may not want to be on a critical path. If a tester is on two different projects or in a support role, their priorities are subject to change at any time and severely impact the amount of time they can dedicate to a project - which in turn means some stories don't get tested, or tested poorly.
==========

Shared experience

2009-06-04T14:31:00.003-06:00

A while ago, I had an email question from Jennifer about logging defects, as her team was starting their first iteration. I asked her to let me know how it went, and here is her update. I loved it because I thought they accomplished some amazing things and had some great lessons learned so I wanted to share. Thank you Jennifer for letting me share this great experience.

Quote:
" wanted to update you on our first sprint, which is now over. I certainly learned a lot! We did not complete the user story we selected (simple login and logout of the GUI and XML systems with both English and French) for several reasons, but largely because we underestimated the amount of time the individuals on the team were spending on non-sprint activities. We also did not include time for continued infrastructure additions, but those were hopefully a one-time cost that we won't experience again.

We had spent 2 sprints previously working on Engineering stuff (i.e. infrastructure) but we missed some items that we ran into as soon as we started the product development sprint, such as teaching QA to run a build, figuring out how to help QA change product configuration so that the application under test was running independently from the SDK, and waiting for a third-party to translate English into French for all our error codes and GUI screens. There were also some assumptions made on the design side that resulted in QA having trouble getting the product to start, such as which database we should be connecting to and what protocol we support, so we're trying not to make so many assumptions going forward.

The tasks that did not get completed in the last sprint were moved into the second sprint, which is currently underway. And I think things are already going better this sprint, based on what we agreed to do to improve communication and timeliness. One of the biggest problems was lack of team notification when code was checked in, so we now have an email being sent to all when a commit is made. Unfortunately, we still don't have automated builds, or a central build server, but QA can manage with on-demand builds for now. I

n the current sprint, I was determined to provide the test case purposes to design within 1 day so that they could starting coding immediately. Right after our sprint planning meeting, QA got together to draft test purposes for our user stories and list questions about the RFC specifications. We then met with our product owner to get answers to our questions and ensure we were on the right track for test cases. We recorded the answers and then met with developers to share the test cases and come to a consensus on internal error codes. Now design is starting to code, based on the test cases we drafted, while QA starts to script the automated tests. Tomorrow we'll have a defined list of English errors codes which can then be sent off for translation for the next sprint, which will contain the French portion (we've learned that French has to be one sprint behind because of the translation delay).

One of the other major things that came out of the last retrospective was the lack of GUI usability design being done up front. It resulted in a non-production-worthy login/logout page and lots of time at the end trying to figure out what it should look like. For the current sprint, we've agreed to get all programmers, QA and the product owner together, as well as our usability specialist, to design the main headers, footers, menus, and main content page. We met today for 2 hours, and it went really well - we came out of the meeting with a hand-drawn copy of our GUI consensus.

With respect to QA, the three of us are working as a single unit for now, so that we can all learn the automation tool together, design the data-driven test formatting in Excel, and understand the types of questions to raise (when, and to who). By next sprint, I expect QA can work more independently, but we still plan to have frequent peer reviews to ensure we stay consistent and brainstorm a good set of cases. I've had feedback from design that our methods of communication are helping them bridge the requirements gap between waterfall and agile, so I think QA is starting to find its stride on the team.

We still don't have a tool that generates load, but that's something I'll take on in-house with a designer once we've got the functional automation working smoothly. Already, I think the quality is better than what it would have been without Agile. It's really fascinating to see Agile working in such a green environment where they had never written anything down previously. I know we have a long way to go, and a lot of learning to do, but I think we're heading in the right direction.
.. end quote

Agile Testing Training Course

2009-06-04T13:54:00.005-06:00

Lisa Crispin and I have created a course on Agile Testing with the focus on how testers transition from a traditional testing team to testing on an agile team.

Over three days, we put theory into action through a variety of exercises. This course teaches testers how to fit into agile projects, contribute to the whole team and overcome common cultural and logistical obstacles in transitioning to an agile development process. It explains the values and principles that help testers adopt an agile testing mindset and how to accomplish traditional testing processes, such as defect tracking, metrics, audits, and conforming to quality models. Students will learn how to complete testing activities in short iterations, and how testers contribute on a daily basis during each iteration and release cycle. Through interactive exercises and group discussions, participants will discover good strategies for driving development with both executable and manual tests. The course is filled with real-life examples of the many ways agile testers add value.

We’re offering this as a public course in partnership with LeanDog in Cleveland and with Program Utvikling in Norway, and will be offering it in other locales as well.

I also offer it to specific client as well, so please contact me if you are interested. See my website for more details. www.janetgregory.ca/CoursesOffered.html

Titles, Roles and Leadership

2009-04-19T18:15:00.002-06:00

A discussion on the agile testing list got me thinking about titles and authority and how they related to a culture in an organization, or on an agile team. To make it more real, let’s talk specifically about testers, although it could apply to any role. If a tester has a title of ‘Senior’, does that make them a leader, or does it just mean they have more experience than someone who has been designated as ‘Junior’.

Many organizations have a strong culture using titles to differentiate between levels of authority. For example, a director gets the corner office – a very visible sign of authority while the manager gets a 10 x 12 sq ft office, and a team lead gets a better cubicle than the rest of the team. We are seeing less of this as organizations realize it hinders communication and collaboration.

I was recently in an organization where titles were closely guarded. Senior testers were the ones who got to go to the requirements meetings, or the project status meetings, etc. They were to pass the information to the rest of the testers they thought was important. We looked closely at this structure and although we couldn’t change the titles because they were tied closely to pay grades, we did examine the reasons behind the ‘rules’. Mostly, it was legacy from the days they were practicing waterfall. The titles didn’t change and some of the rules stuck around without questioning.

We changed things up a bit. Instead of the senior tester always going to talk to the meetings, the tester who had the most knowledge in the area to be discussed would go. One of the side effects was that all the testers became better at their jobs because they had more knowledge about the needs of the customer, and they became more effective at the iteration planning meetings because they could share their knowledge. They didn’t wait for the ‘senior’ to do all the talking. The stories were better because the input was more diverse and the meeting was more animated. An unanticipated side-effect was the requirements sessions with the customers became more effective because we needed an agenda to know who to send.

In his book “Becoming a Technical Leader”, Jerry Weinberg talks about organic change and how people may chose to lead in different ways. In his recent PSL (Problem Solving Leadership) course I was on, Jerry showed us how we each have the opportunity to affect change and be a leader in our own right.

In agile teams, we try to level the playing field. We try not to get hung up on titles because everyone has an important part to play. We let team members try new things so that everyone develops new skills and have the opportunity to take a leadership role – no matter how small it may be.

So, when you ask me who are the senior people on a team, I will answer “They are the ones to whom the juniors ask the questions”. Sometimes it might be you asking me, but sometimes it might be me asking you.

Communication & Language Barriers

2009-03-07T12:19:00.002-07:00

I’ve just finished speaking at a test conference (link to softwaretest) in Copenhagen, Denmark and came away with some incredible learnings and not all test related.

First, I found out I still can’t understand a word of Danish, but that didn’t really matter because everyone spoke English to me (I envy people with fluent second languages). However, over the course of two days, I realized that most, if not all the participants spoke Danish when they were not speaking directly to me. Rightly so, since it is the language they are most comfortable with.

During my workshop, I had the participants break in to groups for discussion. All but one (who had an English only person at their table), promptly started talking in Danish. I found I was a bit lost as I am used to listening and guiding the discussions if needed. I had absolutely no idea of what was being discussed, and had to hope if the groups got stuck, they would ask for help. I didn’t need to worry, as they did fine, and when the groups presented their findings, they all did it in English.

The next day, I had an interesting conversation with Thomas, one of the local conference participants. We started talking about my language issues, and he related some of his experiences working with teams from other countries. One example he shared, was when they had meetings with teams from Finland or India, both teams switched to English for the meeting as it is the common language. This means both teams are speaking in their second language.

Thomas, who speaks excellent English, believes he cannot express about 15% of his ideas in English. If he is like me, he probably understands more than he can express, but still misses some words. However, if he is listening to someone who has trouble expressing his (or her) ideas as well, is the message really being understood? I assume this is typical for most people, so I have to wonder how much information is being lost in translation when both teams are speaking in their second language.

Then there is the other case where one team speaks English as their native tongue. Thomas explained that that sometimes the Danish team felt they couldn’t express themselves well enough to compete or argue against an idea brought up by one of the strong members of the American team. This might be partly a cultural issue, as well as a language barrier.

As a facilitator, I watch for team members who are quiet and not normally expressive. I try to find ways to make them feel comfortable enough to express their ideas safely. As members of global teams, I think we need to be aware of team members who may be at a disadvantage because of language. Something to consider might be engaging a facilitator for every meeting that involves two teams that may have communication or cultural barriers to overcome.

Another partial solution to the language problem may be to use more concrete examples to take away any misinterpretation. Technology enables teams to collaborate in many ways so we should take advantage of that.

Communication is a problem with any teams and when we add language issues, it adds even more complexity. I for one will be much more aware of what I say, and how I say it when I am speaking with teams whose native tongue is not the same as mine.

When does feedback or a discussion become an argument?

2009-02-19T12:03:00.005-07:00

Someone told me that blogs should be personal. I also believe they should be relevant to your readers, and not just a complaint session. This may be one of my few very personal blogs you will see.

I wrote an article ago with Lisa Crispin on “Testers: The Hidden Resource”, and posted to a yahoo group asking for feedback. What I was looking for, was a response to our ideas. Did it ring a bell with anyone? Did our ideas really transfer over to non-agile teams? We thought so, but wanted other people’s opinions.

I have been a proponent of constructive feedback for years, both on teams and personally. I guess I expected constructive criticism from the group (wrong assumption). Don't get me wrong, we did get some of good feedback, but not all was constructive. That leads me to the topic of this blog.

What is the difference between a debate, discussion, feedback and an argument? My adhoc definitions (not from a dictionary) as I think of them are:

Debate: opposing views with rules
Discussion – not necessarily opposing views, just talking about a specific topic
Feedback – giving an opinion on a subject to a person
Argument (as in quarrel): opposing views without rules. Without rules, people play on emotions, and might use any form of one ‘upmanship’ they can find.

When I give feedback to people, I have been trained to watch how people respond to what I say. In email or in forums or yahoo groups, we don’t have the ability to watch how people respond. There are some not so subtle ways by using CAPS to make a point (email etiquette says that is yelling). But mostly, we can only take the words at face value.

As a person giving feedback, I cannot control how people respond to how I say things so I usually am pretty careful, but like everyone, I can trigger negative responses without meaning to. In our article, we triggered a few negative responses – not unexpected. What was unexpected was the ‘attack’… at least how I perceived it.

It is not necessarily what you say but how you say it. If you don’t care what the other person’s feeling, then discussions can quickly deteriorate into what I think of as arguments. Of course, there are people, like me, who react negatively to these types of attacks, and tend to shut down. I personally dislike arguments and yelling – I claim childhood issues. I have had my share of arguments and have not found them very fulfilling or useful. It is a no win situation. There are many reasons why people don’t want to participate in heated discussions and it doesn’t necessarily mean they are wimps for not participating in them.

On a personal level, I can take negative feedback, as long as it is not what I perceive as a personal attack or threat against me or a friend. I am the first one to say - it's not personal, but threats trigger an emotional response whether it is on a public forum or in a private email. I find I would rather shut down the discussion rather than enter into what I perceive as an no win situation or argument. It was suggested that some people might think I was wimp if I chose to do this. Maybe I am, but I can live with it.

I personally don’t read discussions that I perceive fall into a “contest of wills”, so I’m not sure why I would be expected to actually participate in something I don’t believe in.

So, back to the original question? When does feedback degrade into an argument? I think it is whenever someone takes the feedback as a personal attack, whether it is meant that way or not, and then plays to win.

Testers: The Hidden Resource

2009-02-12T12:11:00.002-07:00

An article by Lisa Crispin and myslef just got posted on InformIT - “Testers: The Hidden Resource“. This was an idea that popped into Lisa's head one day when she was driving to work, after we had just finished the book. We think testers have a lot more to offer than most development organizations realize. See what you think, I’d love to hear your comments. Are we too optimistic or do you agree?

Check Lisa's website to see other comments as well.

Traceability

2009-02-09T19:34:00.004-07:00

At the beginning of February, there was a discussion on the agile testing yahoo group on traceability. The original question was “How do you know what piece of code affects what user stories and what documentation and what tests?” since agile values working code over documentation. In our book, Lisa and I have less than a page on traceability and the first paragraph recognizes that traceability matrices were pretty much the only way to know what had changed and when in a traditional project. In agile those restrictions aren’t there. We have many other methods that provide an automatic trace from story to story test to unit test to code. These tests become part of the regression suite so that every time they are run, the story is tested.

If we think about that, why do we need to link the code changes to user stories? User stories, requirements, features evolve over time. Each story or feature builds on previous ones so the likelihood of a test being in sync with a user story is low. The need no longer exists. If you are using a continuous integration tool, you can see what code is being changed on every check-in if you really want to. It is easy enough to add in defect numbers in check in comments so that even defects could be traced if needed.

In my experience I have found the request for traceability often comes because history has shown that side effects occur because the developers haven't told the testers what exactly was changed and the final testing often showed up unanticipated problems. In agile, each story is tested as it is developed, with unit tests and acceptance tests and any other tests that it required (including performance, etc). If there are issues you find them right away with your suite of automation. I used to be a fan of traceability because it was the only way to understand what was changed. I have found I don't need it anymore.

Sometimes auditors will request traceability for regulated industries. If they don’t understand how version control works, or the power of automated tests, you may want to show them how you have automated traceability, and explain the cost of maintaining any type of manual traceability.

A related discussion about 7months ago on "Traceability Matrix in an Agile Project" and was summarized by Vikas Hazrati at http://www.infoq.com/news/2008/06/agile-traceability-matrix. One of the main points I got from that discussion is the difference between traceability and a traceability matrix. A matrix is just one implementation of the need for traceability. I also liked Michael Bolton’s reference “Transparency over Traceability" in his 2007 article entitled "Lean Traceability: a smattering of strategies and solutions" at http://www.cmcrossroads.com/content/view/9089/264/.

Before you blindly start a traceability matrix, ask the reason for the request and question the cost of maintenance against the benefit. In regulated environments look for the simplest thing that meets the need to provide evidence that the software works as specified. It may not need a full traceability matrix. Use your source control to your advantage. Use naming conventions on your tests that help identify what is being tested, or perhaps even integrate your requirments into your automation framework. Think simple to keep maintenance as low as possible.

Review: The Gift of Time

2008-12-23T07:22:00.003-07:00

I started reading “The Gift of Time” with certain expectations, but as I kept reading, I found those expectations challenged.

I thought I would have a nice easy read about tributes to Jerry Weinberg. Instead, I found myself jotting down words and phrases as I was reading each of the essays. As a life long learner and a dabbler in the areas of systems thinking and organizational behaviour, I found so many tidbits of information and from so many perspectives, that my thoughts started running rampant. I wanted to go start researching and reading more.

And it wasn’t only about systems thinking. For example, the essay by James Bach got me thinking again about basic testing premises. The one by Naomi Karten about experiential workshops made me think about my own tutorials and workshops and how I could improve the exercises I use. Congruent feedback by Ester Derby gave me a new way of thinking about giving feedback and understanding the importance of context... yet again. The list goes on.

I am sure everyone who reads it will pick up new ideas to research or will revisit some that need renewing. So much of what Jerry Weinberg has given the world is summed up so nicely in this small book of essays.

Is agile like eating pizza?

2008-12-02T15:56:00.000-07:00

A few weeks ago I was talking to someone from the Netherlands who mentioned an analogy that Lee Copeland used when referring to agile development. It tweaked my interest so I emailed Lee to find out what he was really thinking.

He used the analogy when the group was talking about all of the work to actually get a product out the door into production, and which parts agile development ignored. He compared agile development (Scrum + XP, for example) to how his kids eat a slice of pizza. They start on the pointy end and eat up toward the crust for a while, then toss the slice down and start on another. All the best stuff (meat and cheese) is near the pointy end. That's how he views of agile development -- they do the fun stuff (write the code) while ignoring the "less-tasty" stuff (product planning, scope, design, inter-product relationships, ...).

I thought on this for a while – my gut reaction to was to deny, deny, deny … but realized he is right in many cases. Some teams who start agile development really want all the good stuff – that yummy cheesy mess in the middle of the pizza – the best part. They want the iterative cycles, the daily stand-ups, less documentation, and the small releases. However, they often forget what it really means to release, and concentrate on the project rather than the product.

It may be fairly simple if you are just updating an internal website, or even your own managed external website. But… if you need to release to a client(s), it is a different story. It may be a shrink-wrapped product that is destined for the stores or a custom enterprise application and then the game is different. There is actually a whole lot more to releasing than just giving the code over. This is some of what Lee was talking about.

Agile development has progressed over the years and I think we’ve all come a long way since we first started talking about it. Yes, some teams forget about the “crust”, but I think they all learn very quickly that they can’t just throw it over the wall to customers. For example, they need to understand that customers may not be able to accept as frequent as they want to release because they have so many intergration issues. Agile teams adapt and change, and many are doing it right, and customers are benefiting. The rest... well, they will learn too.

First Posting - Trust

2008-10-29T21:29:00.000-06:00

My first blog… I’ve been meaning to start this for a long time, but other things have kept me busy … things like writing a book. Now that it’s almost done, I can turn to other things I have been neglecting, like this.

Something that has captured my interest in the last few months is the impact of collaboration, or should I say lack of collaboration in a tester’s life. Trust plays such an important part of any team, and I believe that without trust, it is hard to have a truly collaborative team.

I have recently experienced what I think is lack of trust. I do a lot of questioning about stories so that I know I understand what is required for testing. I have one teammate who seems to resent my questioning, and becomes defensive. I believe the underlying problem is lack of trust. He thinks I question because I don’t believe he knows what he is doing. I question, partly because I haven’t heard the questions asked out loud, so not sure we are all on the same page.

Is this lack of trust? I think so, and it makes me realize how absolutely important it is for a team to have truly successful collaboration.