Testing the Big Picture

One of the top seven success factors that Lisa Crispin and I talk about in the summary chapter of our book Agile Testing: A Practical Guide for Testers and Agile Teams is "Look at the Big Picture". We talk about using the agile testing quadrants to help think about all types of testing, and how to keep the customers point of view.


This means understanding what problem the customer is trying to solve with each new feature. One practice I have been coaching teams to try, is defining acceptance tests at the feature level (or theme / epic if you are practicing Scrum). ATDD (Acceptance Test Driven Development), BDD (Behavioural Driven Development) or Specification by Example all talk about defining tests at the story level. We often forget about the feature level – the bigger picture. How do we know when the feature is “DONE”?

In specification workshops, we start with a feature, decompose it into stories, and often lose or forget to keep that feature in mind when testing. Now, if we created acceptance tests – both desired and undesired behaviour – at the feature level, then have something to test.

When we decompose the feature into stories, I suggest creating a story to “Test Feature A”, with the corresponding acceptance tests. I have found the idea of creating a “Feature DONE” definition to be very powerful. There are many tests that do not make sense to do at the story level, but apply to the feature. Many of the Quadrant 4 (technical facing tests that critique the product) fall into this category. Some examples are load tests, usability, and browser compatibility.

The ‘Test Feature’ story is prioritized after all the individual stories are DONE. Tasks for this story might include: Test browser compatibility, Perform load test, Automate GUI tests, Create user documentation, and Perform UAT (User Acceptance Test). Testing at the feature level enables us to run these tests when it makes the most sense. End users can be given the chance to test the complete feature and give feedback as soon as it is complete.

Adding acceptance tests to the feature, defining Feature DONE to include the tasks and tests that make sense at this level helps teams to consider the big picture and the business problem that they are trying to solve

Choosing Functional Test Tools for Large Organizations

In one of my tutorials at StarWest 2011, someone asked the question about choosing tools – specifically functional automation tools. I do believe teams should chose the tools that work best for their application, but sometimes we have to think beyond our immediate project team.


As large organizations adopt agile, many are running into a common issue where each team wants to choose their own tool – after all, they are supposed to be self-organizing? I think this might be a bit short sighted.

One of the premises of eXtreme Programming is shared code ownership, and if you ask any programmers working on one product, they will all be using the same tool for their unit tests. For example, jUnit if they are coding Java. They may use different IDEs (editors) but the unit testing framework remains the same. We need to apply some of those ideas to choosing our functional test tools.

If we think about an extended team, say one that included the support or operational folks, would we still want each individual project team selecting a different tool? If multiple teams are working on the same product, can we really support the automated regression suite mixed and matched arbitrarily?

One suggestion I have for organizations facing these problems is to define a class of tools for teams to select from. For example, for API level tests, you can choose from FitNesse or Cucumber or for tests through the GUI, Selenium or Ruby and Watir. One way to determine these selections is to have product teams experiment with several tools, and then compare pros and cons, and select one for each type of testing they want to perform.

It is important to understand who will using them, and the teams they impact.

I am a member of the Community of Thinkers

Jean Tabaka, Liz Keogh and others devised this statement and to me, it expresses the power behind a community of like thinkers. Patrick Wilson-Welsh mentions the idea of Network Weaving on his blog, and I like that idea as well; that also resonatated with me.

Thank you Jean, Liz, Eric, and others for sharing.

--------------------------------------------------------------------------------

A Community of Thinkers

I am a member of a community of thinkers.

I believe that communities exist as homes for professionals to learn, teach, and reflect on their work.

I challenge each community in the software industry to:

• reflect and honor the practitioners who make its existence possible;
• provide an excellent experience for its members;
• support the excellent experience its members provide for their clients and colleagues in all aspects of their professional interactions;
• exemplify, as a body, the professional and humane behavior of its members;
• engage and collaborate within and across communities through respectful exploration of diverse and divergent insights;
• embrace newcomers to the community openly and to celebrate ongoing journeys; and
• thrive on the sustained health of the community and its members through continual reflection and improvement.

I believe that leaders in each community have a responsibility to exhibit these behaviors, and that people who exhibit these behaviors will become leaders. 

I am a member of a community of thinkers. If I should happen to be a catalyst more than others, I consider that a tribute to those who have inspired me.

--------------------------------------------------------------------------------

”A Community of Thinkers” by Liz Keogh, Jean Tabaka and Eric Willeke is licensed under a Creative Commons Attribution-Share Alike 3.0 License. Please attribute to the distributor of your copy or derivative.