ATDD vs. BDD vs. Specification by Example vs ....

At Agile 2010, there were about 20 of us at the AA-FTT (Agile Alliance Functional Test Tools) workshop. The emphasis was on “the state of the practice” of Acceptance Test Driven Development (ATDD). In one of the breakout sessions, we had an interesting discussion on what we actually meant by ATDD, which made me think a lot about the practice itself. Although we all knew that acceptance tests defined by the customer / business expert is what drives the TDD (test driven development) process, there were discrepancies in how each of us thought of full process.

JB Rainsberger had considered ATDD as two concentric circles with TDD, the developer practice of Test Driven Development, to be the center, with ATDD surrounding it. I felt his view of the practice was from a developer perspective – give me an example, and I will start coding. Many of us at the table thought it was more than that; it included the collaboration necessary to get the examples. Based on that discussion, the next question asked by JB was, “Then, what’s the difference between ATDD and BDD?”

Part of the problem is that we don’t all use the same vocabulary relating to the practice. I started by writing down all the words I know are used in conjunction with the practice. The following words and phrases are some of the ones I could think of:

• Examples, scenarios, acceptance tests, customer tests, behaviour, business-facing tests, story tests, functional tests, acceptance criteria, conditions of satisfaction, business rules, executable specification

There have been many explanations about ATDD and I’ve included a couple here to help explain.

Jennitta Andrea wrote a short article for Iterations about what ATDD is, and why teams should be doing it. She described ATDD as:

“

the practice of expressing functional story requirements as concrete examples or expectations prior to story development. During story development, a collaborative workflow occurs in which: examples written and then automated; granular automated unit tests are developed; and the system code is written and integrated with the rest of the running, tested software. The story is "done"—deemed ready for exploratory and other testing—when these scope-defining automated checks pass”

The full article is at: http://www.stickyminds.com/Media/eNewsLetters/Iterations

A couple of years ago, Elisabeth Hendrickson used an example to describe ATDD in her blog: http://testobsessed.com/2008/12/08/acceptance-test-driven-development-atdd-an-overview/

As a result of the AA-FTT workshop, Declan Whelan, Gojko Adzic and a few others came up with a diagram trying to get a common understanding around Specification by Example. It is on Google Docs - http://tinyurl.com/3x52ap9

Dan North’s description of Behaviour Driven Development (BDD) http://blog.dannorth.net/introducing-bdd/ is a bit more complicated, but I interpret it as using the word ‘behaviour’ rather than ‘test’ , and use the “Given, When, Then” guidelines to describe the precondition, trigger and post-condition. In a recent agile testing yahoo post, Liz Keogh posted this:

“BDD's focus is on the discovery of stuff we didn't know about, particularly around the contexts in which scenarios or examples take place. This is where using words like "should" and "behaviour" comes in, rather than "test" - because for most people "test" presupposes that we know what the behaviour ought to be. "Should" lets us ask, "Should it? Really? Is there a context which we're missing in which it behaves differently?"

Where we choose to call it BDD or ATDD or Specification by Example, we want the same result – a shared common understanding of what is to be built to try to build the ‘thing’ right the first time. We know it never will be, but the less rework, the better.

Using business rules, use cases, business expertise (customers, Business Analysts, domain experts, etc), the development team’s (programmers, testers, DB guys, etc.) expertise, and any other supporting information, we can describe what we expect the software to do in the format of examples which are just a specialized test. In workflow type applications, describing behaviour may be applicable. If it is calculation heavy, then a spreadsheet type test format with specific examples might be more appropriate.

Do these tests have to be automated first? I encourage new teams that are struggling to find the right tool, to focus on the purpose of ATDD which is to drive development to deliver the right ‘thing’. If we can do that with a sentence, let’s do that first. We do have to automate them at sometime during the iteration, but until a team has their rhythm or cadence, the automation may not come first. Once you have the automation, you then have documentation that is up to date and describes the system behaviour.

The picture I use to describe ATDD for new teams is:

I think the story isn’t ‘Done’ until the customer has actually validated the actual completed story which includes pairing with the developer after he’s completed coding for a first look, and the exploratory testing.
I will continue to use the phrase Acceptance Test Driven Development (ATDD) unless the industry decides on a common vocabulary, because I find that business stakeholders not only understand how to give examples, but also understand when I talk about acceptance tests that prove the intent of the story or feature. Once we have that, the team knows enough about the scope to start coding and testing. That's what's important.