Agile Glossary

Acceptance Testing

What is Acceptance Testing?

An acceptance test is a formal description of the behavior of a software product, generally expressed as an example or a usage scenario. A number of different notations and approaches have been proposed for such examples or scenarios. In many cases, the aim is that it should be possible to automate the execution of such tests by a software tool, either ad-hoc to the development team or off the shelf.

Similar to a unit test, an acceptance test generally has a binary result, pass or fail. A failure suggests, though does not prove, the presence of a defect in the product.

Teams mature in their practice of agile use acceptance tests as the main form of functional specification and the only formal expression of business requirements. Other teams use acceptance tests as a complement to specification documents containing use cases or more narrative text.

The terms “functional test”, “acceptance test” and “customer test” are used more or less interchangeably. A more specific term “story test”, referring to user stories is also used, as in the phrase “story test driven development”.

Acceptance testing has the following benefits, complementing those which can be obtained from unit tests:

  • encouraging closer collaboration between developers on the one hand and customers, users, or domain experts on the other, as they entail that business requirements should be expressed
  • providing a clear and unambiguous “contract” between customers and developers; a product that passes acceptance tests will be considered adequate (though customers and developers might refine existing tests or suggest new ones as necessary)
  • decreasing the chance and severity both of new defects and regressions (defects impairing functionality previously reviewed and declared acceptable)

Expressing acceptance tests in an overly technical manner

Customers and domain experts, the primary audience for acceptance tests, find acceptance tests that contain implementation details difficult to review and understand.  To prevent acceptance tests from being overly concerned with technical implementation, involve customers and/or domain experts in the creation and discussion of acceptance tests. See Behavior Driven Development for more information.

Acceptance tests that are unduly focused on technical implementation also run the risk of failing due to minor or cosmetic changes which in reality do not have any impact on the product’s behavior.  For example, if an acceptance test references the label for a text field and that label changes, the acceptance test fails even though the actual functioning of the product is not impacted.

Unlike automated unit tests, automated acceptance tests are not universally viewed as a net benefit and some controversy has arisen after experts such as Jim Shore or Brian Marick questioned whether the following costs were outweighed by the benefits of the practice:

  • many teams report that the creation of automated acceptance tests requires significant effort
  • sometimes due to the “fragile” test issue, teams find the maintenance of automated acceptance tests burdensome
  • the first generation of tools in the Fit/FitNesse tradition resulted in acceptance tests that customers or domain experts could not understand.

The BDD approach may hold promise for a resolution of this controversy.

  • 1996: Automated tests identified as a practice of Extreme Programming, without much emphasis on the distinction between unit and acceptance testing, and with no particular notation or tool recommended
  • 2002: Ward Cunningham, one of the inventors of Extreme Programming, publishes Fit, a tool for acceptance testing based on a tabular, Excel-like notation
  • 2003: Bob Martin combines Fit with Wikis (another invention of Cunningham’s), creating FitNesse
  • 2003-2006: Fit/FitNesse combo eclipses most other tools and becomes the mainstream model for Agile acceptance testing

For a comprehensive survey, see Automated Acceptance Testing: A Literature Review and an Industrial Case Study

Add to Bookmarks Remove Bookmark
Add to Bookmarks Remove from Bookmarks
Add to Bookmarks Remove from Bookmarks

Thank you to our Annual Partners​

Join us today!

Agile Alliance offers many online and in-person events and workshops for our members. If you’re not currently a member, you can join now to take advantage of our many members-only resources and programs.

Get the latest Agile news!

  • This field is for validation purposes and should be left unchanged.

By subscribing, you acknowledge the Agile Alliance Privacy Policy, and agree to receive our emails.

Additional Agile Glossary Terms

Test-driven development (TDD) is a style of programming where coding, testing, and design are tightly interwoven. Benefits include reduction in defect rates.
The team meets regularly to reflect on the most significant events that occurred since the previous such meeting, and identify opportunities for improvement.
A product backlog is a list of the new features, changes to existing features, bug fixes, infrastructure changes or other activities that a team may deliver in order to achieve a specific outcome.
Backlog grooming is when the product owner and some, or all, of the rest of the team refine the backlog on a regular basis to ensure the backlog contains the appropriate items, that they are prioritized, and that the items at the top of the backlog are ready for delivery.
Test-driven development (TDD) is a style of programming where coding, testing, and design are tightly interwoven. Benefits include reduction in defect rates.
The team meets regularly to reflect on the most significant events that occurred since the previous such meeting, and identify opportunities for improvement.
A product backlog is a list of the new features, changes to existing features, bug fixes, infrastructure changes or other activities that a team may deliver in order to achieve a specific outcome.

Help us keep the definitions updated

Discover the many benefits of membership

Your membership enables Agile Alliance to offer a wealth of first-rate resources, present renowned international events, support global community groups, and more — all geared toward helping Agile practitioners reach their full potential and deliver innovative, Agile solutions.

Not yet a member? Sign up now