JBehave Yourself

JBehave Yourself

By Tim Dalton, OCI Senior Software Engineer

June 2010

Introduction

JBehave provides a framework for doing Behavior-Driven Development (BDD) on the Java platform. Behavior-Driven Development is an extension of Test-Driven Development (TDD) that focuses on facets of behavior rather verifying 'units' whether they be methods, classes or other. The emphasis is on specification rather than verification.

One goal of BDD is to provide a ubiquitous language for specifying expected behavior that can be executed as part of an acceptance test and can be understood by developers and stakeholders alike. Therefore, BDD tests have identifiable and verifiable business value.

The form that the specification language takes can vary across BDD frameworks. Many favor a Domain Specific Language (DSL) implemented using plain text while others use an internal DSL formed from features of a host language. This abstraction layer, especially for those using plain text, makes it easier to de-couple the specification from the underlying implementation. Code-based specifications are likely to appeal to other developers since features of a programming language are available such as binding repeated constants to some kind of symbol. Plain text specifications are more business-user friendly since no understanding of a programming language is needed.

Approach of JBehave

JBehave uses a natural language stored in a text file. The contents of one such specification file is also referred to as a story. The top of a JBehave specification is an optional description that can contain narrative phrases beginning with 'As a', 'I want to' or 'In order to'. These contents are there for documentation purposes only and do not alter the behavior of JBehave in any way.

Stories in JBehave are composed of scenarios indicated by a phrase beginning with 'Scenario:'. Scenarios employ three key words, Given, When and Then, at the beginning of phrases within the scenario. These keywords mirror the Arrange-Act-Assert pattern often used in unit testing. Phrases beginning with these keywords are also referred to as steps.

Below is an example scenario for a hypothetical user interface:

Scenario: Unsuccessful Login
    Given the user has entered username 'foo' 
    When the Login button is pressed
    Then an alert dialog 'Please provide a password' is displayed

The 'Scenario:' keyword can be followed by free form text, including new lines, until a line that begins with one of the keywords, Given, When or Then, is reached.

Comments can be nested within the Given, When and Then steps starting with '!--'. Such comments are terminated by the end of the line just like '//' comments in Java.

There is also supplemental keyword, And, that is used to compose multiple instances of the other three.

Example using And keyword and a comment:

Scenario: Successful Login
    Given the user has entered username 'foo'
    And the user has entered password 'bar' !-- comment
    When the Login button is focused
    And enter key is pressed
    Then a page with title 'Main Page' is displayed
    And text 'Welcome foo' is visible

Use of And allows easy composition of various permutations of test arrangements in a natural language readable by humans.

JBehave Scenarios and Steps

JBehave executes textual steps found in scenarios using classes called Steps. Such classes can extend a default implementation of a Steps class within JBehave or can be POJO's (Plain Old Java Objects). Within these classes, there are annotated methods that are invoked when they match a step in a scenario.

Multiple instances of objects implementing steps can be registered with an instance of Scenario. Such instances are evaluated to see if there is a matching method. The Scenario class has a one-to-one mapping to the text file containing the story.

Example: Bowling Behavior

To demonstrate the basics of JBehave, the behavior of a simple function that calculates bowling scores is going to be validated here.

First Scenario:

Scenario: No score after one roll
    Given a strike is bowled
    When scores are tallied
    Then there should be no score at frame 1

The default method of mapping Scenario classes to file names converts the camel case class name to an underscore separated file name with no extension. This file is required to be in the same package as the Scenario class. This mapping scheme can be customized by providing different implementations of ScenarioDefiner and ScenarioNameResolver interfaces within JBehave.

Scenarios

Here is a Scenario class that maps to a story file using the defaults:

  1. package com.ociweb.jnb.jun2010.java.bowling
  2.  
  3. public class BowlingScenario extends Scenario {
  4.     public BowlingScenario() { super(); }
  5. }

This story for the example above will need to on the classpath in this file, "./com/ociweb/jnb/jun2010/java/bowling/bowling_scenario".

Scenario is a descendant of junit.framework.TestCase, so it can be run just like any JUnit test. JUnit will execute a testScenario method implemented by Scenario that will run all scenarios in the file. When the above scenario is run it will produce console output like:

(tfd/java/bowling/bowling_scenario)
Scenario: No score after one roll
Given a strike is bowled (PENDING)
When scores are tallied (PENDING)
Then there should be no score at frame 1 (PENDING)

When a step is (PENDING), it indicates that no matching handler method for it was found in any of the Steps instances associated with the Scenario. Here, no Steps instances have been created for the Scenario so all the steps in the scenario will indicate as being (PENDING). Steps that have been implemented but depend on a pending step will indicate (NOT PERFORMED). The default behavior of JBehave is to not fail the test for any pending step, but this can be changed via configuration.

Steps

Here is an example of a Steps class with an annotation to handle the 'Given a strike is bowled' step above:

  1. public class BowlingSteps extends Steps {
  2.     private ArrayList<Integer> rolls;
  3.  
  4.     @BeforeScenario
  5.     public void resetRolls() {
  6.         rolls = new ArrayList<Integer>();
  7.     }
  8.  
  9.     @Given("a strike is bowled")
  10.     public void aStrikeIsBowled() {
  11.         rolls.add(10);
  12.     }

The @BeforeScenario annotation is very similar to a JUnit @Before annotation. It indicates that a method is to be executed when the 'Scenario:' keyword is encountered in the text specification in a story. There is also an @AfterScenario annotation to invoke a method after the scenario is complete. To invoke methods at the beginning of a story or upon its completion, use the @BeforeStory or @AfterStory annotation. Instances of Steps classes are reused by the Scenario object and therefore stateful. Care must be taken to reset the object to a known state before executing steps within a scenario.

The @Given annotation indicates that a method is invoked when a Given step is encountered that matches its value.

An instance of a Steps class can be be associated with the Scenario in various ways. Here a single instance is added via a constructor parameter:

public BowlingScenario() { super(new BowlingSteps()); }

Executing the BowlingScenario test will now show that the Given step is no longer (PENDING):

(tfd/java/bowling/bowling_scenario)
Scenario: No score after one roll
Given a strike is bowled
When scores are tallied (PENDING)
Then there should be no score at frame 1 (PENDING)

A few more methods need to be implemented in the BowlingSteps class to be able to run the scenario completely. First, a field to hold the result of the evaluation of the rolls is needed along with a @BeforeScenario method to reset its value:

  1. // Validation contains two disjoint types, one indicates
  2. // failure and other indicates success
  3. private Validation<InvalidPinCountException, ArrayList<Integer>> scoreResult;
  4. ...
  5. @BeforeScenario
  6. public void resetScoreResult() {
  7.     scoreResult = null;
  8. }

Here are the methods that match the When and Then steps:

  1. @When("scores are tallied")
  2. public void scoreAreTallied() {
  3.     try {
  4.         scoreResult = Validation.success(new ScoreBowlingGame().f(rolls));
  5.     } catch (InvalidPinCountException ex) {
  6.         scoreResult = Validation.fail(ex);
  7.     }
  8. }
  9.  
  10. @Then("there should be no score at frame $frame")
  11. public void noScoreAtFrame(Integer frame) {
  12.     assertThat(scoreResult.isSuccess() &&
  13.         scoreResult.success().length() < frame, is(true));
  14. }

Assertions in the above example are ones from the Hamcrest project. JBehave has built-in support for Hamcrest but allows the use of other assertion libraries.

Step Arguments

JBehave matches steps by converting the text value of the annotation to a regular expression. Argument specifications are words that begin with '$' and are replaced with '(.*)' to form regular expression groups that can be extracted. For the previous example, the contents of the @Then annotation above would become 'there should be no score at frame (.*)' which allows the argument for $frame to be extracted.

In the example above, JBehave will convert the argument into an Integer. There are built-in converters for String and child classes of Number. Comma-delimited text can be matched to a java.util.List of String and Number descendants as well. Custom converters can be registered with JBehave during scenario configuration.

By default, arguments are matched in the order they appear in the specification step. Example:

  1. @Then("the score at frame $frame should be $score")
  2. public void scoreAtFrameShouldBe(Integer frame, Integer score) {
  3.     assertThat(scoreResult.isSuccess(), is(true));
  4.     assertThat(scoreResult.success().get(frame - 1), is(score));
  5. }

The names of the method or parameters do not play a role in matching steps. It is still a good idea to have the parameter names match the argument names in the annotation and have a method name congruent with the function of the step.

Step Aliases

When multiple steps with the same meaning need to be matched to a single method, JBehave provides @Alias and @Aliases annotations. Example:

Scenario: No score after spare
Given an 8 count is bowled and spare is made
And a 7 count is bowled and only 2 are picked
When scores are tallied
Then the score at frame 1 should be 17
And the score at frame 2 should be 26

Methods to handle steps:

  1. @Given("an $count count is bowled and spare is made")
  2. @Alias("a $count count is bowled and spare is made")
  3. public void countIsBowledAndSpare(Integer count) {
  4.     rolls.add(count);
  5.     rolls.add(10 - count);
  6. }
  7.  
  8. @Given("an $count count is bowled and only $next is picked")
  9. @Aliases(values = {
  10.     "an $count count is bowled and only $next are picked",
  11.     "a $count count is bowled and only $next is picked",
  12.     "a $count count is bowled and only $next are picked"
  13. })
  14. public void countIsBowledAndSomePicked(Integer count, Integer next) {
  15.     rolls.add(count); // add appends a value to a ArrayList
  16.     rolls.add(next);
  17. }

The @Alias and @Aliases annotations are used to support better grammar here.

Named Arguments

If there is a need to match step arguments in a different order than how they appear in the method parameter list, the @Named annotation can be used to map arguments to parameters by name. Example:

Scenario:
Given an 8 count is bowled and spare is made
And a 7 count is bowled and only 2 are picked
When scores are tallied
Then the score at frame 1 should be 17
And 26 the score at frame 2 should be !-- says Yoda

Here is the updated method:

  1. @Then("the score at frame $frame should be $score")
  2. @Alias("$score the score at frame $frame should be")
  3. public void scoreAtFrameShouldBe(@Named("frame") Integer frame,
  4.         @Named("score") Integer score) {
  5.     assertThat(scoreResult.isSuccess(), is(true));
  6.     assertThat(scoreResult.success().get(frame - 1), is(score));
  7. }

The @Named annotation overrides the default behavior of matching arguments to method parameters based on position.

Tabular Data

To prevent the need to duplicate scenarios in the specification for many different data points, JBehave supports Table Examples. Table Examples allows scenarios to be executed repeatedly using data specified in a tabular format in the story text. Example:

Scenario: Table
Given rolls are <rolls>
When scores are tallied
Then scores should be <scores>
 
Examples:
|rolls                 |scores
|9-                    |  9
|9/9                   | 19
|X X 9                 | 29
|X X X                 | 30
|X X X 8               | 30  58
|X X X 81              | 30  58  77 86
|X X X X X X X X X XXX | 30  60  90 120 150 180 210 240 270 300
|X X X X X X X X X XX9 | 30  60  90 120 150 180 210 240 270 299
|X X X X X X X X X XX- | 30  60  90 120 150 180 210 240 270 290
|9/X X X X X X X X XXX | 20  50  80 110 140 170 200 230 260 290
|X X X X X X X X X X9/ | 30  60  90 120 150 180 210 240 269 289
|X X X X X X X X X X9- | 30  60  90 120 150 180 210 240 269 288
|X X X X X X X X X X81 | 30  60  90 120 150 180 210 240 268 287
|X -/X X X X X X X XXX | 20  40  70 100 130 160 190 220 250 280
|X --X X X X X X X XXX | 10  10  40  70 100 130 160 190 220 250
|X 9/X 9/X 9/X 9/X 9/X | 20  40  60  80 100 120 140 160 180 200
|--------------------  |  0   0   0   0   0   0   0   0   0   0

The first row of a Table Example is interpreted as the header row that specifies the arguments used to run the scenario for each row in the table.

The rolls column can be parsed from a notation that resembles those used in bowling and the scores can be split from a space delimited string:

  1. @Given("rolls look like $rolls")
  2. @Alias("rolls are <rolls>")
  3. public void rollsLookLike(@Named("rolls") String rollString) {
  4.     rolls = new ArrayList<Integer>();
  5.     char[] rollChars = rollString.toCharArray();
  6.     Integer lastRoll = null;
  7.     for (int i=0; i < rollChars.length; i++) {
  8.         if (rollChars[i] == 'X') {
  9.             lastRoll = 10;
  10.         } else if (rollChars[i] >= '0' && rollChars[i] <= '9') {
  11.             lastRoll = Integer.parseInt("" + rollChars[i]);
  12.         } else if (rollChars[i] == '/') {
  13.             lastRoll = 10 - lastRoll;
  14.         } else if (rollChars[i] == '-') {
  15.             lastRoll = 0;
  16.         } else {
  17.             lastRoll = null;
  18.         }
  19.         if (lastRoll != null) { rolls.add(lastRoll); };
  20.     }
  21. }
  22.  
  23. @Then("scores should be <scores>")
  24. public void scoreShouldBe(@Named("scores") String expectedScores) {
  25.     assertThat(scoreResult.isSuccess(), is(true));
  26.     String[] scoreArray = expectedScores.split(" +");
  27.     assertThat(scoreResult.success().length(), is(scoreArray.length));
  28.     for (int i=0; i < scoreArray.length; i++) {
  29.         assertThat(scoreResult.success().get(i), is(new Integer(scoreArray[i])));
  30.     }
  31. }

Table examples require arguments to be enclosed in angle brackets (<>) in the annotation value and use of the @Named annotation.

When a Scenario using a Table Example is executed, output for each row in the table is generated with the values substituted in the steps of the scenario like below:

Example: {scores=9, rolls=9-}
Given rolls are 9-
When scores are tallied
Then scores should be 9
 
Example: {scores=19, rolls=9/9}
Given rolls are 9/9
When scores are tallied
Then scores should be 19
 
Example: {scores=29, rolls=X X 9}
...

Reporting

By default, JBehave outputs only to the console (System.out). Scenarios can configured to output in multiple formats and to be directed to a file. Format supported include the following:

Text

This is the default output format used in the output examples above. By default, only steps that are not runnable or fail are output. It can be configured such that output for all steps are echoed to the console or out to a file in plain text format.

HTML

Reports in HTML format simply use <div> tags with various values for their class attributes. Table Examples are converted to HTML <table> elements.

Output for all steps is generated.

Example:

  1. <div class="scenario">
  2. <h2>Scenario: Too many rolls such be ignored</h2>
  3. <div class="step successful">Given rolls look like XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</div>
  4. <div class="step successful">When scores are tallied</div>
  5. <div class="step successful">Then the score at frame 10 should be 300</div>
  6. </div>
  7. <div class="scenario">
  8. <h2>Scenario: Table</h2>
  9. <div class="examples">
  10. <h3>Examples:</h3>
  11. <div class="step">Given rolls are <rolls></div>
  12. <div class="step">When scores are tallied</div>
  13. <div class="step">Then scores should be <scores></div>
  14. <table>
  15. <thead>
  16. <tr>
  17. <th>rolls</th><th>scores</th></tr>
  18. </thead>
  19. <tbody>
  20. <tr>
  21. <td>9-</td><td>9</td></tr>
  22. <tr>
  23. <td>9/9</td><td>19</td></tr>
  24. ...

XML

Here elements such an story, scenario, step and example are used to tag results and again output for all steps is generated:

  1. <scenario keyword="Scenario:" title="Too many rolls should be ignored">
  2. <step outcome="successful">Given rolls look like XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</step>
  3. <step outcome="successful">When scores are tallied</step>
  4. <step outcome="successful">Then the score at frame 10 should be 300</step>
  5. </scenario>
  6. <scenario keyword="Scenario:" title="Table">
  7. <examples keyword="Examples:">
  8. <step>Given rolls are <rolls></step>
  9. <step>When scores are tallied</step>
  10. <step>Then scores should be <scores></step>
  11. <parameters>
  12. <names><name>rolls</name><name>scores</name></names>
  13. <values><value>9-</value><value>9</value></values>
  14. <values><value>9/9</value><value>19</value></values>

Statistics Report

Not a report format, but rather it is a summary of statistics gathered during execution of a JBehave story. Example:

#org.jbehave.scenario.reporters.StatisticsScenarioReporter
#Wed May 19 20:08:34 CDT 2010
stepsFailed=0
examples=17
scenariosFailed=0
scenarios=28
givenScenarios=0
stepsPending=0
stepsIgnorable=0
steps=141
stepsNotPerformed=0
stepsSuccessful=141

Other Features

Here is a summary of JBehave features not covered here:

Table Parameters

When tabular data is required to passed as an argument to a step, JBehave supports parameters using the same '|' delimited format used by Table Examples. A parameter of class ExampleTable is passed into the matching method. The ExampleTable provides a List> where items of the List represents the rows and Map elements allow the lookup of data values based on header names.

Non-English Language support

Specifications can be written in languages other than English by using an I18n locale file and configuring the Scenario to use it.

Plugins for Dependency Injection Frameworks

JBehave support configuration of Steps using the popular dependency injection frameworks for the Java platform. This allows the injection of mocks and stubs of external interfaces required for testing. Frameworks include:

Ant and Maven integration

JBehave has Ant tasks ScenarioRunnerTask and ReportRendererTask for running the scenarios and rendering reports. For Maven, the jbehave-maven-plugin plugin can be used. The Maven plugin provides goals run-scenarios and render-reports for those same purposes.

Web Integration

Packaged separately from JBehave, JBehave Web provides a web-integration extension that consists of to distinct parts:

Given Scenarios

Provides a mechanism for stories to include scenarios from other files. A GivenScenario can be used within a Scenario before the first Given step.

Summary

JBehave provides a lot of benefits while being relatively simple. The ability to compose arrangements, actions and assertions using plain text provides great flexibility. Use of frameworks like JBehave facilitate the 'tests as documentation' ideal valued by Agile programming practices.

Source Code

All source examples use Maven 2 and are set up to run JBehave scenarios as part of the test phase.

References