Overview and Installation

Module Overview

Hi I'm Jason Roberts from Pluralsight, welcome to this course on Automated Acceptance Testing with

SpecFlow and Gherkin. In this module we'll get an overview of what SpecFlow and Gherkin are and

how we can use them to define acceptance tests that we can automate. In this module we'll get an

overview of the SpecFlow tool itself. We'll discuss acceptance tests in general and some of their

characteristics. We'll learn how we can broadly categorize testing into business facing and technology

facing tests. We'll see how SpecFlow can be used in test-first approaches. And we'll look at the

structure of SpecFlow tests and we'll breakdown the different components. We'll look at how we can

install the Visual Studio IDE integration for the SpecFlow tooling. And finally we'll create a new project in

Visual Studio and add our first feature file to it.

What is SpecFlow?

So what is SpecFlow? SpecFlow is an open-source tool for bridging the communication gap between

domain experts and developers. It essentially allows non-technical people, such as product owners or

business analysts, to write acceptance tests for the system that we're developing. SpecFlow enables

these acceptance tests to then be automated and allows them to be executed against the system we're

building to verify that the acceptance tests pass. The project site for SpecFlow is at www. specflow. org.

What is an Acceptance Test?

What is an acceptance test? An acceptance test enables us to validate that we're building the right

system and that the system contains the right features that are required by the users, the business, or

the product owner. Acceptance tests are usually from the point of view of the business or the user or the

customer point of view, but we may have a proxy for these people in the role of the product

owner. Because acceptance tests are from the point of view of the business, they should be written in a

1
non-technical format. This format can have some formatting rules to enable it to be translated

into automatable steps, but it should be an easy to understand and use format. Acceptance tests, when

executed, are a pass or fail affair there's no middle ground. The acceptance tests help to document

what the system should do, they may compliment other documentation or they may form the sole

specification of what the system should do. It depends on the team and project. When acceptance

tests are automated they can become what's described as living documentation. For example, if the

system is documented in a Word document or Excel spreadsheet it's easy for them to become out of

sync with the actual system and how it behaves. By automating acceptance tests we create a link

between the specification and the actual implementation. If the acceptance tests are changed or the

system is changed in an unexpected way, then we can execute our documentation, the acceptance

tests, to verify that things still work as expected. And acceptance tests, when executed, usually exercise

a vertical slice through the system. So in contrast to unit testing where we're just concerned with testing

a single class or integration testing where we might be testing groups of classes and components,

acceptance tests can touch all of the different layers in a system. From the lower data layers right up

through the business layer and even to the UI layer. If the acceptance tests are created collaboratively

and then used to drive what's actually being built, it can enable the whole team to have a

shared understanding of exactly what's required of the system. Acceptance tests can help to define

what done means so the scenarios that make up a feature, once they all pass we can say that the

feature is complete and can be marked as done.

Business Facing Tests

Conceptually one way of grouping tests is to divide them between those tests that face the business

and those tests which face the technology. So acceptance tests can be thought of as business facing

tests in that they're designed to be written and read by business people. So this could include product

owners, users, and business analysts, but it's also a good idea to get the whole team involved

when creating acceptance tests and this includes the testers and developers. Technology facing tests

2
on the other hand are generally only created by developers and these tests test the

technology implementation. So here we're talking about things like unit and integration tests and

processes such as test driven development. Martin Fowler defines business facing tests as "A test

that's intended to be used as an aid to communicating with the non-programming members of a

development team such as customers, users, business analysts, and the like". Whereas our technology

facing tests are really written by and for developers. It would be extremely hard for non-programming

members of the team to make sense of unit tests.

SpecFlow is just A Tool

Ultimately SpecFlow is just a tool. We can use it in a number of different context, whether it be

acceptance test driven development, behavior driven development, some kind of general agile

process, conceptually it could even be used in a waterfall scenario, but this is certainly not what it's

aimed at. And also hybrids of all these approaches. SpecFlow itself doesn't impose any specific

software development lifecycle process.

Use in Test-first Approaches

When SpecFlow is used in test-first approaches we first start off by discussing with the team what the

system should do next. And it's important at this point to get the business or users or product owners

involved in the conversation. Once we've had this conversation we can then write the tests in

SpecFlow. Once we've written the test we then start to write our code. Once we've written our code we

execute the SpecFlow tests to see if they pass. If the tests don't pass we continue writing code until

they do. Once the tests pass we can then define what the system should do next. And go around the

circle again.

SpecFlow Structure

3
So let's take a look at the SpecFlow structure itself. First off we have our Visual Studio integration and

we'll see in just a moment how we can install this. So with SpecFlow we start at the top with a feature

file. So a feature could be something like the addition feature in a calculator. Once we have our feature

file we define a number of scenarios that describe individual usages within this feature. So for calculator

this could be a scenario where two positive numbers were added and another scenario could be

where two negative numbers were added. Logically a scenario consists of a number of steps and in

general terms these describe the state before, the action during, and the expected state afterwards.

Sometimes in unit testing this is described as a range act assert, but when we come to discuss the

Gherkin syntax, in the next module, we'll discuss this further. For each individual step in the scenario we

can write the actual test code that interacts with the system under test. And to create this test code we

make use of a testing framework. So this could be something such as NUnit. The testing framework

such as NUnit can integrate with our test runner in Visual Studio and allow us to make use of the testing

frameworks assert methods to determine ultimately whether a scenario has passed or failed. The

SpecFlow Visual Studio integration, amongst other things, allows us to run tests right from the scenario

level without having to locate the individual test code and execute the tests from that level. The feature

file and its scenarios are written in a business readable language and again we'll cover the Gherkin

language in module 2.

Installing in Visual Studio and Creating A Feature File

So let's see now how we can install the SpecFlow Visual Studio integration. We'll then go and create

our first feature file. We can install the SpecFlow integration by opening the Extensions and Updates

window and searching on line for SpecFlow. So here we're going to install the SpecFlow extension and

hit Install and I'll just close this window. And we just need to restart Visual Studio for the install

to complete. So let's go ahead and do that now. So the first thing we need to do is create a new class

library project in this empty solution. So let's go ahead and do this we'll add a new project and we've

got Class Library project and we'll go ahead and call this Foo. specs. So let's go ahead and click OK.

4
And we'll just delete the auto added class 1 from the solution. So now we have our class library

project to hold our features, we can go and add our first feature file. So we'll go to Add New Item and

notice now that we've got these new SpecFlow items that we can add. So we'll start here by selecting

SpecFlow Feature File and we'll just give it a name AwesomeFeature and we'll just hit Add. So when

we add a feature we get a template that we can use to start writing our feature and this is written in the

Gherkin language which we're going to be looking at in module 2. So notice here in our Solution

Explorer we've got our Feature file now and inside this Feature file we can define a number of

Scenarios to create our acceptance criteria for this feature. Once we've created our Scenarios each

of these steps within the Scenario we can automate and verify that the system behaves as expected.

Module Summary

So that brings us to the end of module 1. In this module we gained an overview of SpecFlow itself. We

learned what acceptance tests are and some of their benefits. We saw how we can group tests into

business facing and technology facing tests. And how we can use SpecFlow in test-first approaches.

We learned about the basic structure of SpecFlow tests and we saw how we can install the

Visual Studio extension for SpecFlow. And how to create a new feature file. Join me in module 2 when

we'll explore the Gherkin language itself.

The Gherkin Language

Module Overview

Hi welcome back to module 2. In this module we'll be looking more closely at the Gherkin language that

we use to write our acceptance tests. In this module we'll be learning how we define features

and scenarios within those features. How we use the given, when, then steps to flush out our individual

scenarios. We'll look at how we can classify scenarios and features by using tags. And how we

can comment out individual lines within our features. We'll look how we can define tabular data to be

passed to the automation for our steps. And how we can use scenario outlines to create data driven

5
scenarios. Finally we'll look at backgrounds and how we can use them to provide common setup for all

of the scenarios in a given feature file. So let's get started by diving into features.

What is Gherkin?

So what is Gherkin? Gherkin is a business readable domain specific language. It represents tests in a

natural language that the business can understand as opposed to code that only programmers

can understand. It's a line-oriented language where new lines represent distinct elements and it uses

indentation to create a sense of structure. Gherkin has a number of keywords such as feature and

scenario. And these keywords are localized in over 40 spoken languages, for example

French, Bulgarian, and Japanese, this means that the Gherkin tests don't have to be written in English.

Features

Features are the top level grouping in Gherkin. A feature contains one or more scenarios and we'll look

at scenarios in just a moment. Features group or contain logically related test scenarios. Each feature

file represents a small discreet feature of the system. You wouldn't, for example, create a single feature

file that has scenarios for everything the system did. Feature files start with the keyword feature. This is

followed by a feature name or a short terse description of the feature. Following the feature name we

can optionally add our own free text description to help describe what the feature covers and the

scenarios contained within. So if we were creating a feature to represent brushing of teeth we'd

start with the feature keyword and in this case the name of the feature is Brush Teeth. And here we

followed this with an optional text description that we create by indenting, either using spaces or tabs,

a number of lines representing the description. So here the text Brushing teeth is good because it helps

keep them clean, doesn't have anything to do with automation, it's purely some additional information

that we can give to the reader. So our feature free text description can pretty much follow any format or

no format that we desire, but there's a number of standard formats that are often used. The first follows

the format as a, I want, so that. So we could rewrite the free text description in our Brush Teeth feature

6
using this format. So first off we start with the role or the person and this is the As a human. Next

we have the I want part, so here we're saying I want to brush my teeth. And finally the outcome or value

that we desire, here we're saying So that they stay pain free. And this format puts the role or the person

first. So we can describe this type of description as role or person centric. Some people have criticized

this format in that the value that we're trying to get doesn't appear until the end. So an alternative to this

format is the in order to, as a, I want format. So we can rewrite the previous as In order to stay pain

free As a human I want to brush my teeth. And in this format we put the value in the first line. So we can

describe this as value centric. Again you don't have to use these formats, but they can help to solidify

exactly what the feature is covering. And again you don't have to, you can use any kind of text that you

feel is useful.

Scenarios

So once we've created our feature file we then start to add scenarios to it. Scenarios are concrete

examples of the expected behavior of the system we're testing. Each scenario describes a particular

situation and use of the feature. Each scenario should be independent and isolated from the

other scenarios. So this means that we should be able to run any given scenario at any time and it

should always produce the same result, pass or fail. For example, we shouldn't depend on another

scenario having just been run for the next scenario to pass. Scenarios can represent the happy paths of

the feature so these are the normal expected usages where everything is okay. Scenarios can also

represent what happens when certain error conditions are met. And we can also have our

scenarios represent those edge cases of things that don't occur very often but things that are important

to us. We mark the start of a scenario by using the scenario keyword. We then follow this by the title of

our scenario. For example, a scenario representing a successful brushing of teeth would be

represented like this, the scenario keyword with a code on followed by our title.

Steps - Given, When, Then

7
Each of our scenarios consist of a number of steps. We can think about these steps as falling into one

of three overall phases. In the first phase we set up the initial state of the system. This is where we get it

ready to perform the specific actions that we want to test. Next we actually perform those actions

against the system. Once we've performed one or more actions we check that the end state of

the system, after we've performed these actions, is as expected. In Gherkin these phases map to the

following keywords. In the setup phase we use the given keyword to create our setup steps. When we

perform actions we use the when keyword in our action steps and for our end state we use the then

keyword. So if we look at this successful brushing scenario we can add our steps to it. So first off in our

setup phase we have our givens and notice these are indented. So here we're saying in our setup

phase we're saying that there's toothpaste on the brush and that the mouth is open, given that these

things are true we next come to our action steps. So here we use the when keyword. When the

back teeth are brushed, when the front teeth are brushed, then the teeth look clean and then the mouth

feels fresh and then the braces aren't damaged. So the thens form our third phase where we check that

the resulting state of the system is as expected. So in this scenario we've got multiple givens, whens,

and thens, but notice that it doesn't really read particularly naturally, so we can try and improve this

scenario by changing it and adding the and keyword. So now our setup phase, the given, reads a bit

more naturally because we've used the and keyword. Given there is toothpaste on the brush and the

mouth is open. We can also use the and keyword in our second phase. So here we're saying when the

back teeth are brushed and the front teeth are brushed, again this reads more naturally than having two

when lines. Finally we can use the and keyword and also the but keyword to make our third phase, the

then phase, read more naturally. We're now saying then the teeth look clean and the mouth feels fresh,

but the braces aren't damaged. So here we've still got our three conceptual phases, but we're making

use of the and and but keywords to improve the readability. And we can go one step further and use

indentation to further improve the readability.

Tags

8
The Gherkin language gives us the idea of tags. They allow us to mark our features and scenarios with

arbitrary tags. Tags can map to categories in the unit test framework we're using to perform

the automation and allow us to execute only those scenarios that have a specified tag. Tags can be

applied at the feature level or on individual scenarios. If we apply a tag to the feature each of the

scenarios within that feature inherit that tag. Scenarios and features can have multiple tags and we

specify a tag by prefixing it with the @ symbol. The ignore tag is a special case in that it enables us

to temporarily remove tests from being executed. So to mark this successful brushing scenario with the

smoke test tag we simply proceed it with @smoketest. There's a number of potential usages for tags.

We can use them to group features into supersets of features, we could mark certain tests as being

more important than others, we could differentiate between slow running and fast running tests,

we could mark certain tests as being works in progress, or we could use tags to indicate how often tests

should be executed. For example, weekly, daily, or hourly and we could also mark tests for example

by specifying @humanexecuted to exclude them from automation and to indicate that the specific

scenarios, for whatever reason, have to be executed manually by a human tester.

Demo: Applying Tags

So let's have a look at tags in action. So here we've just got a Feature file and we've got three

Scenarios. In our Visual Studio Test Explorer we're currently in the Class view. And here we can see

our three scenarios underneath our Feature file. So let's go ahead and switch to this Traits view and

here we can see our three Scenarios and they have no traits applied. When we talk about traits we're

talking about a similar concept to categories in unit tests. So we can start to categorize these scenarios

by using tags. So let's start with Scenario A and let's say we want to add this to the important category.

If we build now we can see that we've got our Scenario A now belonging to the important Category and

our scenario B and C as not belonging to any category. So let's go ahead and mark Scenario B

as being a fast executing test. And Scenario C as being a slow executing test. Again we rebuild and we

can see now our tests in their different categories. We can add multiple tags to a scenario, so Scenario

9
A is now important and fast. If we build we can see now that Scenario A is both part of the fast category

and the important category. We can apply tags to the feature itself so let's mark the entire feature as a

work in progress and build. We can now see that all of the scenarios in this feature have inherited the

work in progress tag. So now our Scenario A has three tags applied. These two at its scenario level and

this one inherited from the feature. And we can see this here. Scenario A belongs to the fast, important,

and work in progress categories.

Demo: Commenting Lines

If we need to we can add comment lines by prefixing the line with the hash symbol. So for example let's

just comment out the important tag by applying a hash and notice that it goes green. And say

for example that we wanted to comment out the When line here and just replace it with another one. If

we wanted to we could also comment out to the tag at the feature level or just add arbitrary comments

between these two scenarios. (Typing)

Data Tables in Steps

We can use data tables in individual scenario steps. Data tables allow us to pass tabular data to the

underlying automation step. And they're useful for specifying larger datasets rather than having multiple

individual steps to specify the data. So this scenario here Given I have paste and brush and water when

the back teeth are brushed then the teeth look clean, we could rewrite this using a data table to specify

the given tools we need. So here we've replaced the multiple given steps with a data table we're saying

given I have the following tools paste, brush, and water, when the back teeth are brushed then the teeth

look clean. So to use a data table in a step we add a code on to the end of the step and then we create

a table in text by using the vertical pipe symbol. So here we've created a table with a single column

and three data rows, paste, brush, water with a column name of tool name. When we come to write the

automation step for this given we get a parameter passed to the automation step containing this

data table. We can then use the items in this data table in our automation.

10
Data-Driven Scenario Outlines

Whereas data tables allow us to pass tabular data to an individual scenario step. Scenario outlines

allow us to execute the same scenario over and over again each time using a different set of test data.

So rather than writing many scenarios with only small variations in data we can create one scenario

outline and then specify the data to be input to this scenario outline when each execution occurs. To

define a scenario outline we use the scenario outline keyword with a colon followed by the name. So

here we're creating a scenario outline called teeth whiteness. When we create a scenario outline we still

use our given, when, and thens and optionally our ands and buts, but this time we can define place

holders for the data that we want to execute the steps with. So here we're saying given I'm using some

kind of brand of toothpaste, when I brush for so many minutes, then the teeth look so much percent

white. So we're not specifying actual values, we're specifying place holders inside these angle brackets.

Now we've created our scenario outline we need to be able to specify the data to be substituted

into these place holders. And we do this by using the examples keyword. Similar to data tables, we

create a table using vertical pipes, but this time our column headers map to place holders in our

steps. Each individual row in the table represents a single execution of our scenario outline. So here

we've got three data rows which means this scenario will get executed three times. The first time with a

brand of Brand X, mins of 1, and percent of 80. The second time with Brand Y and its associated data

and the third time with Brand Z and its associated data.

Backgrounds

Gherkin provides us with the concept of backgrounds. A background provides context to the scenarios

within a given feature. The steps within a background are executed before each and every scenario

runs. To create a background we use the background keyword, followed by colon, and then

the individual steps that make up the background. So here our background states that given I have

teeth and I have some toothpaste. The two steps within this background get executed before

the successful brushing scenario executes and before the toothpaste runs out scenario executes and it

11
would also execute before other scenarios if we added them. So using backgrounds we can reduce

some duplication in the individual scenarios within a feature. We just have to be careful when we're

using backgrounds that the background doesn't get too large and unreadable and that the things in

a background are relatable to the business reader.

Module Summary

So that brings us to the end of module 2. In this module we learned about Gherkin features and

scenarios and how to write scenarios using given, when, and then steps. We also saw how the but and

and steps can help us create more naturally readable scenarios. We saw how we can categorize

features and scenarios using arbitrary tags and how we can add comments to our feature files. We saw

how we can define data tables to be passed to the automation for individual steps. And how we can use

scenario outlines to create data driven scenarios. Finally we looked at backgrounds and how we can

use them to provide common setup steps for each of the scenarios in a given feature file. Join me in

module 3 when we'll see how we can hookup all of our steps in our scenarios to actual code

automation.

12