In this post, I’m going to use SpecFlow and the Gherkin syntax to write tests in a natural language. I want to show how easy that is to setup, when you have code that is already testable, and how it helps you to write better tests in a few minutes.
SpecFlow is a testing component / framework / Visual Studio plug-in, which allows you to write your tests in a natural fashion, and run these tests using your favourite test-runner. As quoted from their web site :
SpecFlow aims at bridging the communication gap between domain experts and developers by binding business readable behavior specifications to the underlying implementation. Our mission is to provide a pragmatic and frictionless approach to Acceptance Test Driven Development and Behavior Driven Development for .NET projects today.
SpecFlow is build upon the Gherkin syntax, which is a grammar that has been designed to write behaviour specifications. This syntax first emerged in the Ruby community, and is linked to a project called Cucumber. For more details, you can check out http://cukes.info :
Cucumber is Aslak Hellesøy’s rewrite of RSpec’s “Story runner”, which was originally written by Dan North. (Which again was a rewrite of his first implementation – RBehave. RBehave was his Ruby port of JBehave)…
Gherkin is Cucumber’s DSL (Domain Specific Language) and as such, is not linked to a particular language or technology. It can therefore be used in .NET by using SpecFlow. It has very few keywords, yet is very powerful. The best way to explain how it works is to show an example of a test scenario :
Scenario: Filter on age
Given I have written a query against the provider
And I have added a Age >= 36 where clause
Given The people finder service filters on Age >= 36
When I execute the query
Then The service parameter should be Age IsGreaterThan 36
And The result count should be 3
And The result should validate the servicePredicate
From now on, you might have noticed that I’m working on my Linq provider again… but in this post I’ll talk only about testing and using SpecFlow, so don’t run away, even if you’re sick of expression trees !
What SpecFlow allows you to do, is to write your specifications and tests cases in plain text, and generate tests classes from it. It has several nice features that help you organize your tests and examples, which I’ll describe soon.
In order to use SpecFlow in your project, the easiest way is to add a reference to it in your test project, using the NuGet package manager. To get Visual Studio integration, you should also install the “SpecFlow integration for Visual Studio extension”.
When doing BDD, the tests are written in order to tests features. For each feature of an application (or component, or class), you can create a feature file. As you can see from the previous example, tests will be organised around the concept of “scenario”. Each feature file can contain several scenarios. The binding between a scenario and the actual tests implementation will be done on a line-by-line basis. Each line of the scenario will end up as a method call :
- the “Given” and associated “And” clauses will correspond to methods that set up the tests,
- the “When” clause will trigger some action or event that drives the behaviour that is aimed to be tested,
- the “Then” and associated “And” clauses will perform the assertions that determine the outcome of the test.
In order to do the matching between lines in the scenario and methods in the test class, the methods are marked with the appropriate SpecFlow attributes, that take the model of the expected sentence as an argument. This model can include groups of capturing expressions, that are used to populate the method parameters at run time.
[Given(@"I have added a (.*) where clause")] public void GivenIHaveAddedAWhereClause(string predicate)
If we go back to the test scenario shown earlier, the best way to see how the test is executed is to look at the generated test output:
Given I have written a query against the provider
-> done: FeaturesTest.GivenIHaveWrittenAQuery() (0,0s)
And I have added a Age >= 36 where clause
-> done: FeaturesTest.GivenIHaveAddedAWhereClause("Age >= 36") (0,0s)
Given The people finder service filters on Age >= 36
-> done: FeaturesTest.GivenThePeopleFinderServiceFiltersOn("Age >= 36") (0,0s)
When I execute the query
-> done: FeaturesTest.WhenIExecuteTheQuery() (0,1s)
Then The service parameter should be Age IsGreaterThan 36
-> done: FeaturesTest.ThenTheServiceParameter("Age IsGreaterThan 36") (0,0s)
And The result count should be 3
-> done: FeaturesTest.ThenTheResultCount(3) (0,0s)
And The result should validate the servicePredicate
-> done: FeaturesTest.ThenTheResultShouldValidatetheServicePredicate() (0,0s)
We can see that each line of the scenario is used to call a method. In order to “accumulate” the conditions and perform the assertions on the results, we use a class that defines the test context :
public class ServiceContext
{
// Data context
public IEnumerable<Person> AllPeople { get; set; }
// Input
public List<string> Query { get; set; }
public List<string> ServiceLambdas { get; set; }
// Service predicate, built upon the ServiceLambdas
public Expression<Func<Person, bool>>
ServicePredicate {get; set;}
// Output
public List<Person> Results { get; set; }
public SearchCriteria PassedCriteria { get; set; }
}
Using this context class, the methods corresponding to the “Given” populate the properties used as the test input, the “When” performs the action, which populates the output based on the input, and the “Then” make assertions on the output.
The next step is to add additional test cases. A simple way could be to simply copy-paste the previous scenario, and change the parameters. But as we always want to void duplication, we’ll use a very handful feature here and use a “Scenario Outline” and examples :
Scenario Outline: Filter on a single criterion
Given I have written a query against the provider
And I have added a <predicate> where clause
Given The people finder service filters on <servicePredicate>
When I execute the query
Then The service parameter should be <serviceParameter>
And The result count should be <resultsCount>
And The result should validate the servicePredicate
Examples:
| predicate | servicePredicate | serviceParameter | resultsCount |
| Age >= 36 | Age >= 36 | Age IsGreaterThan 36 | 3 |
| Age >= 36 | | Age IsGreaterThan 36 | 6 |
| FirstName.StartsWith("Scar") | FirstName.StartsWith("Scar") | FirstName StartsWith Scar | 1 |
| LastName == Alba | | LastName Equals Alba | 6 |
The main advantage of this syntax is that it is self-explanatory ! Once the scenario outline is prepared, we list all the cases that we want to test.
If you’ve read the post carefully up to this point, you might have noticed that my scenario makes assertions on the number of results, but that there is nothing that sets-up the test data. In fact, there is a hidden “Given” clause that is used across the whole feature file. The way to to this is by using the keyword “Background”:
Background:
Given The people are
| FirstName | LastName | Age | Id |
| Scarlett | Johansson | 27 | {8c319634-935d-4681-adcc-02d5347fe6c4} |
| Jessica | Alba | 31 | {32a84597-8c3d-44bc-a1a5-6538188e9d25} |
| Penelope | Cruz | 38 | {5aa0eb59-3961-472f-b829-7d54ac8eeeef} |
| Reese | Witherspoon | 36 | {77fcd741-3839-4692-925f-a3a0eb19cf42} |
| Charlize | Theron | 36 | {e37290f6-d376-44e2-944d-d0af13c1a75c} |
| Mouloud | Achour | 31 | {18af541a-a5dc-41d2-af47-479b1c06e216} |
One last thing : although parsing the lambda given as arguments in the scenario is a topic of its own and could be discussed further, setting up the test data from the previous “Given” clause is delightfully easy, and I couldn’t help not showing you the code :
[Given(@"The people are")]
public void GivenThePeopleAre(Table people)
{
this.context.AllPeople = people.CreateSet<Person>();
}
As a conclusion, I strongly recommend to give SpecFlow a try, you won’t regret it !
All the source code from this post and the Linq Provider Series is now available on my Github at https://github.com/pirrmann/LinqToService. Please feel free to fork and/or contribute, and I’d also be really happy to get feedback !
