Good Cucumber Examples in the Wild

Good Cucumber examples in the wild?

You can read diaspora's cucumber tests. It's a pretty big project so I think you can learn something from it.

RSpec and Cucumber in the RoR wild

[Repeating some of what others said for the sake of completeness:]

A good approach known as BDD (see bdd) works outside-in by

• test-driving a feature with an acceptance test (which in the Ruby world is usually written as a Cucumber feature or RSpec feature spec)
• test-driving details with unit tests to express detailed requirements if necessary.

Using this method to test-drive a feature of a Rails project, you'd first write an acceptance test and then implement it, which would bring into existence the route(s), view(s), controller(s) and model(s) needed for the feature. Since you're test-driving, you haven't written any code that isn't fully tested by the acceptance test and you don't need any controller, model, etc. specs yet. You might then start over with another acceptance test, or you might want to test-drive some detailed requirements using lower-level specs.

Suppose that your acceptance test tested that a user's full name is shown on the page and you want to be sure that it looks correct when a user hasn't entered their last name. If User#full_name returns the user's full name, you could just write a model spec for that method rather than writing another acceptance test.

In my experience well-factored Rails applications written using BDD need few or no controller specs, since controllers should delegate to models and other classes as much as possible and so are completely tested by acceptance tests. Lots of details are best tested in model specs.

Regarding how to write acceptance tests, I need to disagree with others' statements about Cucumber, or at least give a warning: I've seen Cucumber result in a readable, maintainable acceptance test suite for a large production application, and RSpec feature specs result in an bloated, unmaintainable acceptance test suite, so don't dismiss Cucumber, or the needs that it addresses:

• Don't be fooled by the idea that human-readable acceptance tests are only for your product owner or customer; they're absolutely valuable for engineers, allowing you to think at a high level when appropriate instead of getting lost in the noise of code. The larger the project, the more you'll care.

• Cucumber nicely separates acceptance-level descriptions of steps in tests from implementations of those steps (CSS selectors, ActiveRecord queries, etc.), almost automatically making those steps reusable. With RSpec feature specs, it's all up to you; be prepared to come up with your own system of extracting detail into methods whose names make it clear what the user-level point of each step is. (I don't mean page objects; they're a level of detail below what I mean, and are useful in both Cucumber and RSpec feature specs.)

Why isn't Cucumber considered as a testing tool?

TL;DR

Cucumber is a BDD framework. Its main components are:

1. Gherkin: a ubiquitous language used as communication tool than
   anything, and can be used as a springboard for collaboration. It
   helps manage the expectations of the business, and if everyone can
   see the changes that you are making in a digestible format, they'll
   hopefully get less frustrated with the development teams, but also
   you can use it to speed up the reaction time of your team if there
   are bugs, by writing the tests that cucumber wraps with the mindset
   that someone is going to have to come back and debug it at some
   point.
2. CLI implementation (or the CLI): a test runner based on
   Gherkin. It is developed by volunteers who are all donating part of
   their spare time. Every implementation is specific to a programming
   language supporting a production code that is ready for test. It is
   considered as the concrete tool / utility.

The Long Version

The intended use of Gherkin as a communication tool, describing interactions with a system from multiple perspectives (or actors), and it just so happens that you can integrate it with test frameworks, which aids in ensuring that the system in place correctly handles those interactions.

Most commonly, this is from a users perspective:

Given John has logged in
When he receives a message from Brenda
Then he should be able to click through to his message thread with Brenda via the notification

But it can also be from a component/pages perspective too:

Given the customer history table is displayed
And there have been changes to the customer table for that user since the page was first loaded
When it receives a click to refresh the data
Then the new changes should be displayed

It's all about describing the behaviours, and allowing the business and developers to collaborate freely, while breaking down the language barriers that usually end up plaguing communication, and generally making both sides frustrated at each other because of a lack of mutual understanding of an issue

This is where the "fun" begins _{- Anakin, Ep III}

You could use these files to create an environment of "living documentation" throughout your development team (and if successful, the wider business), and in theory - worded and displayed correctly, it would be an incredible boon for customer service workers, who would more easily be able to keep up with changes, and would have extremely well described help documentation - without any real additional effort, but this isn't something that I've seen much in the wild. I've written a script at work that does this by converting the features into markdown, and alongside various other markdown tools (mermaid for graphs, tsdoc-plugin-markdown to generate the API docs, and various extensions for my chosen HTML converter, docsify) I've managed to generate something that isn't hard to navigate and open up communication between teams that previously found it harder to communicate their issues to the dev team (most people know a little markdown these days, even if it has to be described as "the characters you type in reddit threads and youtube comments to make text bold and italics, etc" for people to get what it is, but it means everyone can contribute to it)

It is an extremely useful tool when it comes to debugging tests, especially when used with the screenplay pattern (less so with the standard page object model, because of the lack of additional context that the pom provides, but it's still useful), as everything is described in a way that breeds replication of the issue from a users or components perspective if it fails.

I've paired it with flow charts, where I draw out the user interactions, pinning the features to it and being able to see in a more visual way where users will be able to do something that we might not have planned for, or even figure out some garish scenario that we somehow missed.

The Long Version Longer

My examples here will mostly in javascript, as we've been developing in a node environment, but if you wanted to create your own versions, it shouldn't be too different.

The Docs

Essentially, this is bit is just for displaying the feature files in a way that is easily digestible by the business (I have plans to integrate test reports into this too, and give the ability to switch branches and such)

First, you want to get a simple array of all of the files in your features folder, and pick out the ones with ".feature" on the end.

Essentially, you just need to flatten an ls here (this can be improved, but we have a requirement to use the LTS version of node, rather than the latest version in general)

const fs = require('fs');
const path = require('path');
const walkSync = (d) => fs.statSync(d).isDirectory() ? fs.readdirSync(d).map(f => walkSync(path.join(d, f))) : d;

const flatten = (arr, result = []) => {
  if (!Array.isArray(arr)){
    return [...result, arr];
  }
  arr.forEach((a) => {
    result = flatten(a, result)
  })
  return result
}

function features (folder) {
  const allFiles = flatten(walkSync(path.relative(process.cwd(), folder)))
  let convertible = []
  for (let file of allFiles) {
    if (file.match(/.feature$/)) {
      convertible.push(file)
    }
  }
  return convertible
}

...

Going through all of those files with a Gherkin parser to pull out your scenarios requires some set up, although it's pretty simple to do, as Gherkin has an extremely well defined structure and known keywords.

There can be a lot of self referencing, as when you boil it down to the basics, a lot of cucumber is built on well defined components. For example, you could describe a scenario as a background that can have a description, tags and a name:

class Convert {

   ...

   static background (background) {
    return {
      cuke: `${background.keyword.trim()}:`,
      steps: this.steps(background.steps),
      location: this.location(background.location),
      keyword: background.keyword
    }
  }

  static scenario (scenario) {
    return {
      ...this.background(scenario),
      tags: this.tags(scenario.tags),
      cuke: `${scenario.keyword.trim()}: ${scenario.name}\n`,
      description: `${scenario.description.replace(/(?!^\s+[>].*$)(^.*$)/gm, "$1<br>").trim()}`,
      examples: this.examples(scenario.examples)
    }
  }

  ...
}

You can flesh it out fully to write to either a single file, or output a few markdown files (making sure to reference them in a menu file)

Flowcharts

Flow charts make it easier to help visualise an issue, and there are a few tools that use markdown to help generate them like this:

In the back, it'll end up looking like this:

### Login

Customers should be able to log into their account, as long as they have registered.

...

```mermaid
 graph TD
        navigateToLogin["Navigate to Login"] -->logIn{"Login"}
        logIn -->validCredentials["Valid<br>Credentials"]
        logIn -->invalidCredentials{"Invalid<br>Credentials"}
        invalidCredentials -->blankPass["Blank Password"]
        invalidCredentials -->wrongPass["Wrong Password"]
        invalidCredentials -->blankEmail["Blank Email"]
        invalidCredentials -->wrongEmail["Wrong Email"]
        ...

        click blankPass "/#/areas/login/scenario-blank-password" "View Scenario"
        ...
 ```

It's essentially just a really quick way to visualise issues, and links us to the correct places in the documentation to find an answer. The tool draws out the flowchart, you just have to make the connections between key concepts or ideas on the page (i.e. a new customer gets a different start screen)

Screenplay Pattern, Serenity and Debugging

I think all that really needs to be said here is that when you run a test, this is your output:

✓ Correct information on the billing page
    ✓ Given Valerie has logged into her account
        ✓ Valerie attempts to log in
            ✓ Valerie visits the login page
                ✓ Valerie navigates to '/login'
            ✓ Valerie waits up to 5s until the email field does become visible
            ✓ Valerie enters 'thisisclearlyafakeemail@somemailprovider.com' into the email field
            ✓ Valerie enters 'P@ssword!231' into the password field
            ✓ Valerie clicks on the login button
            ✓ Valerie waits for 1s

It will break down any part of the test into descriptions, which means if the CSS changes, we won't be searching for something that no longer exists, and even someone new to debugging that area of the site will be able to pick up from a test failure.

Communication

I think all of that should show how communication can be improved in a more general sense. It's all about making sure that the projects are accessible to as many people who could input something valuable (which should be everyone in your business)

What is the difference between Serenity BDD Framework and Cucumber tool

No, those are 2 different things.

• Cucumber is the layer to map BDD syntax, which is written in .feature file, with actual code that does the job.

• Serenity BDD is the framework supporting 3 different approaches:

  • Cucumber: same features as stand-alone cucumber, can work with UI or API Automation
  • Page Object: works with UI automation (selenium)
  • Screenplay: a design pattern for UI and API automation

• Serenity BDD does many things for auto testers that:

  • Config-oriented: serenity.properties or serenity.conf. For example: take screenshot when FOR_EACH_ACTION, BEFORE_AND_AFTER_EACH_STEP, AFTER_EACH_STEP, FOR_FAILURES, DISABLED. This is really helpful for debugging. https://serenity-bdd.github.io/theserenitybook/latest/serenity-system-properties.html#_serenity_take_screenshots
  • Living report: much more better comparing to cucumber. https://serenity-bdd.github.io/theserenitybook/latest/living-documentation.html
  • Wrap other libs in nice and clean APIs: serenity-appium, serenity-browserstack, serenity-cucumber, serenity-rest-assured, serenity-saucelabs, serenity-shutterbug1x ... You see all of them at https://github.com/serenity-bdd/serenity-core

---

Related Topics