Configuration in Allure to Cucumber in Ruby

Configuration in Allure to Cucumber in Ruby

History

Allure stores history information to allure-report/history folder during report generation. So you need to copy such folder from previous launch into your allure-results before you generate the report.

History features are handled out of the box by Allure CI plugins

Executor info

To add an information about your test executor create a file executor.json in your allure-results:

{
  "name": "Jenkins",
  "type": "jenkins",
  "url": "http://example.org",
  "buildOrder": 13,
  "buildName": "allure-report_deploy#13", 
  "buildUrl": "http://example.org/build#13",
  "reportUrl": "http://example.org/build#13/AllureReport",
  "reportName": "Demo allure report"
}

In the report such information will be displayed like this:

Allure CI plugins will add executor information for you

Categories

You can also add custom problem categories in the report. Create file categories.json and place it into your results folder:

[
  {
    "name": "Ignored tests",
    "messageRegex": ".*ignored.*",
    "matchedStatuses": [
      "skipped"
    ]
  },
  {
    "name": "Infrastructure problems",
    "messageRegex": ".*RuntimeException.*",
    "matchedStatuses": [
      "broken"
    ]
  },
  {
    "name": "Outdated tests",
    "messageRegex": ".*FileNotFound.*",
    "matchedStatuses": [
      "broken"
    ]
  },
  {
    "name": "Regression",
    "messageRegex": ".*\\sException:.*",
    "matchedStatuses": [
      "broken"
    ]
  }
]

Retries

Retries are handled by default. For each test Allure generates historyId (usually it is md5(fullName + parameterValues)). If report contains few results with the same historyId Allure only displays latest, other are marked as retries and available from Retries section on test result page.

For example, if you run your test 3 times (first 2 are broken, latest is passed, Allure will display such test as passed)

Allure Reports are not getting created when running the tests with cucumber - capybara - selenium

The problem here is that you are including Capybara::DSL in the global scope. Any relatively modern version of Capybara will print a warning to the console like "including Capybara::DSL in the global scope is not recommended!" specifically because it will have all sorts of strange side effects. This is because when you just do

include Capybara::DSL

outside of any classes or modules you end up including all of Capybaras methods on every object in your project. That's not what you want. It's impossible to say exactly what you need to put where without looking at your project, but assuming you have a normal project you probably want to put

World(Capybara::DSL)
World(Capybara::RSpecMatchers)

in your env.rb, or just require 'capybara/cucumber' like instructed - https://github.com/teamcapybara/capybara#using-capybara-with-cucumber - which will get things set up correctly.

Error when running Cucumber test scenario in RubyMine

This is a RubyMine bug. Nothing we can fix on the Cucumber end.

You can either consult a non-recommended monkeypatch / hack. Or downgrade to an early version of Cucumber5.

See https://youtrack.jetbrains.com/issue/RUBY-27294 for more information, including other possible workarounds and a time-frame for the fix from Jetbrains.

Luke - Cucumber Ruby committer.

How I can attach screenshot of failed Cucumber step to Allure report exactly in step, not in Tear Down section?

Idea is using the events to capture the step status and embed the screenshot. If I am correct, As per current design AfterStep does not execute when the step is failed. So we have to use AfterConfiguration in hooks.

Ruby:
Here is the ruby implementation.

AfterConfiguration do |scenario, config|
    config.on_event :test_step_finished do |event|
        if event.result.failed?
          # get screenshot
        end
    end
  end

Check this thread regarding the allure implementation.

TextMate, Cucumber & Spork: Run feature **with** HTML output?

Solution from Jim Drannbauer worked for me: http://jimdrannbauer.com/2011/02/22/cucumber-spork-textmate-drb-drbunknown/

Cucumber JUnit --out directory

I forgot to set up environmental variables

export CI_REPORTS=shippable/testresults

This did the trick

---

Related Topics