How to Execute Cucumber Feature File Parallel

How to execute cucumber feature file parallel

Update: 4.0.0 version is available at maven central repository with bunch of changes.for more details go here.

Update: 2.2.0 version is available at maven central repository.

You can use opensource plugin cucumber-jvm-parallel-plugin which has many advantages over existing solutions. Available at maven repository

   <dependency>
     <groupId>com.github.temyers</groupId>
     <artifactId>cucumber-jvm-parallel-plugin</artifactId>
     <version>2.1.0</version>
   </dependency>

First you need to add this plugin with required configuration in your project pom file.

<plugin>
  <groupId>com.github.temyers</groupId>
  <artifactId>cucumber-jvm-parallel-plugin</artifactId>
  <version>2.1.0</version>
  <executions>
     <execution>
     <id>generateRunners</id>
     <phase>generate-test-sources</phase>
     <goals>
       <goal>generateRunners</goal>
     </goals>
     <configuration>
         <!-- Mandatory -->
         <!-- comma separated list of package names to scan for glue code -->
         <glue>foo, bar</glue>
         <outputDirectory>${project.build.directory}/generated-test-sources/cucumber</outputDirectory>
          <!-- The directory, which must be in the root of the runtime classpath, containing your feature files.  -->
           <featuresDirectory>src/test/resources/features/</featuresDirectory>
          <!-- Directory where the cucumber report files shall be written  -->
          <cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir>
          <!-- comma separated list of output formats json,html,rerun.txt -->
          <format>json</format>
          <!-- CucumberOptions.strict property -->
          <strict>true</strict>
          <!-- CucumberOptions.monochrome property -->
          <monochrome>true</monochrome>
          <!-- The tags to run, maps to CucumberOptions.tags property you can pass ANDed tags like "@tag1","@tag2" and ORed tags like "@tag1,@tag2,@tag3" -->
         <tags></tags>
         <!-- If set to true, only feature files containing the required tags shall be generated. -->
         <filterFeaturesByTags>false</filterFeaturesByTags>
         <!-- Generate TestNG runners instead of default JUnit ones. --> 
         <useTestNG>false</useTestNG>
         <!-- The naming scheme to use for the generated test classes.  One of 'simple' or 'feature-title' --> 
        <namingScheme>simple</namingScheme>
        <!-- The class naming pattern to use.  Only required/used if naming scheme is 'pattern'.-->
        <namingPattern>Parallel{c}IT</namingPattern>
        <!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario.  FEATURE generates a runner per feature. -->
        <parallelScheme>SCENARIO</parallelScheme>
        <!-- This is optional, required only if you want to specify a custom template for the generated sources (this is a relative path) -->
        <customVmTemplate>src/test/resources/cucumber-custom-runner.vm</customVmTemplate>
        </configuration>
       </execution>
     </executions>
   </plugin>

Now add below plugin just below above plugin which will invoke runner classes generated by above plugin

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.19</version>
        <configuration> 
            <forkCount>5</forkCount>
            <reuseForks>true</reuseForks>
            <includes>
                <include>**/*IT.class</include>
            </includes>
        </configuration>
    </plugin>

Above two plugins will do magic for cucumber test running in parallel (provided you machine also have advanced hardware support).
Strictly provided <forkCount>n</forkCount> here 'n' is directly proportional to 1) Advanced Hardware support and 2) you available nodes i.e. registered browser instances to HUB.

One major and most important changes is your WebDriver class must be SHARED and you should not implement driver.quit() method, as closing is take care by shutdown hook.

import cucumber.api.Scenario;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.events.EventFiringWebDriver;

public class SharedDriver extends EventFiringWebDriver {
    private static WebDriver REAL_DRIVER = null;

    private static final Thread CLOSE_THREAD = new Thread() {
        @Override
        public void run() {
            REAL_DRIVER.close();
        }
    };

    static {
        Runtime.getRuntime().addShutdownHook(CLOSE_THREAD);
    }

    public SharedDriver() {
        super(CreateDriver());
    }

    public static WebDriver CreateDriver() {
        WebDriver webDriver;
        if (REAL_DRIVER == null)
            webDriver = new FirefoxDriver();
        setWebDriver(webDriver);
        return webDriver;
    }

    public static void setWebDriver(WebDriver webDriver) {
        this.REAL_DRIVER = webDriver;
    }

    public static WebDriver getWebDriver() {
        return this.REAL_DRIVER;
    }

    @Override
    public void close() {
        if (Thread.currentThread() != CLOSE_THREAD) {
            throw new UnsupportedOperationException("You shouldn't close this WebDriver. It's shared and will close when the JVM exits.");
        }
        super.close();
    }

    @Before
    public void deleteAllCookies() {
        manage().deleteAllCookies();
    }

    @After
    public void embedScreenshot(Scenario scenario) {
        try {
            byte[] screenshot = getScreenshotAs(OutputType.BYTES);
            scenario.embed(screenshot, "image/png");
        } catch (WebDriverException somePlatformsDontSupportScreenshots) {
            System.err.println(somePlatformsDontSupportScreenshots.getMessage());
        }
    }
}

Considering you want to execute more than 50 threads i.e. same no of browser instances are registered to HUB but Hub will die if it doesn't get enough memory therefore to avoid this critical situation you should start hub with -DPOOL_MAX=512 (or larger) as stated in grid2 documentation.
Really large (>50 node) Hub installations may need to increase the jetty threads by setting -DPOOL_MAX=512 (or larger) on the java command line.
java -jar selenium-server-standalone-<version>.jar -role hub -DPOOL_MAX=512

How to Run cucumber Feature files in parallel

You cannot achive that with TestNg. But. You can achieve that with JUnit4. Basically That is how parallelisation works for JUnit 4 according to Cucumber specification.

Cucumber can be executed in parallel using JUnit and Maven test execution plugins. In JUnit the feature files are run in parallel rather than scenarios, which means all the scenarios in a feature file will be executed by the same thread. You can use either Maven Surefire or Failsafe plugin to execute the runners.

Is it possible to execute cucumber scenario's in parallel on different browsers(chrome and firefox) at same time?

You are running a your tests against a matrix of browsers. Typically this matrix configured in CI and provided to the test execution via environment variables. For example using Gitlab CI Matrix:

test:
  stage: test
  script:
    - mvn test
  parallel:
    matrix:
      - OS: Windows
        OS_VERSION: 10
        BROWSER: [Chrome, Firefox, Edge]
      - OS: OS X
        OS_VERSION: Big Sur
        BROWSER: [Chrome, Firefox, Edge, Safari]

You then create the web driver in the before hook using the environment variables.

    @Before
    public void before(Scenario scenario){
        String os = System.getenv("OS");
        String osVersion = System.getenv("OS_VERSION");
        String browser = System.getenv("BROWSER");
        driver = createDriver(os, osVersion, browser);
    }

You could also use Maven Profiles or Gradle Tasks to define these different sets of environment variables.

However key is to let these jobs in parallel on your CI system by starting multiple JVMs rather then only in Cucumber by starting multiple threads.

I need to run cucumber feature files in parallel using gradle

My recent answer to this closely related question should be helpful here:
https://stackoverflow.com/a/65473008/10256045

There is a bit of setup needed to get cucumber running in parallel
properly with gradle.
Every test has to be threadsafe so you need to be careful with static usage.
Multiple drivers need to be made for each thread
logging needs to log the correct thread and scenario
I have a skeleton framework that handles all of this that you can use
as a reference or build from here
For your specific question in the build.gradle you determine the
--threads cucumber option.
Take a look in the build.gradle here
These variables are used to setup parallel runs and determine threads
to use in the cucumber options
def availableThreadCount = Runtime.runtime.availableProcessors().intdiv(2) ?: 1
def cucumberThreadsOption = ['--threads', availableThreadCount.toString()]
if runInParallel task is used it puts setParallel to true. We
then add the --threads arg into the cucumber options This is
where that happens in the cucumber task
Here is where the built cucumber options are used
CreateSharedDrivers.java here is where we handle creating
multiple drivers for threads and have the shutdown hook implemented.
In Hooks.java here there is an example of how we log the thread
and its current scenario

Run scenarios in a cucumber feature file in parallel, one feature file at a time

This is not possible merely with configurations. The reason I say this is because you are expecting a multi level parallel execution strategy from within just a @DataProvider which is NOT possible in TestNG as of today. You would need to leverage a suite xml file for controlling the parallel execution strategy at a higher level, which can then be tweaked at a lower level by leveraging a @DataProvider coupled with a parallel attribute.

The easiest way of getting this done is to do the following:

Create one Java class per feature file.
Create a suite file with "n" classes and define your parallel strategy as "none" (This would force TestNG to pick up each of the classes and run them in sequential manner)
Don't forget to set a desirable data provider thread count (default value is 10) and set the attribute parallel = true in your @DataProvider

Not able to parallel test of two cucumber feature file

Junit:

Yes, you can execute features files in parallel in Cucumber. using JUnit and Maven test execution plugins. In fact using JUnit, only the feature files can be executed in parallel not the scenarios with in same file.Maven Surefire can be used to execute the runners.

Add the Surefire plugin configuration to the build section to the POM.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M1</version>
    <configuration>
        <parallel>methods</parallel>
        <useUnlimitedThreads>true</useUnlimitedThreads>
    </configuration>
</plugin>

Update your runner class as below:

@RunWith(Cucumber.class)
@CucumberOptions(features = {"src/test/java/features/"},
    glue = {"stepDefinitions"},
    monochrome = true, tags = {"@RegressionTest"}
)

TestNG:

However if you are using TextNG not only you can execute feature files in parallel but scenarios as well inside a feature file.

Add the Maven Surefire plugin configuration to the build section of the POM.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M1</version>
</plugin>

Then update your MainRunnerTest method as below:

@Override
    @DataProvider(parallel = true)
    public Object[][] scenarios() {
        return super.scenarios();
    }

How to Execute Cucumber Feature File Parallel