Execute Unit Tests Serially (Rather Than in Parallel)

Execute Xunit tests in theory sequentially (not parallel)

One way to solve this problem is to make use the CollectionAttribute.

Unfortunately you can apply this attribute only for classes.

So, you would need a small refactoring like this:

internal class ValidateDeploymentConfigBase
{
    public async Task ValidateDeployConfigurationsTest(string deployConfigDirectoryPath)
    {
        // Arrange
        var currentDirectory = Directory.GetCurrentDirectory();
        var configurationFilePaths = Directory.GetFiles(deployConfigDirectoryPath);
        foreach (var configurationFilePath in configurationFilePaths)
        {
            var configurationFileName = Path.GetFileName(configurationFilePath);
            var destinationFilePath = Path.Combine(currentDirectory, configurationFileName);
            File.Copy(configurationFilePath, Path.Combine(currentDirectory, destinationFilePath), true);
            _testOutputHelper.WriteLine($"Copied file '{Path.GetFullPath(configurationFilePath)}' to '{destinationFilePath}'");
        }

        var hostBuilder = Program.CreateHostBuilder(null)
                                    .ConfigureWebHostDefaults(webHostBuilder =>
                                    {
                                        webHostBuilder.UseUrls(ServerBaseUrl);
                                        webHostBuilder.UseTestServer();
                                    });

        // Act
        using var host = await hostBuilder.StartAsync();
    }
}

And then your test cases would look like this:

[Collection("Sequential")]
internal class ValidateDevDeploymentConfig: ValidateDeploymentConfigBase
{
    [Fact]
    public async Task ValidateDeployConfigurationsTest(string deployConfigDirectoryPath)
    {
        base.ValidateDeployConfigurationsTest("../../../../../Configurations/Dev/");
    }
}

...

[Collection("Sequential")]
internal class ValidateProdDeploymentConfig : ValidateDeploymentConfigBase
{
    [Fact]
    public async Task ValidateDeployConfigurationsTest(string deployConfigDirectoryPath)
    {
        base.ValidateDeployConfigurationsTest("../../../../../Configurations/Prod/");
    }
}

Are tests executed in parallel in Go or one by one?

It's really easy to test it:

func Test1(t *testing.T) {
    fmt.Println("Test1 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test1 end")
}

func Test2(t *testing.T) {
    fmt.Println("Test2 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test2 end")
}

func Test3(t *testing.T) {
    fmt.Println("Test3 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test3 end")
}

Running it with go test, the output shows it's sequential:

Test1 start
Test1 end
Test2 start
Test2 end
Test3 start
Test3 end

So normal tests are executed one after the other, sequentially, but don't forget that order is not defined: How to run golang tests sequentially?

Also note that a test function can mark itself eligible for parallel execution, paralell with other tests that also do the same using the T.Parallel() method:

Parallel signals that this test is to be run in parallel with (and only with) other parallel tests.

So if we modify the above testing code to:

func Test1(t *testing.T) {
    t.Parallel()
    fmt.Println("Test1 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test1 end")
}

func Test2(t *testing.T) {
    t.Parallel()
    fmt.Println("Test2 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test2 end")
}

func Test3(t *testing.T) {
    fmt.Println("Test3 start")
    time.Sleep(time.Second * 2)
    fmt.Println("Test3 end")
}

Running it again with go test, the output is:

Test3 start
Test3 end
Test1 start
Test2 start
Test2 end
Test1 end

What does this prove? The order of tests is not defined, Test3 was executed first this time. And then Test1 and Test2 were run parallel.

There are some testing flags that control parallel execution. For example the -parallel flag specifies how many of these may run parallel. If you execute it with go test -parallel=1, the output will again become sequential, but the order will be Test3, Test1, Test2.

Also note that Go 1.7 introduced subtests and subbenchmarks. You can read more about this in blog post Using Subtests and Sub-benchmarks:

In Go 1.7, the testing package introduces a Run method on the T and B types that allows for the creation of subtests and sub-benchmarks. The introduction of subtests and sub-benchmarks enables better handling of failures, fine-grained control of which tests to run from the command line, control of parallelism, and often results in simpler and more maintainable code.

Subtests and subbenchmarks may run parallel, and there are a number of flags that may control their execution, e.g. -parallel, -p, -cpu. Run go help testflag to see the complete list of testing flags.

How to run unit tests (MSTest) in parallel?

Most of the answers on this page forget to mention that MSTest parallelizes tests in separate assemblies. You have to split your unittests into multiple .dll's to paralelize it.

But! The recent version - MSTest V2 - now CAN parallelize "in-assembly" (yay!) you just need to install a couple of nuget packages in your test project - TestFramework and TestAdapter - like described here https://blogs.msdn.microsoft.com/devops/2018/01/30/mstest-v2-in-assembly-parallel-test-execution/

And then simply add this to your test project

[assembly: Parallelize(Workers = 4, Scope = ExecutionScope.ClassLevel)]

EDIT: You can also disable parallel execution for a specific test using [DoNotParallelize] on a test method.

Running unit tests independently (.net core )

I finally found the answer here:
How to isolate EF InMemory database per XUnit test
Actually we want that our context not to be shared between the tests hence we have to make a new context at the beginning of each test like this:

using (var Context = new myContext(CreateNewContextOptions()))
{ 
   //here we implement Arrange, Act, and Assert parts 
}

CreateNewContextOptions() is actually a function that helps us to create a new context.

---

Related Topics