How to Effectively Force Minitest to Run My Tests in Order

How to run MiniTest::Unit tests in ordered sequence?

You can use i_suck_and_my_tests_are_order_dependent!() class method.

class MyTest < MiniTest::Unit::TestCase
  i_suck_and_my_tests_are_order_dependent!   # <----

  def test_1
    p 1
  end

  def test_2
    p 2
  end
end

But as the name suggest, it's not a good idea to make the tests to be dependant on orders.

How do you specify the exact order of Minitest tests?

i_suck_and_my_tests_are_order_dependent! (or def self.test_order; :alpha;end define an alphabetic order.

Beside the alphabetic order there is only a random order defined (at least in 'minitest', '>5.5.1', '<=5.6.1'.

But you can patch MiniTest::Test.runnable_methods to get another order.

gem 'minitest'
require 'minitest/autorun'

class MiniTest::Test
    #Add a test order :defined
    def self.runnable_methods
      methods = methods_matching(/^test_/)

      case self.test_order
      when :random, :parallel then
        max = methods.size
        methods.sort.sort_by { rand max }
      when :defined then  # <-new
        methods
      when :alpha, :sorted then
        methods.sort
      else
        raise "Unknown test_order: #{self.test_order.inspect}"
      end
    end
end

class TestOrder < MiniTest::Test
  def self.test_order; :defined; end

  #Alphabetic order
  #~ def self.test_order; :alpha;end
  #~ i_suck_and_my_tests_are_order_dependent!

  def test_4;    p __method__;  end
  def test_3;    p __method__;  end
  def test_2;    p __method__;  end
  def test_1;    p __method__;  end
end

But the test order is only defined per Test-subclass, not global for all tests. So this does not give you access to the order of test methods in multiple test classes.

I estimate you have different test-classes in your files, so this would correspond to your problem. (Not the files is the criteria, but the Test-class.)

If you define only one Test class in your test files then you have the possibility to define your own order.

How to make MiniTests run tests in alphabetical order of Test classes?

We figured out the solution just in case anyone is looking for one

Take a look at the following thread:

https://github.com/seattlerb/minitest/issues/514

Here is a gist from the thread just in case the link is broken:

Correct. As you can see in your gist, the test methods are still running in alphabetical order, but the test classes are not.

Test order dependencies are bugs in your tests that can and will lead to errors in production code. You should seriously consider fixing your tests so that each one is order independent. Full randomization with 100% success should be your goal. If you have a lot of tests, this can be a daunting task, but https://github.com/seattlerb/minitest-bisect can absolutely help with that.

If you're for some reason absolutely 100% dead-set on keeping your test order dependency (errors), you'll have to monkey patch Minitest.__run.

Add a initializer patching Minitest similar to the following

module Minitest
  def self.__run reporter, options
    suites = Runnable.runnables
    parallel, other = suites.partition { |s| s.test_order == :parallel }
    random, sorted  = other.partition  { |s| s.test_order == :random }

    sorted.map { |suite| suite.run reporter, options } +
      random.shuffle.map   { |suite| suite.run reporter, options } +
      parallel.shuffle.map { |suite| suite.run reporter, options }
  end
end

how to call a method immediately before the start of a test run in minitest

You are limited because of the behavior of the spec DSL. If you want to really control when your setup code is run along with the inherited setup, you should use the test-style setup and super.

Class A < Minitest::Spec

  def setup
    # do stuff
    setup_runtime_environment
  end

end

Class B < Class A

  def setup
    super
    # do stuff after setup_runtime_environment
  end

end

Class C < Class B

  def setup
    # do stuff before setup_runtime_environment
    super
  end

end

How to run code before each test case in all tests in MiniTest?

add_setup_hook was removed in 4.6.0.
https://github.com/seattlerb/minitest/commit/792a480ebeb32983b9150adae575b7c396e2ae63

Use before_setup instead.

How can I split test methods for reuse in multiple case with Minitest?

Great idea Keith Bennett ! Lambda is a good approach.

I have my solution now:

My module with lambda who run default tests:
default_tests.rb

module DefaultTest

 Run = ->(*variables_to_test) do
  describe "Default test" do
   variables_to_test.each do |variable|

    it "is Hash" do
      self.class.send(variable).must_be_instance_of Hash
    end

    it "no error return" do
      self.class.send(variable)['error'].must_be_nil
    end

   end
  end
 end

end

test_contacts.rb

module TestAircall
 describe Aircall::Contacts do

  def self.contact_by_id
   @contact_by_id ||= DefaultAircall::AIRCALL.contacts.get_by_id(ENV['TEST_DEFAULT_CONTACT_ID'])
  end

  def self.contact_by_phone_number
   @contact_by_phone_number ||= DefaultAircall::AIRCALL.contacts.get_by_phone_number(ENV['TEST_DEFAULT_PHONE_NUMBER'])
  end

  def self.contact_by_email
   @contact_by_email ||= DefaultAircall::AIRCALL.contacts.get_by_email(ENV['TEST_DEFAULT_EMAIL'])
  end

  DefaultTest::Run.('contact_by_id', 'contact_by_phone_number', 'contact_by_email')
 end
end

test_users.rb

module TestAircall
 describe Aircall::Users do
  def self.user_by_id
   @user_by_id ||= DefaultAircall::AIRCALL.users.get_by_id(ENV['TEST_DEFAULT_USER_ID'])
  end

  DefaultTest::Run.('user_by_id')
 end
end

I can call and run my default tests in every file test for any variable.

---

Related Topics