In Ruby, How to I Control the Order in Which Test::Unit Tests Are Run

In Ruby, how to I control the order in which Test::Unit tests are run?

Name the tests you want to run first with a low-sorting alphabetical name.

def test_AAA_fizz

For code readability, this could be considered ugly, or helpful, depending on your point of view.

How to run MiniTest::Unit tests in ordered sequence?

You can use i_suck_and_my_tests_are_order_dependent!() class method.

class MyTest < MiniTest::Unit::TestCase
i_suck_and_my_tests_are_order_dependent! # <----

def test_1
p 1
end

def test_2
p 2
end
end

But as the name suggest, it's not a good idea to make the tests to be dependant on orders.

How do you specify the exact order of Minitest tests?

i_suck_and_my_tests_are_order_dependent! (or def self.test_order; :alpha;end define an alphabetic order.

Beside the alphabetic order there is only a random order defined (at least in 'minitest', '>5.5.1', '<=5.6.1'.

But you can patch MiniTest::Test.runnable_methods to get another order.

gem 'minitest'
require 'minitest/autorun'

class MiniTest::Test
#Add a test order :defined
def self.runnable_methods
methods = methods_matching(/^test_/)

case self.test_order
when :random, :parallel then
max = methods.size
methods.sort.sort_by { rand max }
when :defined then # <-new
methods
when :alpha, :sorted then
methods.sort
else
raise "Unknown test_order: #{self.test_order.inspect}"
end
end
end

class TestOrder < MiniTest::Test
def self.test_order; :defined; end

#Alphabetic order
#~ def self.test_order; :alpha;end
#~ i_suck_and_my_tests_are_order_dependent!

def test_4; p __method__; end
def test_3; p __method__; end
def test_2; p __method__; end
def test_1; p __method__; end
end

But the test order is only defined per Test-subclass, not global for all tests. So this does not give you access to the order of test methods in multiple test classes.

I estimate you have different test-classes in your files, so this would correspond to your problem. (Not the files is the criteria, but the Test-class.)

If you define only one Test class in your test files then you have the possibility to define your own order.

Why does Test::Unit.test_order= not working as expected?

I detected the solution, or better my fault:

gem 'test-unit'
require 'test/unit'
class Mytest < Test::Unit::TestCase
self.test_order = :defined
#~ self.test_order = :random
#~ self.test_order = :alphabetic #default
def test_b
p :b
end
def test_a
p :a
end
def test_c
p :c
end
end

The difference: I used test_order = :defined in my class.
What happened: A local variable test_order was created.

With self.test_order = :defined the method test_order= is called.

Execute code at end of Module/Class, like Ruby test/unit

Test::Unit uses at_exit for this, which runs code immediately before your application quits:

at_exit do
puts "printed before quit"
end

.. other stuff

I don't think there is any way to run code specifically after a class or module definition is closed.

how to call a method immediately before the start of a test run in minitest

You are limited because of the behavior of the spec DSL. If you want to really control when your setup code is run along with the inherited setup, you should use the test-style setup and super.

Class A < Minitest::Spec

def setup
# do stuff
setup_runtime_environment
end

end

Class B < Class A

def setup
super
# do stuff after setup_runtime_environment
end

end

Class C < Class B

def setup
# do stuff before setup_runtime_environment
super
end

end

How do I run a Test::Unit suite to completion with a tally of tests/failures?

Some more background would be helpful. I can't say I understand the behavior you're seeing.

I'm using both the core test/unit lib that comes with ruby 1.8, as well as several versions of the gem with ruby 1.9. The normal behavior for both of these is to run the entire loaded suite to completion and summarize the results.

Running a script that says require 'test/unit' will add an on-exit hook to run Test::Unit::AutoRunner with a Test::Unit::Collector::ObjectSpace collector (i.e. runs every instance of Test::Unit::TestCase currently loaded in the global object space).

It's also fairly easy to write your own custom test runner which manually loads your test classes and packs them into a Test::Unit::TestSuite, and capture its results.

But with every version of test/unit I've used, I've always seen the entire suite finish and report both failures and errors. In lieu of further info, I'd suggest experimenting with a single dummy test so see how you should expect test/unit to behave.

For example

require 'test/unit'
class Foo < Test::Unit::TestCase
def testFoo
flunk 'bad foo'
end
end
class Bar < Test::Unit::TestCase
def testBar
raise 'bar bar'
end
end

gives

Loaded suite foo
Started EF Finished in 0.001000 seconds.

1) Error:
testBar(Bar)
RuntimeError: bad bar
foo.rb:9:in `testBar`

2) Failure:
testFoo(Foo) [foo.rb:4]:
bad foo

2 tests, 1 assertions, 1 failures, 1 errors, 0 skips

Lastly: where are you trying to rescue/ensure? In the test methods? Under normal circumstances, there's no reason to catch an AssertionFailedError. This is for Test::Unit::TestCase to do, as a way to count failures and give you the report you want. Catching this interferes with what test/unit is written to do.



Related Topics



Leave a reply



Submit