Test runner in Rails 5

This blog is part of our Rails 5 series.

If you run bin/rails -h in a Rails 5 app, you will see a new command for running tests.

$ bin/rails -h
Usage: rails COMMAND [ARGS]

The most common rails commands are:
 generate    Generate new code (short-cut alias: "g")
 console     Start the Rails console (short-cut alias: "c")
 server      Start the Rails server (short-cut alias: "s")
 test        Run tests (short-cut alias: "t")

Before Rails 5, we had to use bin/rake test to run tests. But in Rails 5, we can use bin/rails test. It is not just replacement of old rake task but it is backed by a test runner inspired from RSpec, minitest-reporters, maxitest and others.

Let’s see what bin/rails test can do.

Running a single test

Now it is possible to run a single test out of the box using the line number of the test.

$ bin/rails test test/models/user_test.rb:27

Rails will intelligently run the test from user_test.rb having line number 27. Note that the line number 27 does not need to be first line of the test. Below is an example.

def test_valid_user
  hash = { email: 'bob@exmaple.com',
           first_name: 'John',
           last_name: 'Smith' }

  user = User.new hash

  assert user.valid?
end

In the above case test_valid_user can be run as long as line number provide is between 22 and 30.

Running multiple tests

You can also pass multiple test paths and Rails will run all of those tests.

$ bin/rails test test/models/user_test.rb:27 test/models/post_test.rb:42

It is also possible to run all tests from a directory.

$ bin/rails test test/controllers test/integration

Improved failure messages

When a test fails, Rails displays a command which can be used to just run the failed test.

$ bin/rails t
Run options: --seed 51858

# Running:

.F

Failure:
PostsControllerTest#test_should_get_new:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>


bin/rails test test/controllers/posts_controller_test.rb:15

I can simply copy bin/rails test test/controllers/posts_controller_test.rb:15 and rerun the failing test.

Failing fast

By default when a test fails then rails reports about the test failures and then moves on to the next test. If you want to stop running the test when a test fails then use option -f.

$ bin/rails t -f
Run options: -f --seed 59599

# Running:

..F

Failure:
PostsControllerTest#test_should_get_new:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>


bin/rails test test/controllers/posts_controller_test.rb:15

Interrupted. Exiting...


Finished in 0.179125s, 16.7481 runs/s, 22.3308 assertions/s.

3 runs, 4 assertions, 1 failures, 0 errors, 0 skips

Defer test output until the end of the full test run

By default when a test fails then rails prints F and then details about the failure like what assertion failed and how to re run the test etc.

If you want to have a clean output of . and F and would like all the test failures report to come at the every end then use option -d.

$ bin/rails t -d
Run options: -d --seed 29906

# Running:

..F...F

Finished in 0.201320s, 34.7704 runs/s, 49.6721 assertions/s.

  1) Failure:
PostsControllerTest#test_should_create_post [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19]:
"Post.count" didn't change by 1.
Expected: 3
  Actual: 2


  2) Failure:
PostsControllerTest#test_should_get_new [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:15]:
Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>

7 runs, 10 assertions, 2 failures, 0 errors, 0 skips

Failed tests:

bin/rails test test/controllers/posts_controller_test.rb:19
bin/rails test test/controllers/posts_controller_test.rb:15

Better backtrace output

By default when an error is encountered while running the test then the output does not contain full stacktrace. This makes debugging little bit difficult.

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007f86bc62b728>
    app/controllers/posts_controller.rb:29:in `create'
    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest

Now we can use -b switch, which will display complete backtrace of error message.

$ bin/rails t -b

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007fc53c4eb868>
    /rails-5-test-runner-app/app/controllers/posts_controller.rb:29:in `create'
    /sources/rails/actionpack/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
    /sources/rails/actionpack/lib/abstract_controller/base.rb:183:in `process_action'
    /sources/rails/actionpack/lib/action_controller/metal/rendering.rb:30:in `process_action'
    /sources/rails/actionpack/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
    /sources/rails/activesupport/lib/active_support/callbacks.rb:126:in `call'
.....
    /sources/rails/activesupport/lib/active_support/testing/assertions.rb:71:in `assert_difference'
    /rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'

Leveraging power of Minitest

The test runner also leverages power of minitest by providing some handy options.

Switch -s to provide your own seed

Now we can also provide our own seed using -s switch.

$ bin/rails t --s 42000

Switch -n to run matching tests

Switch -n will run tests matching the given string or regular expression pattern.

$ bin/rails t -n "/create/"
Run options: -n /create/ --seed 24558

# Running:

E

Error:
PostsControllerTest#test_should_create_post:
NameError: undefined local variable or method `boom' for #<PostsController:0x007faa39c2df90>
    app/controllers/posts_controller.rb:29:in `create'
    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'


bin/rails test test/controllers/posts_controller_test.rb:18

Finished in 0.073857s, 13.5396 runs/s, 0.0000 assertions/s.

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

Verbose output

It is also possible to see the verbose output using -v switch. It shows time required to run each test. This would help in detecting slow running tests.

$ bin/rails t -v
Run options: -v --seed 30118

# Running:

PostsControllerTest#test_should_destroy_post = 0.07 s = .
PostsControllerTest#test_should_update_post = 0.01 s = .
PostsControllerTest#test_should_show_post = 0.10 s = .
PostsControllerTest#test_should_create_post = 0.00 s = F

Failure:
PostsControllerTest#test_should_create_post:
"Post.count" didn't change by 1.
Expected: 3
  Actual: 2

bin/rails test test/controllers/posts_controller_test.rb:19

PostsControllerTest#test_should_get_new = 0.02 s = .
PostsControllerTest#test_should_get_index = 0.01 s = .
PostsControllerTest#test_should_get_edit = 0.00 s = .

Finished in 0.210071s, 33.3220 runs/s, 47.6028 assertions/s.

7 runs, 10 assertions, 1 failures, 0 errors, 0 skips

Colored output

Now by default we will get colored output. No need to add additional gem to colored output.

colored test output

With all these awesome features, testing Rails 5 apps has definitely become a better experience. Rails has shipped all these features within the framework itself so you don’t have to use multiple gems and libraries to achieve all of these things.