Rspec Hook After Report Gets Created

rspec hook after report gets created

One way you could do this is by implementing your own formatter based on the JSON formatter. Something like this could work:

class CustomFormatter < RSpec::Core::Formatters::JsonFormatter 
  RSpec::Core::Formatters.register self, :example_started

  def close(_notification)
    super

    # Do your post processing here...
  end
end

And then you can use your custom formatter like this

rspec --require ./custom_formatter.rb --format CustomFormatter

The RSpec::Core::Formatters::JsonFormatter is marked as private so it can change anytime. You have to think if you want to take the risk to need to change and adapt in a future RSpec upgrade.

Otherwise I would recommend to just use a custom script. It should be very simple with just && or | in it like

rspec --format json | ./run_postprocessing

https://relishapp.com/rspec/rspec-core/docs/formatters/custom-formatters
https://github.com/rspec/rspec-core/blob/main/lib/rspec/core/formatters/json_formatter.rb#L56

Unable to detrmine if a test failed in an after hook

Here is the answer (and an idea of the solution too) https://github.com/rspec/rspec-core/issues/2011#issuecomment-114669886:

Anyhow, the status is not set until AFTER the after hooks run. That is intentional because the after hook is itself part of the example, and if an exception occurs in the after hook we'll set the status of the example to :failed. after hooks are great for cleanup/teardown logic but isn't intended for observing the status of examples. Instead, I recommend you use a formatter for that...

When will the examples.txt file will be created in Rspec?

This is where it happens:
https://github.com/rspec/rspec-core/blob/e7c5d030966a7e8dad3e0a67c61920c4f2437c15/lib/rspec/core/runner.rb#L90

      # Configures and runs a spec suite.
      #
      # @param err [IO] error stream
      # @param out [IO] output stream
      def run(err, out)
        setup(err, out)
        return @configuration.reporter.exit_early(@configuration.failure_exit_code) if RSpec.world.wants_to_quit

        run_specs(@world.ordered_example_groups).tap do
          persist_example_statuses
        end
      end

If you explore run_specs in the same file (https://github.com/rspec/rspec-core/blob/e7c5d030966a7e8dad3e0a67c61920c4f2437c15/lib/rspec/core/runner.rb#L113) you'll see the hooks are already done before this method returns and the examples.txt is persisted. And no one anticipated your case, so no "after_persistence_file_saved" hook is there for you to use.

Some possible options would be:

monkey patch def persist_example_status method and do the extra copying (hacky, comes with all problems of monkeypatching), or
run rspec ... && cp reports/examples.txt reports.examples_1.txt (much simpler, less hacky, makes very little assumptions).
open a PR to rspec-core introducing a hook that you need (I don't think it will be likely to be accepted since much simpler solution exists)

Each has its pros and cons, but those seem to be out of the scope of this question.

Run before block once with RSpec 3

As the error message states, let and subject are specifically for managing per-example state. But before(:context)/before(:all) hooks get run outside the scope of any specific example, so they are fundamentally incompatible. If you want to use before(:context), you can't reference any let definitions from the hook. You'll have to manage the post_params state yourself without using let. Here's a simple way to do that:

require 'rails_helper'

RSpec.describe V1::UsersController do
  describe '#create' do
    before(:context) do
      @post_params = {
        first_nm: Faker::Name.first_name,
        last_nm: Faker::Name.last_name ,
        password: "test123456",
        password_confirmation: "test123456",
        email_address: Faker::Internet.email
      }

      post :create, params: @post_params
    end

    context 'successful create' do
      subject(:user) { User.find_by_email(@post_params[:email_address]) }

      it 'persists the user' do
        expect(user).not_to be_nil
      end

      it 'user data is correct' do
        @post_params.except(:password, :password_confirmation).each do |k, v|
          expect(user.send(k)).to eq(v)
        end
      end

      it 'returns responsde code of 201' do
        expect(response.status).to eq(201)
      end
    end
  end
end

That should solve your problem; however it's not the approach I would recommend. Instead, I recommend you use the aggregate_failures feature of RSpec 3.3+ and put all of this in a single example, like so:

require 'rails_helper'

RSpec.describe V1::UsersController do
  describe '#create' do
    let(:post_params) do
      {
        first_nm: Faker::Name.first_name,
        last_nm: Faker::Name.last_name ,
        password: "test123456",
        password_confirmation: "test123456",
        email_address: Faker::Internet.email
      }
    end

    it 'successfully creates a user with the requested params', :aggregate_failures do
      post :create, params: post_params

      expect(response.status).to eq(201)
      user = User.find_by_email(post_params[:email_address])
      expect(user).not_to be_nil

      post_params.except(:password, :password_confirmation).each do |k, v|
        expect(user.send(k)).to eq(v)
      end
    end
  end
end

aggregate_failures gives you a failure report indicating each expectation that failed (rather than just the first one like normal), just like if you had separated it into 3 separate examples, while allowing you to actually make it a single example. This allows you to incapsulate the action you are testing in a single example, allowing you to only perform the action once like you want. In a lot of ways, this fits better with the per-example state sandboxing provided by RSpec's features like before hooks, let declarations and the DB-transaction rollback provided by rspec-rails, anyway. And

I like the aggregate_failures feature so much that I tend to configure RSpec to automatically apply it to every example in spec_helper.rb:

RSpec.configure do |c|
  c.define_derived_metadata do |meta|
    meta[:aggregate_failures] = true unless meta.key?(:aggregate_failures)
  end
end

How to make Cucumber test continue running scenarios when RSpec ExpectationnotMet is thrown

Disclaimer: As Anthony pointed out, this is not best practice, but I'll assume you have a good reason for asking. :)

You need to build some form of error collector. Catch each RSpec exception and add that error to your collector. In an After hook, check to see if the collector contains something and fail the scenario if it does. You would also want to flesh out the error message collected to contain more information about which step failed and where it failed. This is just bare bones to give you an idea of what to do.

The key is to rescuing and logging your errors and then dealing with them later.

test.feature

  Scenario: Test out someting
    Given this step passes
    And this step has a collected error
    Then this step passes

test_stepdef.rb

Given(/^this step passes$/) do
  # keep on keeping on
end

Given(/^this step has a collected error$/) do
  begin
    1.should eq(0)
  rescue RSpec::Expectations::ExpectationNotMetError => e
    @collected_errors.push e.message
  end

end

support/hooks.rb

Before do |scenario|
  @collected_errors = []
end

After do |scenario|
  fail "#{@collected_errors}" unless @collected_errors.empty?
end

Output

  Scenario: Test out someting           # features/test.feature:6
    Given this step passes              # features/stepdefs/test_stepdef.rb:2
    And this step has a collected error # features/stepdefs/test_stepdef.rb:6
    Then this step passes               # features/stepdefs/test_stepdef.rb:2
      ["expected: 0
          got: 1
          (compared using ==)"] (RuntimeError)
      features/support/hooks.rb:21:in `After'

Failing Scenarios:
cucumber features/test.feature:6 # Scenario: Test out someting

RSpec record created seemingly before(:suite) how do I track this record down?

For the record of how I found the persistent record:

By using before(:suite) { debugger }, I removed the record and ran the entire test suite and notice a test was failing.

Within this test, was something similar to the following:

context "blah" do
    model = FactoryGirl.create(:model)
end

I simply put the code within a before(:each) block, and the test was passing and now so were mine.

TIL: Whenever RSpec loads the suite, it evaluates all the code that is not within a transaction. Which is why I could not try to pin point a problematic file by looking at when the record was being created.

Why are my view specs behaving inconsistently?

This is not a definitive answer: it appears that, for some weird reason, the evaluate method in the Nokogiri's Searchable module considers the concat() function as a custom XPath function instead of the internal XPath function.

External XPath functions are called as normal ruby methods in the context of the handler attribute of evaluate. Normally the handler is the SubstitutionContext class but in your case it seems that sometimes this gets the context of the TextHelper module where the concat method is defined, accepting just 1 parameter (whereas the XPath's concat() function accepts any number of parameters). I think that this can lead to the errors you observe.

Could you open the searchable.rb file (see the stack trace for its location) on this line and add before it some debug messages to inspect the contents of the following variables?

puts "ctx #{ctx.inspect}"
puts "path #{path.inspect}"
puts "handler #{handler.inspect}"
if handler.respond_to?(:concat)
  puts "concat #{handler.method(:concat).inspect}"
else
  puts "not responding to :concat"
end

Then, can you provide the output printed for one of the failing tests?

Update: the output showed that indeed the tests context was being polluted by the ActionView::TextHelpers module and in the end it turned out that there was a ActionView::Helpers module included in a model used in the tests.

Rspec Hook After Report Gets Created