Speed up RSpec with set()

May 25, 2010

I enjoy using the let() method as it makes my specs easier to read and maintain than setting up instance variables in before(:each) blocks. The let() method can be used like this:

describe Account do
  let(:account) { Factory(:account) }

  subject { account }

  it { should be_enabled }

  context "when #disable!" do
    before do
      account.disable!
    end

    it { should be_disabled }
  end
end

My main concern was that the block gets evaluated everytime the method is called. In the example above, Factory(:account) will run and create a new record for every single spec.

To increase our specs performances let’s refactor this and setup the account in a before(:all) block.

describe Account do
  before(:all) do
    @account = Factory(:account)
  end

  let(:account) { @account.reload }
  #...
end

The account is now setup once before the specs get run. Each spec will be run in a separate transaction to prevent side effects. The account will be rolled back to its initial state before each spec then. Since ActiveRecord is not aware of the rollback we reload the account object from the database every time it’s called.

Specs are now faster but I want them to be as pretty as they were. Let’s make a little helper called set().

Sweeeet! You can now write the following:

describe Account do
  set(:account) { Factory(:account) }

  #...
end

The records created by set() will remain in your database. You can use DatabaseCleaner with the :truncation strategy to clean up your database. So far in RSpec 2.0, before(:all) runs before all describe/context/it while after(:all) runs after every single describe/context/it. Just make sure that you call DatabaseCleaner.clean in a before(:all) or after(:suite) blocks then. :)

I hope you’ll enjoy using this little helper. It’s very young and it has been tested with RSpec 2 only, so fill free to fill up the comments with enhancements and bug reports!

rspec-set is now available as a gem

. Bookmark the permalink. Both comments and trackbacks are currently closed.

http://wwww.21croissants.com Jean-Michel

Good post Phillip :-) On large projects, implementing that strategy can speed up a LOOOOOT tests!

In the last Apero ruby in Lyon, I explained I use a RSpec “plugin” to solve that same problem:

http://blog.lawrencepit.com/2009/06/08/machinery-to-create-object-graphs-and-speed-up-tests

This is small patch of RSpec but it also provide a cool DSL for creating objet graphs with sugar such as inheritance
http://wwww.21croissants.com Jean-Michel

One more thoint, I don’t fancy the name of the set method, it does not explain what it does:

let(attribute) { instance_variable_get(“@#{attribute}”).reload }

Code is DRY which is great but I am not sure new comers will catch immediatly that the method will create instances of test data. Machinery makes it a bit more obvious:

scenario :all_ships do
scenarios :earth_ships, :space_ships
@sunken_ship = Ship.create!(:name => “Sunk”)
end

Even if we don’t know when the scenarios are loaded, the initialization of @sunken_ship is obvious. I strongly advocate for using super simple code, especially in tests. I have spent enough hours debugging test code (RSpec controllers specs) to have learned the lesson the hard way!
- http://eggsonbread.com Philippe Creux
  
  Greg Bell doesn’t like the name either -_-.
  
  I like it because it’s very close to let. Also, let lazy evaluates (“let this variable initialize when needed”) while set evaluates once (“set this variable to this value”). Anyway, I didn’t find any better name! :)
  
  I feel that Machinery would add extra complexity to my specs… Maybe because I mimic it using factories inheritance.
http://wwww.21croissants.com Jean-Michel

1 more comment ;-)

Sounds like the shoulda community has been the same problem and found different solutions:
http://m.onkey.org/2009/9/20/make-your-shoulda-tests-faster-with-fast_context

I really like the idea of test seed data created with factories a la http://github.com/myronmarston/factory_data_preloader
Indeed, there are some test data set up which is shared by 80% of the tests, what’s the point of creating them through cucumber steps or RSpec before :each again and again …
- http://eggsonbread.com Philippe Creux
  
  Sweet! I love your comments. They are the ‘Useful links’ section of my posts. :)
  
  I agree with you for the test data shared by 80% of the tests. That’s what fixtures where made for right? :)
  
  For cucumber, I’m about to setup the data once before running the full test suite. Basically, I will Database.cleaner(:truncation), Factory(:...)... then rely on transactions to avoid any side effects.
  
  I might do the same with the specs…. I’ll keep you posted about that! :)
http://allpoetry.com Kevin

Interesting stuff. I’d been just doing this:

before(:all) do
@p = Item.make!
end

before(:each) {DatabaseCleaner.start}
after(:each) {DatabaseCleaner.clean}

Your let() syntax is definitely slicker, at least when only one object is created.

I’m curious that you had to do object.reload – I hadn’t been doing that, but perhaps this explains some of the odd issues I’ve been experiencing sometimes :)

P.s. check out the cranky factory replacement http://github.com/ginty/cranky – I love it!
Martin Streicher

Has this been updated for the latest Rspec 2? I tried the snippet and some of my variables previously defined by let() go missing using set().
- http://eggsonbread.com Philippe Creux
  
  This works with RSpec 2.0.
Martin Streicher

Where do I place set() to use it?
- http://eggsonbread.com Philippe Creux
  
  You can put set in spec/support/set.rb and require this file in spec/spec_helper.rb
http://kconrails.com Jaime Bellmyer
Hi Philippe! I enjoyed your previous best practices post, but I have to disagree with you here. First, I wanted to ask if you’re aware that you can use the subject freely in your specs? Your first example could be written as:
```
describe Account do
  subject { Factory(:account) }
 
  it { should be_enabled }
 
  context "when #disable!" do
    before do
      subject.disable!
    end
 
    it { should be_disabled }
  end
end
```
This eliminates the need for the let block altogether.

Also, the before(:all) block has a bad reputation because your tests are no longer atomic and independent. If you change part of the object in one test, it will be changed for subsequent tests in that group. For instance, the next test you add will start with the account being disabled. This may cause hard-to-find failures. Using before(:each) means being certain you’re not daisy-chaining tests that change the subject along the way.

One speed technique you might want to try is only creating objects on tests that really need them. *Most* tests can be run without the database. Your “it { should be_enabled }” test is one example.

Here’s how my version would look:
```
describe Account do
  subject { Factory.build(:account) }
 
  it { should be_enabled }
 
  context "#disable!" do
    it "should disable the account" do
      subject.should_not be_disabled
      subject.disable!
      subject.should be_disabled
    end
  end
end
```
I’m building the account instead of creating, since the first test doesn’t need it anyway. The disable! test will still work correctly, whether it’s saving the model or not. I’m assuming it probably does save, so this approach has removed 2 out of the 3 original database writes, while preserving the clean slate that each test should have when it starts.

I don’t have a before block in the context, because I don’t abstract setup code until there is more than one test requiring the same setup. I did give it its own context, though – I always do that for methods. It makes my tests look more consistent, and the test names are formatted better. And since I agree with your approach that each spec should test as *little* as possible, no method usually stays single-test for long :)

I also ensure that disabled? is returning false before the disable! call, like you expect it to. I always test before and after my method call, to ensure that things have changed correctly, and also catches times where I setup the test wrong to begin with.

Thanks for a great site, I look forward to reading more of your work!
- http://eggsonbread.com Philippe Creux
  
  Thanks for your comment Jaime.
  
  I’m not a big fan of using: subject.should .... I prefere account.should ... by far even if I have to add a let statement. :)
  
  I didn’t have much success with Factory.build. You can’t really use Factory.build whenever your model depends on others (an account must have a user, a balance and an account_email) or you use gems that saves object for you (state_machine does) or you use callbacks (disabling an account will suspend all transactions associated to it). And I want my integration tests to ensure that the changes are stored in the DB anyway. :)
  
  Concerning before(:all) vs before(:each) I totally agree with you. But when your integration specs takes more than 15 minutes to run, you have to speed that up. #set will keep your specs isolated as it reloads the object from the db before every test (and RSpec run tests in isolated db transactions). You can find more examples here: https://github.com/pcreux/rspec-set/blob/master/features/lib/rspec-set.feature
  
  There are some edge cases where the changes are propagated from one test to another though. Using before(:each) fix this kind of issues but we could also setup transactions around contexts. Xavier Shay did that for Postgres: http://rhnh.net/2010/10/06/transactional-before-all-with-rspec-and-datamapper. I’d love to find some time to implement that in rspec-set.
  
  Thanks again for your comment!
http://kconrails.com Jaime Bellmyer

…by the way, I do indent my code :) I was hoping the code tag would preserve indentations. I created a code snippet with the proper indenting. Thanks again!
- http://eggsonbread.com Philippe Creux
  
  The pre would have done it. I updated your comment. It looks gorgeous now. :)
http://kconrails.com Jaime Bellmyer

hahaha, thanks :)

I’m not opposed to the careful use of before(:all), and I certainly understand the compromises we have to make to keep our test suites running quickly. And you obviously understand that too.

My answer to the build issue has been mocking and stubbing, which I know can clutter the tests at times, and make it more difficult for junior developers to understand/maintain. I think in a perfect world, you wouldn’t deviate from before(:each) and I wouldn’t fake any of my models or associations :) The common culprit is test speed, and so far there isn’t a perfect answer.

I’m to the point where I want to start exploring the idea of serious multicore servers (onsite) dedicated to my company’s testing suites and used by our developers. Our rails shop is growing, so it might even be cost effective – or at least justifiable. Plus, I’ve always wanted a good excuse to own that kind of hardware!

Oh, there’s a gem called dataset you might like. It has the speed of fixtures because it loads the database the same way before tests, but it allows you to have *multiple* sets. So you can have the simplest set of records for a collection of tests, making it easy to remember what everything does. I don’t know if it’s been maintained, I’ll have to check because I could use it myself. And Postgres’ transactions might help, too. So many options, so little time.

Thanks for responding to my comments, it’s great talking to others with this level of dedication to the craft!
narshlob

What’s the difference between set() and let!()?
- http://eggsonbread.com Philippe Creux
  
  let! creates an object before each expectations
  set creates an object once and reload it before each expectation

Hi, my name is Philippe Creux.

This blog is about Agile project management, Ruby programming and other cool things. I haven't published anything here since 2011. Find the latest and greatest on pcreux.com!
Here & around
- Entries (RSS)
- Github
- LinkedIn
- Twitter

Speed up RSpec with set()

Hi, my name is Philippe Creux.

Here & around