[rspec-users] specs pass individually, but fail using autotest

Doug E. doug at emeryit.com
Mon Nov 22 11:31:59 EST 2010



On Nov 22, 12:02 am, David Chelimsky <dchelim... at gmail.com> wrote:
> On Nov 20, 2010, at 9:49 PM, Doug E. wrote:
>
> > Problem discovered, but not solved. It isn't an rspec problem. It's
> > caused by nifty-authentication's use of Mocha's any_instance.
> > users_controller_spec.rb's last test calls:
>
> >    User.any_instance.stubs(:valid?).and_returns(true)
>


> > This is fine if the tests are run singly, but with autotest the call
> > in the controller spec still affects *every* subsequent User instance,
> > and so it's impossible later in the same autotest batch to test the
> > validity of later user objects. They just return true.
>
> > An aside: It seems that mocha should make it possible to undo the
> > effect of any_instance.stubs(:sym).
>
> AFAIK, this should happen automatically, assuming your suite is configured to use mocha. Do you have config.mock_with(:mocha) (usually in 'spec/spec_helper.rb')?
>
Thanks, David. I rewrote the tests to use rspec mocks. All the other
tests already use them.

I'm learning rspec 2, and am trying to stick strictly with rspec tests
and mocks. In this application, I've replaced the mocha based specs
created by the nifty scaffold generator with rspec scaffold specs.
Rspec, of course, doesn't have a generator parallel to the nifty
authentication generator, so I left those tests in place, not knowing
that the two would not play nice together. Now that I understand
what's happened, I've removed mocha and rewritten mocha-based specs.

Doug


>
>
> > On Nov 20, 9:49 pm, "Doug E." <d... at emeryit.com> wrote:
> >> On Nov 20, 8:43 pm, "Doug E." <d... at emeryit.com> wrote:
>
> >>> Hi,
> >>> I'm trying to run autotest and I'm using ryanb's nifty authentication
> >>> with Rails 3 and Rpec 2. When I run autotest, the user model spec
> >>> fails all its validation tests with the "error_on" matcher.
>
> >>> If I run the spec by itself, it passes. If I'm running autotest and
> >>> make a trivial change to the spec and save it, so that autotest  runs
> >>> just the user spec by itself, it passes. All the other tests in the
> >>> spec pass. If I hit Ctrl-C to rerun the tests, these tests will fail
> >>> again.
>
> >>> Only the ones of the form "new_user(:password => 'bad').should
> >>> have(1).error_on(:password)" fail.
>
> >>> Below is the output from autotest, including the initial run, and
> >>> second run after I make a trivial change to the spec.
>
> >>> Thanks.
> >>> Doug
>
> >>> Failures:
> >>>   1) User should require username
> >>>      Failure/Error: new_user(:username => '').should
> >>> have(1).error_on(:username)
> >>>      expected 1 error on :username, got 0
> >>>      # ./spec/models/user_spec.rb:21:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   2) User should require password
> >>>      Failure/Error: new_user(:password => '').should
> >>> have(1).error_on(:password)
> >>>      expected 1 error on :password, got 0
> >>>      # ./spec/models/user_spec.rb:25:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   3) User should require well formed email
> >>>      Failure/Error: new_user(:email => 'foo at b...@example.com').should
> >>> have(1).error_on(:email)
> >>>      expected 1 error on :email, got 0
> >>>      # ./spec/models/user_spec.rb:29:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   4) User should validate uniqueness of email
> >>>      Failure/Error: new_user(:email => '... at example.com').should
> >>> have(1).error_on(:email)
> >>>      expected 1 error on :email, got 0
> >>>      # ./spec/models/user_spec.rb:34:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   5) User should validate uniqueness of username
> >>>      Failure/Error: new_user(:username => 'uniquename').should
> >>> have(1).error_on(:username)
> >>>      expected 1 error on :username, got 0
> >>>      # ./spec/models/user_spec.rb:39:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   6) User should not allow odd characters in username
> >>>      Failure/Error: new_user(:username => 'odd ^&(@)').should
> >>> have(1).error_on(:username)
> >>>      expected 1 error on :username, got 0
> >>>      # ./spec/models/user_spec.rb:43:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   7) User should validate password is longer than 3 characters
> >>>      Failure/Error: new_user(:password => 'bad').should
> >>> have(1).error_on(:password)
> >>>      expected 1 error on :password, got 0
> >>>      # ./spec/models/user_spec.rb:47:in `block (2 levels) in <top
> >>> (required)>'
>
> >>>   8) User should require matching password confirmation
> >>>      Failure/Error: new_user(:password_confirmation =>
> >>> 'nonmatching').should have(1).error_on(:password)
> >>>      expected 1 error on :password, got 0
> >>>      # ./spec/models/user_spec.rb:51:in `block (2 levels) in <top
> >>> (required)>'
>
> >>> Finished in 3.46 seconds
> >>> 111 examples, 8 failures, 4 pending
> >>> bundle exec /Users/doug/.rvm/rubies/ruby-1.9.2-p0/bin/ruby -S /Users/
> >>> doug/.rvm/gems/ruby-1.9.2-p0 at class-proto/gems/rspec-core-2.0.1/bin/
> >>> rspec --autotest '/Users/doug/dev/hg/class-proto/spec/models/
> >>> user_spec.rb'
> >>> ...Trivial change
> >>> ...........
>
> >>> Finished in 1.17 seconds
> >>> 14 examples, 0 failures
> >>> _______________________________________________
> >>> rspec-users mailing list
> >>> rspec-us... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/rspec-users
>
> >> I'll add that this behavior only occurs with the model User. I set up
> >> some similar tests for other classes (named Sample and
> >> SampleClassification). These do not have the same problem. Their
> >> validation tests pass during batch and single test runs.
>
> >> I created a new User model from scratch with a single
> >> validates_presence_of call, and got the same unexpected behavior: in
> >> single spec runs they pass, and in batch runs they fail.
>
> >> Doug
> >> _______________________________________________
> >> rspec-users mailing list
> >> rspec-us... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/rspec-users
> > _______________________________________________
> > rspec-users mailing list
> > rspec-us... at rubyforge.org
> >http://rubyforge.org/mailman/listinfo/rspec-users
>
> Cheers,
> David
>
> _______________________________________________
> rspec-users mailing list
> rspec-us... at rubyforge.orghttp://rubyforge.org/mailman/listinfo/rspec-users


More information about the rspec-users mailing list