Bugs: Browse | Submit New | Admin

[#9300] exit called from at_exit blocks does not properly return the exit code

Date:
2007-03-15 05:35		 Priority:
3
Submitted By:
Jeff Xiong (gigix) 		Assigned To:
Nathaniel Talbott (ntalbott)
Category:
Language / Runtime / Core Libraries		 State:
Closed
Platform:
  
Summary:
exit called from at_exit blocks does not properly return the exit code

Detailed description 
If create a sub-process, and run failing tests in it, the sub-process will not exit with error code as expected: the
exit code is always 0.

Ubuntu Linux 7.04
Ruby 1.8.6
Rails 1.2.3

Add A Comment: Notepad

Please login


Followup

Message 
Date: 2007-06-14 03:44
Sender: Aleksey Kondratenko

Here's simple workaround for Test::Unit which I used to fix CI
our process. Apply against ruby's test/unit.rb.

--- unit.rb-old 2007-02-13 01:01:00.000000000 +0200
+++ unit.rb     2007-06-14 10:16:19.000000000 +0300
@@ -273,8 +273,14 @@
   end
 end

+unless ($test_unit_exit_cont = (callcc {|c| c})).kind_of?
Continuation
+  exit $test_unit_exit_cont
+end
+
+
 at_exit do
   unless $! || Test::Unit.run?
-    exit Test::Unit::AutoRunner.run
+    rv = Test::Unit::AutoRunner.run
+    $test_unit_exit_cont.call(rv)
   end
 end

Date: 2007-03-26 15:54
Sender: Brian Takita

Nm. I didn't read the other comments.
Date: 2007-03-26 14:27
Sender: Brian Takita

Heres a simpler reproduction.

at_exit do
  exit 1
end

Date: 2007-03-26 13:53
Sender: Brian Takita

I'm also getting this issue on Ubuntu 6.0.4 and Ruby 1.8.6.

Rails is not loaded.

Here is my reproduction:

require 'test/unit'
at_exit do
  unless $! || Test::Unit.run?
    begin
      result = Test::Unit::AutoRunner.run
      exit result
    rescue => e
      $stderr.puts e.message
      $stderr.puts e.backtrace
      exit 1
    end
  end
end

class Test::Unit::AutoRunner
  def run
    raise "Foobar"
  end
end

We overrode at_exit to properly handle exceptions in Autorunner.
Btw, this also has an exit code of 0.

require 'test/unit'
class Test::Unit::AutoRunner
  def run
    return false
  end
end

Date: 2007-03-26 12:17
Sender: Nathaniel Talbott

Nobu has checked in a fix (and tests) for this now, so I'm closing
it. The issue of a maintenance release is still open, and is
being discussed on ruby-core.
Date: 2007-03-23 10:32
Sender: Nathaniel Talbott

Nobu posted a fix in ruby-core:10748, and I posted a failing
test case in ruby-core:10760. I will close this bug once Nobu
has committed the fix.
Date: 2007-03-23 00:27
Sender: Alexey Verkhovsky

Finally, the same problem is reproducible on the current 1.8
branch
Date: 2007-03-22 23:39
Sender: Alexey Verkhovsky

OK, take Test::Unit out of the equation, too. Ruby 1.8.6 ignores
the argument to exit(), if inside the at_exit block.

alexeyv@alexeyv-laptop:~/foo$ cat at_exit_false.rb 
puts 'here'

at_exit do
  exit false
end

alexeyv@alexeyv-laptop:~/foo$ ruby at_exit_false.rb 
here
alexeyv@alexeyv-laptop:~/foo$ echo $?
0

Date: 2007-03-22 23:25
Sender: Alexey Verkhovsky

Hmm, I don't see a way to attach a file to a comment.

Here is the test:

alexeyv@alexeyv-laptop:~/foo$ cat test.rb 
require 'test/unit'

class FooTest < Test::Unit::TestCase
  def test_that_fails
    fail 'oops'
  end
end

Here is the console session that demonstrates the problem:

alexeyv@alexeyv-laptop:~$ cd foo
alexeyv@alexeyv-laptop:~/foo$ which ruby
/usr/bin/ruby
alexeyv@alexeyv-laptop:~/foo$ 'ruby -v
> 
alexeyv@alexeyv-laptop:~/foo$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]
alexeyv@alexeyv-laptop:~/foo$ ./run_test_and_print_exit_code.sh 
Loaded suite test
Started
E
Finished in 0.000679 seconds.

  1) Error:
test_that_fails(FooTest):
RuntimeError: oops
    test.rb:5:in `test_that_fails'

1 tests, 0 assertions, 0 failures, 1 errors
1
alexeyv@alexeyv-laptop:~/foo$ switch_ruby
alexeyv@alexeyv-laptop:~/foo$ which ruby
/home/alexeyv/ruby1.8.6/bin/ruby
alexeyv@alexeyv-laptop:~/foo$ ruby -v
ruby 1.8.6 (2007-03-05 patchlevel 0) [i686-linux]
alexeyv@alexeyv-laptop:~/foo$ ./run_test_and_print_exit_code.sh 
Loaded suite test
Started
E
Finished in 0.000614 seconds.

  1) Error:
test_that_fails(FooTest):
RuntimeError: oops
    test.rb:5:in `test_that_fails'

1 tests, 0 assertions, 0 failures, 1 errors
0
^^^
THERE SHOULD BE '1' INSTEAD OF '0' HERE
Date: 2007-03-22 23:20
Sender: Alexey Verkhovsky

By the way, note that the issue Jeff talks about is a zero exit
code returned by a FAILING test. Your printout seems to be checking
exit codes of passing tests.
Date: 2007-03-22 23:17
Sender: Alexey Verkhovsky

Nathaniel, hi,

I'm reproducing the same issue, too. With Ruby 1.8.6 release. 

Attached archive (foo.tgz) contains a very simple failing test,
and a script that runs it and prints out the exit code. No Rails,
Rake or any other third-party stuff involved. At all.

I'm going to try figure out the reason now.

Alex Verkhovsky
Date: 2007-03-21 22:50
Sender: Nathaniel Talbott

It's working for me, under 1.8.4, 1.8.5 and 1.8.6:

  ntalbott@joshua:~/tmp/issue_refined$ multiruby process_test.rb

  VERSION = 1.8.4

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.010974 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  VERSION = 1.8.5

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.01149 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  VERSION = 1.8.6

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.011605 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  TOTAL RESULT = 0 failures

I'm not sure what to tell you - it appears that for some reason
your Ruby is broken. Can you try building Ruby from scratch on
your Linux box and seeing if you get the same error?

I'm going to re-close this - if you can figure out more specifically
what the issue is with your box, please re-open it and I'll
investigate again.
Date: 2007-03-21 08:04
Sender: Jeff Xiong

I wrote a new test case for this issue. It doesn't depend on
any other libraries. And it seems to me like at_exit
structure has some problem with it. With following code:

at_exit do
  p "I'm going to die..."
  exit false
end

the exit code should be non-zero. But in my environment
(Ruby 1.8.6 on Linux and Windows) the exit code is 0.

Please check out the second attachment I uploaded
(issue_refined.zip) and try it out.
Date: 2007-03-20 22:47
Sender: Nathaniel Talbott

I'm closing this in lieu of a reproducible failure. Feel free
to re-open it if/when you can provide one.
Date: 2007-03-19 16:33
Sender: Nathaniel Talbott

Can you give me an example that runs independent of a Rails app?
For instance, you must have something extra installed in your
app, since assert_false is not a standard test/unit assertion.

In my testing, both in a Rails app, and an independent test I
set up, I couldn't duplication the problem.
Date: 2007-03-17 00:56
Sender: Jeff Xiong

I posted an attachment (issue.zip). You can unzip it to a
Rails app's root directory, then run process_test.rb.
Date: 2007-03-16 10:01
Sender: Nathaniel Talbott

Can you post an example that demonstrates the problem?

Attached Files:

Name Description Download
issue.zip put files to root dir of any rails app, then run process_test.rb Download
issue_refined.zip unpack files to any dir, then run "ruby process_test.rb" Download

Changes:

Field Old Value Date By
close_date2007-03-26 12:172007-03-26 12:17ntalbott
resolution_idRejected2007-03-26 12:17ntalbott
status_idOpen2007-03-26 12:17ntalbott
status_idClosed2007-03-23 10:32ntalbott
summarysub-process with Test::Unit does not exit error code as expected2007-03-23 10:32ntalbott
artifact_group_id1.8.x2007-03-23 10:32ntalbott
status_idOpen2007-03-21 22:50ntalbott
close_date2007-03-21 22:502007-03-21 22:50ntalbott
status_idClosed2007-03-21 08:04gigix
File Added1600: issue_refined.zip2007-03-21 08:04gigix
assigned_tonone2007-03-20 22:47ntalbott
resolution_idNone2007-03-20 22:47ntalbott
close_date2007-03-20 22:472007-03-20 22:47ntalbott
status_idOpen2007-03-20 22:47ntalbott
File Added1581: issue.zip2007-03-15 05:35gigix