Hi,

A recent problem with a test suite made me think that there should be a way to timeout a test, and treat it as a failure
if it times out.

Our problem was that the test itself was hanging indefinitely. This is not ideal behavior, and could cause problems
with continuous integration software.

Personally, I would prefer that all tests be wrapped in timeout blocks, and be considered failures if they take more
than, say, 20 seconds. Make this configurable. Something like:

timeout => 30
def test_something
...
end

What do you think?

Regards,

Dan

Date: 2011-02-14 17:16
Sender: Daniel Berger

Thanks James. Any chance of integrating something like this?

Date: 2010-02-15 17:47
Sender: James Tucker

It is not unusual in IO / complex scheduling environments to
want to do this too, sometimes with a custom callback.

One example from eventmachine:

http://github.com/eventmachine/eventmachine/blob/3008ae6ea9e33abe
b9c7628d9b8a72256f55e725/tests/test_basic.rb#L155-162

Timeouts are a bit of a problem in ruby as they can violate ensure,
as well as have odd thread safety semantics.

SystemTimer is recommended if it's available, if not, this may
be of use, although I have not tested it to any extreme yet.

class SoftTimeout
  class Error < Timeout::Error; end

  def self.timeout(t = 10)
    thread = Thread.current
    mutex = Mutex.new
    timeout = Thread.new do
      sleep t
      mutex.synchronize { thread.raise Error, "Timeout after
#{t} seconds" }
    end
    v = nil
    begin
      v = yield
    ensure
      mutex.synchronize { timeout.kill }
    end
    v
  end
end

require "test/unit"

# require "soft_timeout"

class TestSoftTimeout < Test::Unit::TestCase
  def test_timeout
    start = Time.now.to_f
    assert_raise(SoftTimeout::Error) do
      SoftTimeout.timeout(0.02) do
        sleep
      end
    end
    finish = Time.now.to_f
    assert_in_delta(finish - start, 0.02, 0.005)
  end

  def test_completer
    a = false
    SoftTimeout.timeout do
      a = true
    end
    assert a
  end

  def test_threads_cleardown
    n = SoftTimeout.timeout { Thread.list.size }
    assert_equal 1, Thread.list.size
    assert_equal 2, n
  end

  def test_same_thread_running
    t = nil
    SoftTimeout.timeout {
      t = Thread.current
    }
    assert_equal Thread.current, t
  end

  def test_returns_value
    v = SoftTimeout.timeout { :value }
    assert_equal :value, v
  end

  def test_exception
    SoftTimeout.timeout do
      raise 'boom'
    end
  rescue RuntimeError
    assert_equal 'boom', $!.message
    assert_equal 1, Thread.list.size
  end

  def test_softimeout_error
    SoftTimeout.timeout(0) do
      sleep
    end
  rescue SoftTimeout::Error => e
    assert_equal "Timeout after 0 seconds", e.message
    assert_equal 1, Thread.list.size
  end
end