From headius at gmail.com Mon Jan 3 10:06:21 2005 From: headius at gmail.com (Charles O Nutter) Date: Mon Jan 3 10:04:21 2005 Subject: [Rubytests-devel] Re: rubicon/builtin TestIO.rb In-Reply-To: <200501031213.j03CDTdS006187@rubyforge.org> References: <200501031213.j03CDTdS006187@rubyforge.org> Message-ID: We'll try to visit the select tests from a JRuby perspective. Obviously we don't have a specific select implementation, since it's JVM-based, but we may try to emulate some level of support. - Charlie On Mon, 03 Jan 2005 12:13:29 +0000, holmberg@rubyforge.org wrote: > Update of /var/cvs/rubytests/rubicon/builtin > In directory rubyforge.org:/tmp/cvs-serv6094/builtin > > Modified Files: > TestIO.rb > Log Message: > The detailed operation of select(2) on different platforms seem to > vary quite a lot. The testcase "test_s_select" have now been modified > to reflect this. The platforms with "different" behaviour are Solaris > and Windows. > > This change to the testcase was done to eliminate a "phony" error > in Rubicon (I guess Ruby will never try to enforce the the same behaviour > when the underlying platform differs, as it is for "select"). > > Index: TestIO.rb > =================================================================== > RCS file: /var/cvs/rubytests/rubicon/builtin/TestIO.rb,v > retrieving revision 1.8 > retrieving revision 1.9 > diff -C2 -d -r1.8 -r1.9 > *** TestIO.rb 14 Dec 2004 17:12:54 -0000 1.8 > --- TestIO.rb 3 Jan 2005 12:13:27 -0000 1.9 > *************** > *** 266,270 **** > File.open(@file) do |file| > res = IO.select([file], [$stdout, $stderr], [file,$stdout,$stderr], 1) > ! assert_equal([[file], [$stdout, $stderr], []], res) > end > > --- 266,296 ---- > File.open(@file) do |file| > res = IO.select([file], [$stdout, $stderr], [file,$stdout,$stderr], 1) > ! assert_equal( 3, res.length ) > ! assert_equal( [file], res[0] ) > ! assert_equal( [$stdout, $stderr], res[1] ) > ! > ! case > ! when $os == Solaris > ! # select is documented to work this way on Solaris. > ! # From the select(3C) man page: > ! # > ! # File descriptors associated with regular files always > ! # select true for ready to read, ready to write, and error > ! # conditions. > ! # > ! assert_equal( [file], res[2] ) > ! > ! when $os == MsWin32 > ! # seems to work like this on Windows, but I have not found any > ! # documentation supporting it. > ! # > ! assert_equal( [file, $stdout, $stderr], res[2]) > ! > ! else > ! # The "normal" case for Linux, FreeBSD, etc. > ! # > ! assert_equal( [], res[2] ) > ! > ! end > end > > From headius at gmail.com Mon Jan 3 10:09:45 2005 From: headius at gmail.com (Charles O Nutter) Date: Mon Jan 3 10:07:44 2005 Subject: [Rubytests-devel] Anything needed for a release Message-ID: For all the rubytests people out there: Is there any more you think we should complete before putting out a release? This isn't really intended to be a major milestone release, but more a baseline for bringing anyone using the tests up-to-date, and providing a starting point for our ongoing changes. The last release was in April 2004, so there's obviously been a lot of time passed since then. This release would include recent test cleanup, threading fixes, some JRuby stuff, and whatever else you all think should be in it. It would be numbered 0.37 and that's about all there is to say. Anything you all want in 0.37 before we do a release? - Charlie From holmberg at iar.se Mon Jan 3 10:25:53 2005 From: holmberg at iar.se (Johan Holmberg) Date: Mon Jan 3 10:24:00 2005 Subject: [Rubytests-devel] [ANN] CVS-commit email list Message-ID: Hi ! There is now a CVS commit email list up-and-running. You can subscribe at: http://rubyforge.org/mailman/listinfo/rubytests-commit The new list can also be reached via the project page (the same way this list is found). The list receives an email for each CVS commit done in our project. /Johan Holmberg From holmberg at iar.se Mon Jan 3 13:12:18 2005 From: holmberg at iar.se (Johan Holmberg) Date: Mon Jan 3 13:10:24 2005 Subject: [Rubytests-devel] Anything needed for a release In-Reply-To: References: Message-ID: On Mon, 3 Jan 2005, Charles O Nutter wrote: > > Anything you all want in 0.37 before we do a release? > > - Charlie > I think we should agree on which "platforms" and which Ruby versions we think the tests "work well" on. The versions I use regularly are: Ruby: 1.8.2 platforms: FreeBSD 5.3 Linux Redhat 9 Solaris 8 Windows XP & VC7 What about other versions of Ruby ? Do we care ? I have occasionally tried with both 1.8.1 and "latest from CVS" (what will become 1.9). But it was a while since I tried last time. And what other plaforms? JRuby of course since Charles&Co use that. But other than that? I think we should state in the release what we think about these matters. I also think we should have a list of what we think gives errors, at least when running with Ruby 1.8.2. I get the following errors just now: FreeBSD 5.3 & Ruby 1.8.2: - everything passes Solaris 8 & Ruby 1.8.2: - one error in builtin/TestMath.rb: + Math.log(0) gives an exception (Errno::ERANGE) instead of -Infinity (as on FreeBSD) Linux Redhat 9 & Ruby 1.8.2: - one error in builtin/TestMath.rb: + Math.log(0) gives an exception (Errno::ERANGE) instead of -Infinity (as on FreeBSD) Windows XP & Ruby 1.8.2: - error related to "hard links" on NTFS: + builtin/TestFileTest.rb in method `test_hlink' + builtin/TestFile__Stat.rb in method `test_nlink' - errors in "test_s_exec" and "test_s_system" having to do with wildcard expansion of filenames on the command line - one error in builtin/TestMath.rb: + Math.log(0) gives an exception (Errno::ERANGE) instead of -Infinity (as on FreeBSD) Perhaps we should change what is considered correct for Math.log(0), so Linux, Solaris and Windows becomes right, and just FreeBSD wrong. I guess that is what most users (Linux + Windows) live with ... /Johan From holmberg at iar.se Mon Jan 3 13:18:55 2005 From: holmberg at iar.se (Johan Holmberg) Date: Mon Jan 3 13:23:48 2005 Subject: [Rubytests-devel] Anything needed for a release In-Reply-To: References: Message-ID: On Mon, 3 Jan 2005, Charles O Nutter wrote: > > Anything you all want in 0.37 before we do a release? > Perhaps we should update the ChangeLog. I have never done that as part of my checkins. Would it be OK to run "cvs2cl" now afterwards? Or is there a better way to maintain that file ? /Johan From holmberg at iar.se Tue Jan 4 05:11:57 2005 From: holmberg at iar.se (Johan Holmberg) Date: Tue Jan 4 05:10:06 2005 Subject: [Rubytests-devel] Anything needed for a release In-Reply-To: References: Message-ID: On Mon, 3 Jan 2005, Johan Holmberg wrote: > > Windows XP & Ruby 1.8.2: > > - error related to "hard links" on NTFS: > > + builtin/TestFileTest.rb in method `test_hlink' > > + builtin/TestFile__Stat.rb in method `test_nlink' > > - errors in "test_s_exec" and "test_s_system" having to do with > wildcard expansion of filenames on the command line > > > - one error in builtin/TestMath.rb: > > + Math.log(0) gives an exception (Errno::ERANGE) > instead of -Infinity (as on FreeBSD) > I forgot: - errors in builting/TestThread.rb: + in the the test methods test_abort_on_exception= and test_s_abort_on_exception=. I think the error has nothig to do with threads, and instead is a problem with the test method itself (and the "File.reopen" method on Windows). /Johan Holmberg From headius at gmail.com Tue Jan 4 09:59:11 2005 From: headius at gmail.com (Charles O Nutter) Date: Tue Jan 4 10:07:03 2005 Subject: [Rubytests-devel] Anything needed for a release In-Reply-To: References: Message-ID: I think we should focus on Ruby 1.8.2 and the upcoming JRuby 0.8.0 release as versions. Naturally, JRuby still doesn't pass most of the tests, so this is somewhat of a formality. We're really just saying "this release is what we recommend using to test anything based on Ruby 1.8.x, with a focus on Ruby 1.8.2+ and 0.8.0". I agree with listing the known failures on the platforms you list. I don't think we need a focus on 1.9 because we haven't had a development focus on 1.9. As 1.9 moves closer to 2.0, we may need to consider branching the tests for 2.0, to avoid having to special-case a bunch of tests for release version. Then there would be a 2.0 line of tests and a 1.8 line of tests, that test each of those specified platforms. I wasn't able to find a good discussion about the correct behavior of log(0) on various Ruby platforms, but I tend toward the negative infinity version as being correct. The Java log function performs the same way, and this is one of the key examples of why there's a value for "negative infinity" in IEEE floating point math. I am inclined to think the other platforms are valid failures, although I can't comment on whether those failures are avoidable. Pickaxe2 doesn't say anything about expected values for inputs of NaN, negative nonzero, negative or positive zero, or positive infinity, which are all possible inputs. - Charlie On Mon, 3 Jan 2005 19:18:55 +0100 (MET), Johan Holmberg wrote: > > On Mon, 3 Jan 2005, Charles O Nutter wrote: > > > > Anything you all want in 0.37 before we do a release? > > > > Perhaps we should update the ChangeLog. > I have never done that as part of my checkins. > > Would it be OK to run "cvs2cl" now afterwards? > Or is there a better way to maintain that file ? > > /Johan > > _______________________________________________ > Rubytests-devel mailing list > Rubytests-devel@rubyforge.org > http://rubyforge.org/mailman/listinfo/rubytests-devel > From holmberg at iar.se Wed Jan 5 10:10:22 2005 From: holmberg at iar.se (Johan Holmberg) Date: Wed Jan 5 10:08:30 2005 Subject: [Rubytests-devel] Testing on Apples OS-X? Message-ID: Is there anybody on the list with access to an OS-X machine that could test Ruby 1.8.2 with the current Rubicon testcases? I think it would interesting to know if it works as good as on FreeBSD and Linux. /Johan From chad at chadfowler.com Wed Jan 5 11:40:50 2005 From: chad at chadfowler.com (chad@chadfowler.com) Date: Wed Jan 5 10:34:34 2005 Subject: [Rubytests-devel] Testing on Apples OS-X? In-Reply-To: References: Message-ID: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> > > Is there anybody on the list with access to an OS-X machine that > could test Ruby 1.8.2 with the current Rubicon testcases? > > I think it would interesting to know if it works as good as on > FreeBSD and Linux. > I'll try to do it tonight when I get home. I don't have my Mac at work. Chad From headius at gmail.com Wed Jan 5 15:46:00 2005 From: headius at gmail.com (Charles O Nutter) Date: Wed Jan 5 15:43:55 2005 Subject: [Rubytests-devel] Testing on Apples OS-X? In-Reply-To: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> References: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> Message-ID: I'd be interesting in seeing 1.8.1 results on OS X as well, for posterity. - Charlie On Wed, 5 Jan 2005 11:40:50 -0500 (EST), chad@chadfowler.com wrote: > > > > Is there anybody on the list with access to an OS-X machine that > > could test Ruby 1.8.2 with the current Rubicon testcases? > > > > I think it would interesting to know if it works as good as on > > FreeBSD and Linux. > > > > I'll try to do it tonight when I get home. I don't have my Mac at work. > > Chad > > > _______________________________________________ > Rubytests-devel mailing list > Rubytests-devel@rubyforge.org > http://rubyforge.org/mailman/listinfo/rubytests-devel > From chad at chadfowler.com Wed Jan 5 21:02:39 2005 From: chad at chadfowler.com (Chad Fowler) Date: Wed Jan 5 21:00:46 2005 Subject: [Rubytests-devel] Testing on Apples OS-X? In-Reply-To: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> References: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> Message-ID: <0AF632D1-5F87-11D9-B291-000D9337C01C@chadfowler.com> On 05-Jan-05, at 11:40 AM, chad@chadfowler.com wrote: >> >> Is there anybody on the list with access to an OS-X machine that >> could test Ruby 1.8.2 with the current Rubicon testcases? >> >> I think it would interesting to know if it works as good as on >> FreeBSD and Linux. >> > > I'll try to do it tonight when I get home. I don't have my Mac at > work. > > Chad > > $ ruby -v ruby 1.8.2 (2004-12-25) [powerpc-darwin7.7.0] ======================================================================== All Tests Test Results V0.3.5 ======================================================================== Name OK? Tests Asserts Failures Errors ------------------------------------------------------------ Rubicon::TestCase 1 0 AccessControl 3 18 Array 69 614 Arrays 1 0 Assignment 4 75 BasicExpressions 3 15 Bignum 29 1598 Binding 1 0 BlocksProcs 3 61 BooleanExpressions 1 2 CatchThrow 1 1 Class 5 16 Classes 1 0 Comparable 7 26 Constants 2 15 Continuation 5 5 Dir FAIL 21 114 1 Enumerable 23 176 Eval 8 39 Exception 6 16 Exceptions 6 6 FalseClass 6 17 File 33 323 FileTest 4 136 File__Stat FAIL 35 352 1 Fixnum 35 414 Flip 6 28 Float 28 178 Floats 3 34 GC 4 1 Hash 44 296 Hashes 1 0 IO FAIL 53 1229 1 IfUnless 1 2 Integer 9 347 Integers 1 0 Invocation 5 11 KanjiIndex 20 190 Kernel 109 12880 LoopStuff 15 65 Marshal 6 31 MatchData 11 18 Math 2 62 Method 4 11 Methods 2 6 Module 25 101 ModulePrivate 15 41 Modules 1 0 Names 1 0 NilClass 8 21 Numeric 10 21 Object 2 11 ObjectSpace 4 6 PredefinedVariables 1 3 Proc 4 5 Range 14 159 Ranges 1 0 Regexp 14 39 RegularExpressions 3 2993 Scope 12 27 SourceLayout 7 23 String 87 1252 Strings 5 27 Struct 11 47 Struct__Tms 1 9 Symbol 8 20 Symbols 1 0 Thread 33 103 ThreadGroup 4 7 Time FAIL 41 1820 1 TrueClass 6 17 ======================================================================== All 71 files FAIL 956 26180 4 0 ======================================================================== Failure Report ======================================================================== TestDir: -------- ./builtin/TestDir.rb:172:in `test_s_chdir' ./builtin/TestDir.rb:161:in `chdir' ./builtin/TestDir.rb:161:in `test_s_chdir' ./builtin/TestDir.rb:158:in `chdir' ./builtin/TestDir.rb:158:in `test_s_chdir' .... exception expected but was Class: Message: <"Permission denied - getcwd"> ---Backtrace--- ./builtin/TestDir.rb:173:in `chdir' ./builtin/TestDir.rb:173:in `test_s_chdir' ./builtin/TestDir.rb:172:in `assert_raise' ./builtin/TestDir.rb:172:in `test_s_chdir' ./builtin/TestDir.rb:161:in `chdir' ./builtin/TestDir.rb:161:in `test_s_chdir' ./builtin/TestDir.rb:158:in `chdir' ./builtin/TestDir.rb:158:in `test_s_chdir' --------------- ------------------------------------------------------------ TestFile__Stat: --------------- ./builtin/TestFile__Stat.rb:219:in `test_pipe?' ./builtin/TestFile__Stat.rb:218:in `popen' ./builtin/TestFile__Stat.rb:218:in `test_pipe?' ./builtin/TestFile__Stat.rb:217:in `dont' ./builtin/TestFile__Stat.rb:217:in `test_pipe?' .... expected but was . ------------------------------------------------------------ TestIO: ------- ./builtin/TestIO.rb:292:in `test_s_select' ./builtin/TestIO.rb:266:in `open' ./builtin/TestIO.rb:266:in `test_s_select' ....<[]> expected but was <[#]>. ------------------------------------------------------------ TestTime: --------- ./builtin/TestTime.rb:257:in `test_isdst' ./builtin/TestTime.rb:255:in `each' ./builtin/TestTime.rb:255:in `test_isdst' .... Expected 2000,10,29,1,59 to be dst=true. expected but was . ======================================================================== make: *** [test] Error 4 From holmberg at iar.se Fri Jan 7 08:13:41 2005 From: holmberg at iar.se (Johan Holmberg) Date: Fri Jan 7 08:11:48 2005 Subject: [Rubytests-devel] Testing on Apples OS-X? In-Reply-To: <0AF632D1-5F87-11D9-B291-000D9337C01C@chadfowler.com> References: <13169.216.77.69.67.1104943250.squirrel@216.77.69.67> <0AF632D1-5F87-11D9-B291-000D9337C01C@chadfowler.com> Message-ID: On Wed, 5 Jan 2005, Chad Fowler wrote: >>> >>> Is there anybody on the list with access to an OS-X machine that >>> could test Ruby 1.8.2 with the current Rubicon testcases? >>> [...] >> >> I'll try to do it tonight when I get home. I don't have my Mac at work. >> >> Chad >> Thanks. I think I have fixed two of these problems now. The other two requires more thought. See my comments below. > > TestDir: > -------- > ./builtin/TestDir.rb:172:in `test_s_chdir' > ./builtin/TestDir.rb:161:in `chdir' > ./builtin/TestDir.rb:161:in `test_s_chdir' > ./builtin/TestDir.rb:158:in `chdir' > ./builtin/TestDir.rb:158:in `test_s_chdir' > .... exception expected but was > Class: > Message: <"Permission denied - getcwd"> > ---Backtrace--- > ./builtin/TestDir.rb:173:in `chdir' > ./builtin/TestDir.rb:173:in `test_s_chdir' > ./builtin/TestDir.rb:172:in `assert_raise' > ./builtin/TestDir.rb:172:in `test_s_chdir' > ./builtin/TestDir.rb:161:in `chdir' > ./builtin/TestDir.rb:161:in `test_s_chdir' > ./builtin/TestDir.rb:158:in `chdir' > ./builtin/TestDir.rb:158:in `test_s_chdir' > --------------- > This is my fault. A while ago I introduced a too complicated regression test for a "misfeature" that existed aroung sep. 2004. I have now removed this test. > > TestFile__Stat: > --------------- > ./builtin/TestFile__Stat.rb:219:in `test_pipe?' > ./builtin/TestFile__Stat.rb:218:in `popen' > ./builtin/TestFile__Stat.rb:218:in `test_pipe?' > ./builtin/TestFile__Stat.rb:217:in `dont' > ./builtin/TestFile__Stat.rb:217:in `test_pipe?' > .... expected but was > . > This one I have no clue why it doesn't work. I think it can only be solved with someone with access to a MacOS machine. > TestIO: > ------- > ./builtin/TestIO.rb:292:in `test_s_select' > ./builtin/TestIO.rb:266:in `open' > ./builtin/TestIO.rb:266:in `test_s_select' > ....<[]> expected but was > <[#]>. > Solved now (I hope). It seems like select(2) on MacOS behaves the same way as on Solaris. > > TestTime: > --------- > ./builtin/TestTime.rb:257:in `test_isdst' > ./builtin/TestTime.rb:255:in `each' > ./builtin/TestTime.rb:255:in `test_isdst' > .... > Expected 2000,10,29,1,59 to be dst=true. > expected but was > . > This is not MacOS specific. The testcase depend on the timezone of the computer running the tests, and in Europe nothing is tested. By a simple setting like $ setenv TZ EST I could get this test to fail even on Solaris in Sweden ;-) I have commited changes for the two cases I knew how to fix. /Johan