[Win32utils-devel] IO.read questions

Daniel Berger djberg96 at gmail.com
Thu May 3 15:40:13 UTC 2012


On Thu, May 3, 2012 at 8:48 AM, Daniel Berger <djberg96 at gmail.com> wrote:
> On Thu, May 3, 2012 at 5:58 AM, Luis Lavena <luislavena at gmail.com> wrote:
>> On Thu, May 3, 2012 at 2:25 AM, Daniel Berger <djberg96 at gmail.com> wrote:
>>>
>>> Oh, I plan on it. In fact I was playing with the ffi branch of the
>>> win32-nio library again today. The NIO.read benchmarks are pretty
>>> good. Much faster than the default IO.read method.
>>>
>>
>> Please note that NIO.read might not be doing all the legwork IO.read
>> is already doing in relation to encodings.
>>
>> Which, along with others, I think is overkill.
>
> True, I thought that option was a bit silly, since users can always
> encode after the fact if they want.
>
> Performance is being hampered somewhat by having to do newline
> conversions, but the results are good:
>
> c:\Users\djberge\Repositories\win32-nio>rake bench
> ruby -Ilib benchmarks/win32_nio_benchmarks.rb
> Small file created (588k)
> Medium file created (6mb)
> Large file created (58mb)
> Huge file created (618mb)
>
>                           user     system      total        real
> IO.read(small)         0.016000   0.016000   0.032000 (  0.027001)
> IO.read(medium)        0.234000   0.062000   0.296000 (  0.295017)
> IO.read(large)         2.184000   0.484000   2.668000 (  2.687154)
> IO.read(huge)         22.245000   5.007000  27.252000 ( 27.283561)
>
> NIO.read(small)        0.016000   0.000000   0.016000 (  0.017001)
> NIO.read(medium)       0.172000   0.188000   0.360000 (  0.617035)
> NIO.read(large)        1.263000   1.014000   2.277000 (  4.820276)
> NIO.read(huge)        13.822000   7.909000  21.731000 ( 32.729872)
>
> Minor improvements for smaller files, but the savings gets better as
> the files get bigger. It's a shame there's no text mode for
> CreateFile.
>
> BTW, using FILE_FLAG_NO_BUFFERING with a sector aligned buffer didn't
> yield any noticeable improvements that I could see.

If I ignore the line ending transformation, things get *much* better:

c:\Users\djberge\Repositories\win32-nio>rake bench
ruby -Ilib benchmarks/win32_nio_benchmarks.rb
Small file created (588k)
Medium file created (6mb)
Large file created (58mb)
Huge file created (618mb)

                           user     system      total        real
IO.read(small)         0.032000   0.016000   0.048000 (  0.034001)
IO.read(medium)        0.202000   0.109000   0.311000 (  0.318018)
IO.read(large)         2.200000   0.484000   2.684000 (  2.682153)
IO.read(huge)         22.449000   4.914000  27.363000 ( 27.389567)

NIO.read(small)        0.000000   0.000000   0.000000 (  0.005000)
NIO.read(medium)       0.031000   0.046000   0.077000 (  0.072004)
NIO.read(large)        0.156000   0.484000   0.640000 (  0.639037)
NIO.read(huge)         1.919000   5.164000   7.083000 (  8.560490)

Though to be fair, things are much better if you pass the binary flag
with plain Ruby. In fact, the difference is negligible until you get
into large files.

                           user     system      total        real
IO.read(small)            0.016000   0.016000   0.032000 (  0.032002)
IO.read(small) with rb    0.015000   0.000000   0.015000 (  0.010001)

IO.read(medium)           0.250000   0.078000   0.328000 (  0.321018)
IO.read(medium) with rb   0.015000   0.062000   0.077000 (  0.084005)

IO.read(large)            2.216000   0.468000   2.684000 (  2.679153)
IO.read(large) with rb    0.171000   0.499000   0.670000 (  0.671039)

IO.read(huge)             22.324000   4.945000  27.269000 ( 27.425568)
IO.read(huge)             3.276000   8.986000  12.262000 ( 30.654754)

Regards,

Dan


More information about the win32utils-devel mailing list