Bugs: Browse | Submit New | Admin

[#19090] Floating point .to_i errors

Date:
2008-03-25 03:15
Priority:
3
Submitted By:
Benjamin Black (bblack)
Assigned To:
Akinori MUSHA (knu)
Category:
Math
State:
Closed
Platform:
 
Summary:
Floating point .to_i errors

Detailed description
irb(main):001:0> (9.54 / 0.01)
=> 954.0
irb(main):002:0> (9.54 / 0.01).to_i
=> 953
irb(main):003:0> (9.54 / 0.01).to_s.to_i
=> 954
irb(main):004:0> (9.54 / 0.1)
=> 95.4
irb(main):005:0> (9.54 / 0.1).to_i
=> 95
irb(main):006:0> (9.54 / 0.001).to_i
=> 9539
irb(main):007:0> (9.54 / 0.001).to_s.to_i
=> 9540
irb(main):008:0> 

Yes, I read the previous bug report opened regarding this problem.  The link given to the Sun page explaining floating
point math is simply not enough.  See how .to_i gives the wrong value, but .to_s.to_i is correct?  This is an actual
bug.

Add A Comment: Notepad

Please login


Followup

Message
Date: 2008-04-01 23:38
Sender: Nobuyoshi Nakada 

I don't consider changing the default behavior is nice, otherwise
truncation of a positive number would produce a larger number.

I've thought about extending #truncate, #ceil, and #floor like
as #round, instead.
Date: 2008-03-30 01:05
Sender: Benjamin Black

I'm going to try once more.  Yes, you use rounding, but ultimately
the goal is
truncation and, as your own code shows, that can take a little
work.  The
existence of common techniques for fixing up the value, just
as you showed,
means there is the opportunity for Ruby to do the more sophisticated
and
less confusing thing _by default_.  That's what I keep asking
for and you keep
misunderstanding that.

As you don't intend to actually do anything about the issue,
this does not
matter.
Date: 2008-03-30 00:44
Sender: Nobuyoshi Nakada 

It works exactly and consistently.  The point you ignore or
don't understand is that decimal 9.54 and 0.001 can't be
expressed in binary representation.

Also your C code showed you needed rounding.  float has less
digits than double, so convertion from double to float is to
round actually.
Date: 2008-03-29 12:49
Sender: Benjamin Black

Responding to your points in order:

1) My issue was not with exactness, but with consistency.  Given
that
#truncate is need not be that fast, there is little reason not
to make it return
the "expected" value.

2) No, I have already explained, and you have shown in your code
example,
that rounding can be used to construct the solution, but is not
the solution
on its own.  I really do want truncation, but I want truncation
that works
consistently.

3) I agree.  See #1.
Date: 2008-03-29 09:12
Sender: Nobuyoshi Nakada 

I haven't said that I'd considered your case "unique".
It's a rather common issue caused by a misapprehension.

Points are:
* on currently major computer systems, floating point numbers
are not always mathematically exact,

* what you want is rounding, but not truncation, and

* BigDecimal is useful if decimal representational exactness
is the most important issue.
Date: 2008-03-26 11:52
Sender: Benjamin Black

My "case" is getting consistent results from Float#truncate.
It's strange and
disappointing that you consider that unique to me.
Date: 2008-03-26 03:31
Sender: Nobuyoshi Nakada 

It isn't a bug fix but is an extension, which is just useful
for your case.
Date: 2008-03-26 02:18
Sender: Benjamin Black

Also, your truncate_rounding code can overflow internally.  That's
not a real fix,
either.
Date: 2008-03-26 02:12
Sender: Benjamin Black

If it wasn't a bug you wouldn't need that change in 1.9.  So,
you agree it is a bug
in 1.8 and it has been fixed in 1.9.  Great, thanks.
Date: 2008-03-26 02:04
Sender: Nobuyoshi Nakada 

Ah, you use 1.9, so Float#round accepts an optional argument.

  f = (9.54 / 0.001)
  p f.round(11).truncate # => 9540
  p f.round(12).truncate # => 9539
Date: 2008-03-26 01:59
Sender: Nobuyoshi Nakada 

Since Ruby doesn't use float, it's irrelevant.

  class Float
    def truncate_rounding(figs)
      if figs > 0
        n = 10 ** figs
        (self * n).round / n
      else
        n = 10 ** -figs
        (self / n).round / 10 * (n * 10)
      end
    end
  end

  f = (9.54 / 0.001)
  p f.truncate_rounding(11) # => 9540
  p f.truncate_rounding(12) # => 9539


Date: 2008-03-26 00:18
Sender: Benjamin Black

Digging in more, the central issue here is that Ruby converts
between float
and double at will within the Float class, so the behavior is
not consistent
depending on which method is used.  For example, to_i uses the
double
representation, but to_s seems to be working with a float (I
haven't yet dug
through the nest of C #defines to figure out if that is
the case).

If that is so, the results are inconsistent not because of IEEE
floating point
specs, but because of how Ruby uses the C types.  If you want
to claim that is
not a bug, then we can agree to disagree.




Date: 2008-03-25 22:57
Sender: Benjamin Black

Yes, _in this specific example_ #round produces what #truncate
should, but that
might not always be the case.  How does someone know in advance
that they
should use #round instead of #truncate?
Date: 2008-03-25 22:45
Sender: Nobuyoshi Nakada 

Actually, what you want is rounding.
It's less than 9540, so why it becomes 9540 by truncating?
Date: 2008-03-25 21:54
Sender: Benjamin Black

Float#round is not truncation so I have no idea why you are suggesting
its use.
The only way to reliably get correct truncation behavior from
Float is to use
Float#to_s followed by String#to_i.  That is broken and it is
a shame you refuse
to fix it.
Date: 2008-03-25 21:35
Sender: Nobuyoshi Nakada 

Use Float#round instead, or BigDecimal.

Your C code is wrong.  It's impossible to pass a float to printf(),
it will be always promoted to a double silently.
By using double instead of float, it shows the same behavior.
Date: 2008-03-25 11:50
Sender: Benjamin Black

Test program in C showing correct behavior:

#include <stdio.h>

int
main()
{
	float f = 9.54 / 0.001;

	printf("%f %i\n", f, (int)f);
	
	return 0;
}

$ ./test
9540.000000 9540
$

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

Field Old Value Date By
close_date2008-04-01 23:382008-04-01 23:38nobu
close_date2008-03-30 01:052008-03-30 01:05bblack
close_date2008-03-30 00:442008-03-30 00:44nobu
close_date2008-03-29 12:492008-03-29 12:49bblack
close_date2008-03-29 09:122008-03-29 09:12nobu
close_date2008-03-26 11:522008-03-26 11:52bblack
close_date2008-03-26 03:312008-03-26 03:31nobu
close_date2008-03-26 02:182008-03-26 02:18bblack
close_date2008-03-26 02:122008-03-26 02:12bblack
close_date2008-03-26 02:042008-03-26 02:04nobu
close_date2008-03-26 01:592008-03-26 01:59nobu
close_date2008-03-26 00:182008-03-26 00:18bblack
close_date2008-03-25 22:572008-03-25 22:57bblack
close_date2008-03-25 22:452008-03-25 22:45nobu
close_date2008-03-25 21:542008-03-25 21:54bblack
resolution_idNone2008-03-25 21:35nobu
close_date2008-03-25 21:352008-03-25 21:35nobu
status_idOpen2008-03-25 21:35nobu