I FIXED THE BUG. How?
Okay, short story time:
As I mentioned last time, Renegade had a problem. And it was driving me insane.
I had just solved my
first issue involving lack of precision with floating point numbers
using the Big Decimal method .to_d. Things were finally running
smoothly….or so I thought.
After using it for a
week or so, I discovered that one particular calculation would
sometimes be $0.01 off. For example, if given $530.04 - $500.02, the
system would output $30.01 instead of $30.02.
It only occurred
about 10% of the time. I couldn't even figure out the exact
conditions under which it happened. Regardless, Renegade deals with
money. It can't afford to be off, even by $0.01.
Exasperated,
I turned to my good old friend Google (and
the lovely, brave
folks at Stack Overflow) to
figure
out what to do. I ran in circles for a while before I realized
what had happened.
There
are two ways to invoke Big Decimal – one way is to use the .to_d
method. The other way is to create a new Big Decimal class using
BigDecimal.new().
There's
a big difference between the two options. When
you use BigDecimal.new(), you
put a number in the Big Decimal class before
you manipulate it.
This
means that the number is
precise from the very
beginning. It's like
using waterproof materials to make a boot.
You
invoke the .to_d method after
the number has already been represented as a floating point number –
which, as explained here,
is not very precise. It's like spraying a regular boot
with waterproofer. This is what I had been doing.
If
you know the difference between a waterproof boot and a boot that's
been sprayed with waterproofer, you'll see where my problem was.
When
I realized this (and partly because I didn't know what else to do), I
put all of my numbers under the Big Decimal class. New inputs would
have the precision of Big Decimal from the beginning.
It
changed everything. Suddenly Renegade was so
precise that it gave out more accurate results than I got when I
was
using
a manual
10 key
calculator. It
was beautiful. I almost cried.
Moral
of the story: Use
the
.to_d
method
with
caution. If
you're dealing with currency, BigDecimal.new() may be a better option
for you.
Have
you ever had an unexpected issue with the Big Decimal method or
class?
* You can find the source code for Renegade (which is written in Ruby), as well as the
commits that inspired this post, here on Github. Try not to laugh too hard at the GUI, or lack thereof. *
