Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unexpected interaction between binary multiplication operator and Test::More::cmp_ok() #14718

Closed
p5pRT opened this issue May 28, 2015 · 9 comments
Closed

Unexpected interaction between binary multiplication operator and Test::More::cmp_ok() #14718

p5pRT opened this issue May 28, 2015 · 9 comments
Labels
Not a Bug type-numerical

Comments

@p5pRT
Copy link

p5pRT commented May 28, 2015

Migrated from rt.perl.org#125266 (status was 'rejected')

Searchable as RT125266$
@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From @jkeenan

Today I was bitten by an unexpected interaction between Perl's binary multiplication operator (*) and Test​::More​::cmp_ok(). I got expected results only on very old versions of Perl and Test​::More. Though I was able to compensate for this problem on newer versions of Perl and Test​::More, the fact that I got unexpected results suggests either the presence of a bug or the need for some cautionary documentation somewhere.

The situation​: I needed to test whether I was getting expected values of prices stored in US dollars (USD) but converted to Japanese yen (JPY). For testing purposes, I assumed that the JPY/USD exchange rate was 8​:1.

Consider the program attached, 'cmp_ok.pl', which illustrates four different usages of Test​::More​::cmp_ok(). Cet. par., I would expect all four to PASS and, when run with perl-5.6.2 and the version of Test​::More that came with that perl, 0.47, the do.

#####
$ perlbrew switch perl-5.6.2
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.006002
ok 2 - Test​::More version​: 0.47
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
#####

However, when I switch to perl-5.8.9 and the version of Test​::More that came with that perl, 0.80, the third test FAILs. In the third test, I calculate the expected JPY price by multiplying the USD price by the JPY/USD exchange rate -- multiplication of two floats -- and assign the result to $calculated_jpy, which I then use in the next instance of 'cmp_ok'. The output of 'cmp_ok' suggests that the two values are not mathematically equal even though the 'got' and 'expected' outputs are identical!

$ perlbrew switch perl-5.8.9
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.008009
ok 2 - Test​::More version​: 0.8
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
not ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
# Failed test 'One float explicitly assigned, one calculated via '*'; compare with '==''
# at cmp_ok.pl line 20.
# got​: 1.68
# expected​: 1.68
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
# Looks like you failed 1 test of 7.
#####

Testing on more recent perl/Test​::More combinations, I get the same results as I did on 5.8.9/0.80. Example​:

#####
$ perlbrew switch perl-5.20.1
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.020001
ok 2 - Test​::More version​: 1.001002
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
not ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
# Failed test 'One float explicitly assigned, one calculated via '*'; compare with '==''
# at cmp_ok.pl line 20.
# got​: 1.68
# expected​: 1.68
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
# Looks like you failed 1 test of 7.
#####

Note that when I sprintf-ed the product of the multiplication and used the result in 'cmp_ok', I got the expected PASS (fourth instance in each perl version).

I haven't yet had time to figure out whether the change in behavior between 5.6.2/0.47 and 5.8.9/0.80 occurred in perl or in Test​::More. But I suspect that this surprising behavior needs to be documented somewhere.

Recommendations?

Thank you very much.
Jim Keenan
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From @jkeenan

Today I was bitten by an unexpected interaction between Perl's binary multiplication operator (*) and Test​::More​::cmp_ok(). I got expected results only on very old versions of Perl and Test​::More. Though I was able to compensate for this problem on newer versions of Perl and Test​::More, the fact that I got unexpected results suggests either the presence of a bug or the need for some cautionary documentation somewhere.

The situation​: I needed to test whether I was getting expected values of prices stored in US dollars (USD) but converted to Japanese yen (JPY). For testing purposes, I assumed that the JPY/USD exchange rate was 8​:1.

Consider the program attached, 'cmp_ok.pl', which illustrates four different usages of Test​::More​::cmp_ok(). Cet. par., I would expect all four to PASS and, when run with perl-5.6.2 and the version of Test​::More that came with that perl, 0.47, the do.

#####
$ perlbrew switch perl-5.6.2
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.006002
ok 2 - Test​::More version​: 0.47
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
#####

However, when I switch to perl-5.8.9 and the version of Test​::More that came with that perl, 0.80, the third test FAILs. In the third test, I calculate the expected JPY price by multiplying the USD price by the JPY/USD exchange rate -- multiplication of two floats -- and assign the result to $calculated_jpy, which I then use in the next instance of 'cmp_ok'. The output of 'cmp_ok' suggests that the two values are not mathematically equal even though the 'got' and 'expected' outputs are identical!

$ perlbrew switch perl-5.8.9
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.008009
ok 2 - Test​::More version​: 0.8
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
not ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
# Failed test 'One float explicitly assigned, one calculated via '*'; compare with '==''
# at cmp_ok.pl line 20.
# got​: 1.68
# expected​: 1.68
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
# Looks like you failed 1 test of 7.
#####

Testing on more recent perl/Test​::More combinations, I get the same results as I did on 5.8.9/0.80. Example​:

#####
$ perlbrew switch perl-5.20.1
$ perl cmp_ok.pl
ok 1 - Perl version​: 5.020001
ok 2 - Test​::More version​: 1.001002
ok 3 - Use Test​::More​::cmp_ok() to compare two floats
ok 4 - Both floats explicitly assigned; compare with '=='
ok 5 - Both floats explicitly assigned; compare with 'eq'
not ok 6 - One float explicitly assigned, one calculated via '*'; compare with '=='
# Failed test 'One float explicitly assigned, one calculated via '*'; compare with '==''
# at cmp_ok.pl line 20.
# got​: 1.68
# expected​: 1.68
ok 7 - One floatexplicitly assigned, one calculated via '*' then sprintf-ed; compare with '=='
1..7
# Looks like you failed 1 test of 7.
#####

Note that when I sprintf-ed the product of the multiplication and used the result in 'cmp_ok', I got the expected PASS (fourth instance in each perl version).

I haven't yet had time to figure out whether the change in behavior between 5.6.2/0.47 and 5.8.9/0.80 occurred in perl or in Test​::More. But I suspect that this surprising behavior needs to be documented somewhere.

Recommendations?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From @jkeenan

Summary of my perl5 (revision 5 version 20 subversion 1) configuration​:
 
  Platform​:
  osname=linux, osvers=3.13.0-35-generic, archname=x86_64-linux
  uname='linux zareason 3.13.0-35-generic #62-ubuntu smp fri aug 15 01​:58​:42 utc 2014 x86_64 x86_64 x86_64 gnulinux '
  config_args='-de -Dprefix=/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1 -Aeval​:scriptdir=/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/bin'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.8.2', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
  USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
  USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  Built under linux
  Compiled at Sep 16 2014 18​:46​:21
  %ENV​:
  PERL5LIB="/home/jkeenan/perl5/lib/perl5"
  PERLBREW_BASHRC_VERSION="0.67"
  PERLBREW_HOME="/home/jkeenan/.perlbrew"
  PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/man"
  PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/bin"
  PERLBREW_PERL="perl-5.20.1"
  PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
  PERLBREW_VERSION="0.67"
  PERL_LOCAL_LIB_ROOT="/home/jkeenan/perl5"
  PERL_MB_OPT="--install_base "/home/jkeenan/perl5""
  PERL_MM_OPT="INSTALL_BASE=/home/jkeenan/perl5"
  PERL_WORKDIR="gitwork/perl"
  @​INC​:
  /home/jkeenan/perl5/lib/perl5/x86_64-linux
  /home/jkeenan/perl5/lib/perl5
  /home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/lib/site_perl/5.20.1/x86_64-linux
  /home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/lib/site_perl/5.20.1
  /home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/lib/5.20.1/x86_64-linux
  /home/jkeenan/perl5/perlbrew/perls/perl-5.20.1/lib/5.20.1
  .

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From @jkeenan

Attached the wrong file. Now attaching the 'cmp_ok.pl' program.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From @jkeenan

cmp_ok.pl

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From [Unknown Contact. See original ticket]

Attached the wrong file. Now attaching the 'cmp_ok.pl' program.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

From zefram@fysh.org

James E Keenan wrote​:

Today I was bitten by an unexpected interaction between Perl's binary
multiplication operator (*) and Test​::More​::cmp_ok().

It's nothing to do with cmp_ok, and little to do with *; it's just
floating point rounding, specifically on conversion from decimal to
floating point. 0.21 does not have a terminating binary representation,
nor does 1.68, so in general you can't expect to get exact results.
It's not in general surprising that floating point computations produce
such mathematical discrepancies. The only strange bit is that the result
changed between Perl versions​:

$ perl5.8.0 -lwe 'printf "%.70f\n" x 2, 1.68, 8 * 0.21'
1.6799999999999999378275106209912337362766265869140625000000000000000000
1.6799999999999999378275106209912337362766265869140625000000000000000000
$ perl5.8.1 -lwe 'printf "%.70f\n" x 2, 1.68, 8 * 0.21'
1.6800000000000001598721155460225418210029602050781250000000000000000000
1.6799999999999999378275106209912337362766265869140625000000000000000000

Both versions agree on the floating point conversion of 0.21, and indeed
have the NV whose value is closest to 0.21. Both correctly multiply that
converted value by 8, which is an exact computation in floating point.
Where they disagree is on the direct floating point conversion of 1.68.
As you can see, 5.8.0 and below have picked the NV closest to 1.68,
thus agreeing with the conversion of 0.21. But 5.8.1 and above have
picked the NV the other side of 1.68, which is less close and disagrees
with the conversion of 0.21.

We already know the decimal->float conversion is broken; see [perl
#41202]. Interestingly, that ticket's test case also shows a change
from correctly-rounded conversion to incorrect between 5.8.0 and 5.8.1​:

$ perl5.8.0 perl -lwe 'printf "%f\n", 1180591620717411303424.0'
1180591620717411303424.000000
$ perl5.8.1 perl -lwe 'printf "%f\n", 1180591620717411303424.0'
1180591620717411172352.000000

So I suspect that the dodgy conversion of 1.68 is merely another example
of [perl #41202].

-zefram

@p5pRT
Copy link
Author

p5pRT commented May 28, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT p5pRT closed this as completed Sep 24, 2015
@p5pRT
Copy link
Author

p5pRT commented Sep 24, 2015

@iabyn - Status changed from 'open' to 'rejected'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Not a Bug type-numerical
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@p5pRT