Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BBC: MaxMind-DB-Writer-0.300003 fails since perl 5.31.2 #17245

Closed
eserte opened this issue Nov 3, 2019 · 9 comments
Closed

BBC: MaxMind-DB-Writer-0.300003 fails since perl 5.31.2 #17245

eserte opened this issue Nov 3, 2019 · 9 comments
Labels
BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) Needs Triage
Milestone

Comments

@eserte
Copy link
Contributor

eserte commented Nov 3, 2019

Description
Since perl 5.31.2 all tests in the MaxMind-DB-Writer-0.300003 test suite segfault (@maxmind: FYI).
Overview of test reports: http://fast-matrix.cpantesters.org/?dist=MaxMind-DB-Writer%200.300003

A gdb backtrace for perl5.31.5 -Mblib t/MaxMind/DB/Writer/Serializer-deduplication.t (sorry, perl was not built with -g):

Program received signal SIGSEGV, Segmentation fault.
0x0000000000425e6d in Perl_op_linklist ()
(gdb) bt
#0  0x0000000000425e6d in Perl_op_linklist ()
#1  0x000000000042dbfa in Perl_newCONDOP ()
#2  0x000000000046e835 in Perl_yyparse ()
#3  0x0000000000500fac in S_doeval_compile ()
#4  0x0000000000502495 in Perl_pp_require ()
#5  0x00000000004c3753 in Perl_runops_standard ()
#6  0x00000000004405a8 in Perl_call_sv ()
#7  0x00000000004429c2 in Perl_call_list ()
#8  0x0000000000424cca in S_process_special_blocks.isra.10 ()
#9  0x000000000043a39b in Perl_newATTRSUB_x ()
#10 0x000000000043d3d9 in Perl_utilize ()
#11 0x000000000046fe2d in Perl_yyparse ()
#12 0x0000000000500fac in S_doeval_compile ()
#13 0x0000000000502495 in Perl_pp_require ()
#14 0x00000000004c3753 in Perl_runops_standard ()
#15 0x00000000004405a8 in Perl_call_sv ()
#16 0x00000000004429c2 in Perl_call_list ()
#17 0x0000000000424cca in S_process_special_blocks.isra.10 ()
#18 0x000000000043a39b in Perl_newATTRSUB_x ()
#19 0x000000000043d3d9 in Perl_utilize ()
#20 0x000000000046fe2d in Perl_yyparse ()
#21 0x0000000000445de0 in perl_parse ()
#22 0x0000000000421f00 in main ()
@eserte eserte added BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) Needs Triage affects-5.31 labels Nov 3, 2019
@dur-randir
Copy link
Member

Bisect points to

commit 7d3bb7a
Author: David Mitchell davem@iabyn.com
Date: Wed May 29 15:57:06 2019 +0100

make Perl_op_linklist() non-recursive

@tonycoz
Copy link
Contributor

tonycoz commented Nov 4, 2019

I suspect this is a duplicate of #17114 (MaxMind::DB::Writer depends on Sereal::Decoder) which is an op parent issue, there's a PR against Sereal at Sereal/Sereal#210

@tonycoz
Copy link
Contributor

tonycoz commented Nov 4, 2019

Applying the Sereal::Decoder fix didn't fix the above. All of the tests assert in a -DDEBUGGING build:

perl5.31.6: op.c:1686: Perl_op_linklist: Assertion `!o->op_next' failed.

Backtrace:

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff6cf642a in __GI_abort () at abort.c:89
#2  0x00007ffff6cede67 in __assert_fail_base (fmt=<optimized out>, 
    assertion=assertion@entry=0x555555930be0 "!o->op_next", 
    file=file@entry=0x55555592e411 "op.c", line=line@entry=1686, 
    function=function@entry=0x555555937230 <__PRETTY_FUNCTION__.15991> "Perl_op_linklist") at assert.c:92
#3  0x00007ffff6cedf12 in __GI___assert_fail (
    assertion=0x555555930be0 "!o->op_next", file=0x55555592e411 "op.c", 
    line=1686, 
    function=0x555555937230 <__PRETTY_FUNCTION__.15991> "Perl_op_linklist")
    at assert.c:101
#4  0x00005555555a6699 in Perl_op_linklist (o=0x5555564f45e0) at op.c:1686
#5  0x00005555555be07f in Perl_newCONDOP (flags=0, first=0x5555564a74e8, 
    trueop=0x5555564f4450, falseop=0x5555564f42f0) at op.c:8860
#6  0x000055555566f32f in Perl_yyparse (gramtype=258) at perly.y:1116
#7  0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, 
    seq=6648, hh=0x0) at pp_ctl.c:3543
#8  0x000055555584a1ed in S_require_file (sv=0x555556271c88) at pp_ctl.c:4363
#9  0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387
#10 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571
#11 0x00005555555ee589 in Perl_call_sv (sv=0x5555563bb278, flags=13)
    at perl.c:3041
#12 0x00005555555f825f in Perl_call_list (oldscope=10, 
    paramList=0x555556271898) at perl.c:5082
#13 0x00005555555cc6f7 in S_process_special_blocks (floor=269, 
    fullname=0x555555e0dea0 "BEGIN", gv=0x555556271a78, cv=0x5555563bb278)
    at op.c:10815
#14 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=269, o=0x5555562ce7d8, 
    proto=0x0, attrs=0x0, block=0x5555562ce7a0, o_is_gv=false) at op.c:10740
#15 0x00005555555ba9ef in Perl_utilize (aver=1, floor=269, version=0x0, 
    idop=0x55555636efd8, arg=0x555556276590) at op.c:7907
#16 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346
#17 0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, 
    seq=5286, hh=0x0) at pp_ctl.c:3543
#18 0x000055555584a1ed in S_require_file (sv=0x555555e11380) at pp_ctl.c:4363
#19 0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387
#20 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571
#21 0x00005555555ee589 in Perl_call_sv (sv=0x555555e108d0, flags=13)
    at perl.c:3041
#22 0x00005555555f825f in Perl_call_list (oldscope=6, paramList=0x555555e10fa8)
    at perl.c:5082
#23 0x00005555555cc6f7 in S_process_special_blocks (floor=157, 
    fullname=0x555555e0dea0 "BEGIN", gv=0x555555e10bd0, cv=0x555555e108d0)
    at op.c:10815
#24 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=157, o=0x55555627cb70, 
    proto=0x0, attrs=0x0, block=0x55555627cb38, o_is_gv=false) at op.c:10740
#25 0x00005555555ba9ef in Perl_utilize (aver=1, floor=157, version=0x0, 
    idop=0x555555e17920, arg=0x0) at op.c:7907
#26 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346
#27 0x00005555558436b4 in S_doeval_compile (gimme=2 '\002', outside=0x0, 
    seq=4113, hh=0x0) at pp_ctl.c:3543
#28 0x000055555584a1ed in S_require_file (sv=0x555555e10918) at pp_ctl.c:4363
#29 0x000055555584a31b in Perl_pp_require () at pp_ctl.c:4387
#30 0x0000555555713aa4 in Perl_runops_debug () at dump.c:2571
#31 0x00005555555ee589 in Perl_call_sv (sv=0x555555e11590, flags=13)
    at perl.c:3041
#32 0x00005555555f825f in Perl_call_list (oldscope=2, paramList=0x555555e037d8)
    at perl.c:5082
#33 0x00005555555cc6f7 in S_process_special_blocks (floor=45, 
    fullname=0x555555e0dea0 "BEGIN", gv=0x555555e037f0, cv=0x555555e11590)
    at op.c:10815
#34 0x00005555555cbe45 in Perl_newATTRSUB_x (floor=45, o=0x555555e0dcc8, 
    proto=0x0, attrs=0x0, block=0x555555e0dc90, o_is_gv=false) at op.c:10740
#35 0x00005555555ba9ef in Perl_utilize (aver=1, floor=45, version=0x0, 
    idop=0x555555e0d6f0, arg=0x555555e60f50) at op.c:7907
#36 0x000055555566c844 in Perl_yyparse (gramtype=258) at perly.y:346
#37 0x00005555555ebf77 in S_parse_body (env=0x0, 
    xsinit=0x5555555a0377 <xs_init>) at perl.c:2526
#38 0x00005555555ea232 in perl_parse (my_perl=0x555555dd8010, 
    xsinit=0x5555555a0377 <xs_init>, argc=3, argv=0x7fffffffb6e8, env=0x0)
    at perl.c:1817
#39 0x00005555555a02ba in main (argc=3, argv=0x7fffffffb6e8, 
    env=0x7fffffffb708) at perlmain.c:126

Maybe Sereal::Encoder?

(gdb) up
#5  0x00005555555be07f in Perl_newCONDOP (flags=0, first=0x5555564a74e8, 
    trueop=0x5555564f4450, falseop=0x5555564f42f0) at op.c:8860
8860        logop = alloc_LOGOP(OP_COND_EXPR, first, LINKLIST(trueop));
(gdb) l
8855                /* Mark the op as being unbindable with =~ */
8856                live->op_flags |= OPf_SPECIAL;
8857            live->op_folded = 1;
8858            return live;
8859        }
8860        logop = alloc_LOGOP(OP_COND_EXPR, first, LINKLIST(trueop));
8861        logop->op_flags |= (U8)flags;
8862        logop->op_private = (U8)(1 | (flags >> 8));
8863        logop->op_next = LINKLIST(falseop);
8864
(gdb) call Perl_op_dump(trueop)
     
8    sereal_encode_with_object UNOP(0x5555564f4450) ===> [0x0]
     PARENT ===> [0x0]
     FLAGS = (UNKNOWN,KIDS,SLABBED)
     |   
9    +--padsv OP(0x5555564a7478) ===> [SELF]
     |   TARG = 3
     |   FLAGS = (SCALAR,MOD,SLABBED,MORESIB)
     |   
10   +--aelem BINOP(0x5555564f44f8) ===> 11 [gv 0x5555564f45a8]
         *** WILD PARENT 0x5555564f45e0
         FLAGS = (SCALAR,KIDS,MOD,SLABBED)
         PRIVATE = (LVDEFER,0x2)
         |   
12       +--rv2av UNOP(0x5555564f4570) ===> 13 [const 0x5555564f4538]
         |   FLAGS = (SCALAR,KIDS,REF,SLABBED,MORESIB)
         |   PRIVATE = (STRICT,0x1)
         |   |   
11       |   +--gv SVOP(0x5555564f45a8) ===> 12 [rv2av 0x5555564f4570]
         |       FLAGS = (SCALAR,SLABBED)
         |       GV = main::_ (0x555555dda460)
         |   
13       +--const SVOP(0x5555564f4538) ===> 10 [aelem 0x5555564f44f8]
             FLAGS = (SCALAR,SLABBED)
             SV = IV(0)

A similar monkey-fix to Sereal::Encoder didn't prevent the assertion failure in MaxMind::DB::Writer

@eserte
Copy link
Contributor Author

eserte commented Feb 3, 2020

Data-Serializer-Sereal-1.05 also fails since perl 5.31.2: http://fast-matrix.cpantesters.org/?dist=Data-Serializer-Sereal%201.05
The backtrace also stops at Perl_op_linklist, so it might be the same issue:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055da7a4b9e97 in Perl_op_linklist ()
(gdb) bt
#0  0x000055da7a4b9e97 in Perl_op_linklist ()
#1  0x000055da7a4cf6a8 in Perl_newATTRSUB_x ()
#2  0x000055da7a509e6b in Perl_yyparse ()
#3  0x000055da7a59eac1 in S_doeval_compile ()
#4  0x000055da7a59fe94 in Perl_pp_require ()
#5  0x000055da7a560893 in Perl_runops_standard ()
#6  0x000055da7a4d6107 in Perl_call_sv ()
#7  0x000055da7a4d87de in Perl_call_list ()
#8  0x000055da7a4b7cbc in S_process_special_blocks.isra.13 ()
#9  0x000055da7a4cf437 in Perl_newATTRSUB_x ()
#10 0x000055da7a4d2d61 in Perl_utilize ()
#11 0x000055da7a509815 in Perl_yyparse ()
#12 0x000055da7a59eac1 in S_doeval_compile ()
#13 0x000055da7a5a9921 in Perl_pp_entereval ()
#14 0x000055da7a560893 in Perl_runops_standard ()
#15 0x000055da7a4d6107 in Perl_call_sv ()
#16 0x000055da7a4d87de in Perl_call_list ()
#17 0x000055da7a4b7cbc in S_process_special_blocks.isra.13 ()
#18 0x000055da7a4cf437 in Perl_newATTRSUB_x ()
#19 0x000055da7a509e6b in Perl_yyparse ()
#20 0x000055da7a4dc7e9 in perl_parse ()
#21 0x000055da7a4b61f6 in main ()

@demerphq
Copy link
Collaborator

demerphq commented Feb 4, 2020 via email

demerphq added a commit to Sereal/Sereal that referenced this issue Feb 4, 2020
Also some whitespace tweaks to Decoder.xs so that the two are the same
for this piece of logic.

In 041e46b I ported changed made
by Dave M to the Decoder into the Encoder. Unfortunately in the process
of making the ported patch backwards compatible with Perl's lacking
op_sibling_splice() I mistakenly put one statement on the wrong side
of a #if/#endif, which meant it did not work correctly (even though
it compiled fine.)

To add insult to injury we weren't testing the custom opcode
implementation properly, with this result this went unnoticed until
BBC (Blead Breaks CPAN) tests showed that MaxMind-DB-Writer-0.300003
was broken and also revealed that Data-Serializer-Sereal-1.05 was
also broken, both pointing at issues with Sereal::Encoder via
sereal_encoder_with_object which is implemented as a custom opcode.

This patch fixes all of this, and adds tests 190_customop.t to
both packages to ensure that this is tested properly moving forward.

Thanks to Dave M, Slaven Rezić, Tony Cook, Sergey Aleynikov for
playing a role in getting this identified and fixed.

This patch should mean that Perl/perl5#17245
can be closed.
@demerphq
Copy link
Collaborator

demerphq commented Feb 4, 2020 via email

@demerphq
Copy link
Collaborator

demerphq commented Feb 4, 2020 via email

@demerphq
Copy link
Collaborator

demerphq commented Feb 4, 2020 via email

@demerphq
Copy link
Collaborator

demerphq commented Feb 4, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) Needs Triage
Projects
None yet
Development

No branches or pull requests

5 participants