Skip to content

Conversation

@wilzbach
Copy link
Contributor

To investigate the repeated, spurious failures on CircleCi.
This should allow us to get a better stack trace.
It also tweaks the gdb command.

To investigate the repeated, spurious failures on CircleCi.
This should allow us to get a better stack trace.
It also tweaks the gdb command.
@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

@PetarKirov
Copy link
Member

PetarKirov commented Dec 15, 2017

The make files of druntime and phobos expect that dmd is always built in release mode, so it would be easier to not add BUILD="debug" there. I also considered changing the DMD variable from
DMD = $(DMD_DIR)/generated/$(OS)/release/$(MODEL)/dmd to
DMD = $(DMD_DIR)/generated/$(OS)/$(BUILD)/$(MODEL)/dmd, but now I think that it's a reasonable default.

@wilzbach
Copy link
Contributor Author

The make files of druntime and phobos expect that dmd is always built in release mode, so it would be easier to not add BUILD="debug" there.

Okay that was unexpected :/

I also considered changing the DMD variable from
DMD = $(DMD_DIR)/generated/$(OS)/release/$(MODEL)/dmd to
DMD = $(DMD_DIR)/generated/$(OS)/$(BUILD)/$(MODEL)/dmd, but now I think that it's a reasonable default.

$(BUILD) already defaults to release, however when overwritten I think it's "reasonable: to get the debug build everywhere.

@wilzbach
Copy link
Contributor Author

OK another PR at DMD (dlang/dmd#7444) and the we should hopefully be able to use BUILD=debug :)

@wilzbach
Copy link
Contributor Author

wilzbach commented Dec 16, 2017

So building Phobos with BUILD=debug now works and we got a proper stacktrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000a995f0 in gc.impl.conservative() (this=..., ptop=0x5dd6a40, pbot=0x5dd6a00) at src/gc/impl/conservative/gc.d:1990
1990	            auto p = *p1;
#0  0x0000000000a995f0 in gc.impl.conservative() (this=..., ptop=0x5dd6a40, pbot=0x5dd6a00) at src/gc/impl/conservative/gc.d:1990
#1  0x0000000000a99ef5 in gc.impl.conservative() (this=0x7fffffff9d60, __applyArg0=...) at src/gc/impl/conservative/gc.d:2188
#2  0x0000000000aa3ac1 in rt.util.container.treap() (this=0x7fffffff9d20, e=...) at src/rt/util/container/treap.d:47
#3  0x0000000000aa3e99 in rt.util.container.treap() (dg=..., node=0x47ba4c0) at src/rt/util/container/treap.d:221
#4  0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x1459cd0) at src/rt/util/container/treap.d:224
#5  0x0000000000aa3e76 in rt.util.container.treap() (dg=..., node=0x2ee44e0) at src/rt/util/container/treap.d:218
#6  0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x5c3be20) at src/rt/util/container/treap.d:224
#7  0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x5c3b760) at src/rt/util/container/treap.d:224
#8  0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x5aa4130) at src/rt/util/container/treap.d:224
#9  0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x5aa3a10) at src/rt/util/container/treap.d:224
#10 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x590d180) at src/rt/util/container/treap.d:224
#11 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x501d9d0) at src/rt/util/container/treap.d:224
#12 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x4fb1820) at src/rt/util/container/treap.d:224
#13 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x4fa3e50) at src/rt/util/container/treap.d:224
#14 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x4957320) at src/rt/util/container/treap.d:224
#15 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x4783b60) at src/rt/util/container/treap.d:224
#16 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x34a38e0) at src/rt/util/container/treap.d:224
#17 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x34876a0) at src/rt/util/container/treap.d:224
#18 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0x1747270) at src/rt/util/container/treap.d:224
#19 0x0000000000aa3ec5 in rt.util.container.treap() (dg=..., node=0xf03890) at src/rt/util/container/treap.d:224
#20 0x0000000000aa3af7 in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:52
#21 0x0000000000aa3a93 in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:47
#22 0x0000000000a99e74 in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2185
#23 0x0000000000a9a93a in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2417
#24 0x0000000000a98a7d in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9ff8: 64, bin=2 '\002') at src/gc/impl/conservative/gc.d:1711
#25 0x0000000000a9890e in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9ff8: 64, size=49) at src/gc/impl/conservative/gc.d:1676
#26 0x0000000000a96881 in gc.impl.conservative() (this=0xeae0d0, ti=0xe54470, alloc_size=@0x7fffffff9ff8: 64, bits=8, size=49) at src/gc/impl/conservative/gc.d:517
#27 0x0000000000a9c982 in gc.impl.conservative() (this=0xeae0d0, _param_3=@0x7fffffffa018: 0xe54470, _param_2=@0x7fffffff9ff8: 64, _param_1=@0x7fffffffa020: 8, _param_0=@0x7fffffffa028: 49) at src/gc/impl/conservative/gc.d:390
#28 0x0000000000a96923 in gc.impl.conservative() (this=0xeae0d0, __HID11=0x7fffffffa048, ti=0xe54470, bits=8, size=49) at src/gc/impl/conservative/gc.d:543
#29 0x0000000000a5a93f in gc_qalloc (__HID9=0x7fffffffa098, sz=49, ba=8, ti=0xe54470) at src/gc/proxy.d:122
#30 0x0000000000a5a113 in core.memory.GC.qalloc() (__HID2=0x7fffffffa120, ti=0xe54470, ba=8, sz=49) at src/core/memory.d:406
#31 0x0000000000a9f453 in rt.lifetime.__arrayAlloc() (__HID23=0x7fffffffa210, tinext=0xe2dea0, ti=0xe54470, info=..., arrsize=48) at src/rt/lifetime.d:445
#32 0x0000000000a5e54a in _d_arraysetlengthT (ti=0xe54470, newlength=3, p=0x2aaaaabcf3b8) at src/rt/lifetime.d:1585
#33 0x00000000009d4dfc in analysis.redundant_attributes.RedundantAttributesCheck.pushScope() (this=0x2aaaaabcf380) at src/analysis/redundant_attributes.d:167
#34 0x00000000009d4875 in analysis.redundant_attributes.RedundantAttributesCheck.__mixin20.visit() (this=0x2aaaaabcf380, n=0x342ff30) at src/analysis/redundant_attributes.d:159

(a good thing that CircleCi uses an older Linux Distro and thus line numbers on stack traces work :P)

CC @rainers @DmitryOlshansky

@CyberShadow
Copy link
Member

Is that a reentrant GC invocation? Should that not have raised a MemoryOperationError?

@rainers
Copy link
Member

rainers commented Dec 16, 2017

Is that a reentrant GC invocation? Should that not have raised a MemoryOperationError?

No, the GC is scanning ranges by iterating the Treap container. I suspect that a range has not been removed from the GC, but the memory has been freed. This only crashes if the page is actually unmapped from the virtual address space (which doesn't happen too often and might be causing the failures to be spurious).

@wilzbach
Copy link
Contributor Author

Hmm, but if we upgrade to CircleCi 2.0, we only get

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
core.exception.OutOfMemoryError@src/core/exception.d(702): Memory allocation failed
----------------
[Inferior 1 (process 2163) exited with code 01]
No stack.

See: #5930 (comment)

I suspect that a range has not been removed from the GC, but the memory has been freed.

Hmm, but that would explain why it SEGFAULTs in different places randomly then?

Any ideas/pointers on how we could find the faulty code? DScanner is rather large as it has many checks and uses libdparse, emsicontainers, libddoc, dsymbol, and inifiled.

@PetarKirov
Copy link
Member

PetarKirov commented Dec 16, 2017

As discussed with @wilzbach, this makes debugging via ssh on CircleCI much easier, so we'll go forward with this PR, even though it doesn't fix the underlying problem.

@wilzbach
Copy link
Contributor Author

I will-forced this PR now as it allows to debug the other failing PRs better.
We still don't know the underlying cause for the DScanner SEGFAULTs though :/

@wilzbach wilzbach merged commit 2f539a4 into dlang:master Dec 16, 2017
@wilzbach wilzbach deleted the dscanner-gdb branch December 16, 2017 13:34
@rainers
Copy link
Member

rainers commented Dec 16, 2017

core.exception.OutOfMemoryError@src/core/exception.d(702): Memory allocation failed

This is usually thrown when a C malloc/calloc/realloc failed. Without more stack trace it's unclear what happens there.

Hmm, but that would explain why it SEGFAULTs in different places randomly then?

Two of those stack traces look identical to the one above, but the others do not. Any crash could happen if a wrong GC range has been removed instead of just omitting a removal.

Any ideas/pointers on how we could find the faulty code?

Can you reproduce it locally? If not, can you get a core dump of the crash from the CI? If my suspicion is correct, you could log GC activity by enabling debug=PRINTF, debug=COLLECT_PRINTF, debug=PRINTF_TO_FILE in gc.impl.conservative.gc and try to figure out what happened to the crash address.

@PetarKirov
Copy link
Member

PetarKirov commented Dec 16, 2017

I ssh-ed on one of the CircleCI build VMs, and whether I add the following code, and whether I comment the writeln in there:

diff --git a/std/file.d b/std/file.d
index d9a8d88..f845867 100644
--- a/std/file.d
+++ b/std/file.d
@@ -3831,6 +3831,11 @@ private struct DirIteratorImpl
                 if (core.stdc.string.strcmp(&fdata.d_name[0], ".") &&
                     core.stdc.string.strcmp(&fdata.d_name[0], ".."))
                 {
+                    import std.stdio;
+                    auto var = _stack[$-1].dirpath;
+                    writef("%s, %s: ", var.ptr, var.length);
+                    //writeln(var);
+
                     _cur = DirEntry(_stack[$-1].dirpath, fdata);
                     return true;
                 }

I get either:

../dscanner-bb32e9f1e3e5206deb11b3dbc43ad44e23fabf96/dsc --config .dscanner.ini --styleCheck etc std -I.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x000000000095c0fc in std.range() (this=...) at ../dmd/generated/linux/debug/64/../../../../../phobos/std/range/package.d:999
999                        return fixRef(source[i].front);
#0  0x000000000095c0fc in std.range() (this=...) at ../dmd/generated/linux/debug/64/../../../../../phobos/std/range/package.d:999
#1  0x00000000008b07d1 in std.path() (segments=...) at ../dmd/generated/linux/debug/64/../../../../../phobos/std/path.d:1380
#2  0x00000000008b05e4 in std.path() (paths=...) at ../dmd/generated/linux/debug/64/../../../../../phobos/std/path.d:1396
#3  0x0000000000a595cb in std.file.DirEntry.__ctor() (this=..., fd=0x7ffff24a5031, path=...) at std/file.d:3103
#4  0x0000000000a5a244 in std.file.DirIteratorImpl.next() (this=...) at std/file.d:3834
#5  0x0000000000a5a582 in std.file.DirIteratorImpl.popFront() (this=...) at std/file.d:3932
#6  0x0000000000a5a841 in std.file.DirIterator.popFront() (this=...) at std/file.d:3954
#7  0x00000000008ab1b5 in dsymbol.modulecache.ModuleCache.addImportPaths() (this=0x7ffff7eeb530, root=...) at dsymbol/src/dsymbol/modulecache.d:133
#8  0x00000000008ab1a9 in dsymbol.modulecache.ModuleCache.addImportPaths() (this=0x7ffff7eeb530, root=...) at dsymbol/src/dsymbol/modulecache.d:132
#9  0x00000000008ab1a9 in dsymbol.modulecache.ModuleCache.addImportPaths() (this=0x7ffff7eeb530, root=...) at dsymbol/src/dsymbol/modulecache.d:132
#10 0x00000000008ab1a9 in dsymbol.modulecache.ModuleCache.addImportPaths() (this=0x7ffff7eeb530, root=...) at dsymbol/src/dsymbol/modulecache.d:132
#11 0x00000000008aafdf in dsymbol.modulecache.ModuleCache.addImportPaths() (this=..., paths=...) at dsymbol/src/dsymbol/modulecache.d:136
#12 0x0000000000a3a199 in D main (args=...) at src/main.d:151
#13 0x0000000000a4fecc in rt.dmain2._d_run_main() (this=0x7fffffffdda0) at src/rt/dmain2.d:500
#14 0x0000000000a4fd14 in rt.dmain2._d_run_main() (this=0x7fffffffdda0, dg=...) at src/rt/dmain2.d:461
#15 0x0000000000a4fe21 in rt.dmain2._d_run_main() (this=0x7fffffffdda0) at src/rt/dmain2.d:500
#16 0x0000000000a4fd14 in rt.dmain2._d_run_main() (this=0x7fffffffdda0, dg=...) at src/rt/dmain2.d:461
#17 0x0000000000a4fc78 in _d_run_main (argc=7, argv=0x7fffffffdea8, mainFunc=0xa39bbc <D main>) at src/rt/dmain2.d:520
#18 0x0000000000a475fa in main ()
gdb -q -ex run -ex bt -batch --args ../dscanner-bb32e9f1e3e5206deb11b3dbc43ad44e23fabf96/dsc --config .dscanner.ini --styleCheck etc std -I.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

<snip my write[f|ln](..) output>

Program received signal SIGSEGV, Segmentation fault.
0x0000000000a8dbe0 in gc.impl.conservative() (this=..., ptop=0x5db2540, pbot=0x5db2500) at src/gc/impl/conservative/gc.d:1990
1990	            auto p = *p1;
#0  0x0000000000a8dbe0 in gc.impl.conservative() (this=..., ptop=0x5db2540, pbot=0x5db2500) at src/gc/impl/conservative/gc.d:1990
#1  0x0000000000a8e4e5 in gc.impl.conservative() (this=0x7fffffff9a90, __applyArg0=...) at src/gc/impl/conservative/gc.d:2188
#2  0x0000000000a980b1 in rt.util.container.treap() (this=0x7fffffff9a50, e=...) at src/rt/util/container/treap.d:47
#3  0x0000000000a98489 in rt.util.container.treap() (dg=..., node=0x14590b0) at src/rt/util/container/treap.d:221
#4  0x0000000000a98466 in rt.util.container.treap() (dg=..., node=0x1fbdac0) at src/rt/util/container/treap.d:218
#5  0x0000000000a98466 in rt.util.container.treap() (dg=..., node=0x2f02910) at src/rt/util/container/treap.d:218
#6  0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x151c180) at src/rt/util/container/treap.d:224
#7  0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x11fc940) at src/rt/util/container/treap.d:224
#8  0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x58e8d90) at src/rt/util/container/treap.d:224
#9  0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x2f115c0) at src/rt/util/container/treap.d:224
#10 0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x4f96de0) at src/rt/util/container/treap.d:224
#11 0x0000000000a984b5 in rt.util.container.treap() (dg=..., node=0x4f8f390) at src/rt/util/container/treap.d:224
#12 0x0000000000a980e7 in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:52
#13 0x0000000000a98083 in rt.util.container.treap() (this=..., dg=...) at src/rt/util/container/treap.d:47
#14 0x0000000000a8e464 in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2185
#15 0x0000000000a8ef2a in gc.impl.conservative() (this=..., nostack=false) at src/gc/impl/conservative/gc.d:2417
#16 0x0000000000a8d06d in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9d28: 128, bin=3 '\003') at src/gc/impl/conservative/gc.d:1711
#17 0x0000000000a8cefe in gc.impl.conservative() (this=..., bits=8, alloc_size=@0x7fffffff9d28: 128, size=81) at src/gc/impl/conservative/gc.d:1676
#18 0x0000000000a8ae71 in gc.impl.conservative() (this=0xe9e0d0, ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, alloc_size=@0x7fffffff9d28: 128, bits=8, size=81) at src/gc/impl/conservative/gc.d:517
#19 0x0000000000a90f72 in gc.impl.conservative() (this=0xe9e0d0, _param_3=@0x7fffffff9d48: 0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, _param_2=@0x7fffffff9d28: 128, _param_1=@0x7fffffff9d50: 8, _param_0=@0x7fffffff9d58: 81) at src/gc/impl/conservative/gc.d:390
#20 0x0000000000a8af13 in gc.impl.conservative() (this=0xe9e0d0, __HID11=0x7fffffff9d78, ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, bits=8, size=81) at src/gc/impl/conservative/gc.d:543
#21 0x0000000000a4ed8f in gc_qalloc (__HID9=0x7fffffff9dc8, sz=81, ba=8, ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>) at src/gc/proxy.d:122
#22 0x0000000000a4e563 in core.memory.GC.qalloc() (__HID2=0x7fffffff9e50, ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, ba=8, sz=81) at src/core/memory.d:406
#23 0x0000000000a93a43 in rt.lifetime.__arrayAlloc() (__HID23=0x7fffffff9f40, tinext=0xe1e960 <TypeInfo_AxC6dparse3ast9Attribute.__init()>, ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, info=..., arrsize=80) at src/rt/lifetime.d:445
#24 0x0000000000a5299a in _d_arraysetlengthT (ti=0xe45b30 <TypeInfo_AAxC6dparse3ast9Attribute.__init()>, newlength=5, p=0x7ffff7fbed38) at src/rt/lifetime.d:1585
#25 0x00000000009cd888 in analysis.redundant_attributes.RedundantAttributesCheck.pushScope() (this=0x7ffff7fbed00) at src/analysis/redundant_attributes.d:167
#26 0x00000000009cd369 in analysis.redundant_attributes.RedundantAttributesCheck.__mixin22.visit() (this=0x7ffff7fbed00, n=0x250b1b0) at src/analysis/redundant_attributes.d:159
#27 0x00000000008d5afa in dparse.ast.Statement.accept() (this=0x250abf8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2650:2651
#28 0x00000000008c86f5 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, statement=0x250abf8) at libdparse/src/dparse/ast.d:289
#29 0x00000000008ceaee in dparse.ast.DeclarationOrStatement.accept() (this=0x250abd0, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1383:1384
#30 0x00000000008c7c55 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsOrStatement=0x250abd0) at libdparse/src/dparse/ast.d:204
#31 0x00000000008ce98d in dparse.ast.DeclarationsAndStatements.accept() (this=0x24fabf0, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1370:1370
#32 0x00000000008c7c75 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsAndStatements=0x24fabf0) at libdparse/src/dparse/ast.d:205
#33 0x00000000008cbae1 in dparse.ast.BlockStatement.accept() (this=0x24fabc0, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-973:973
#34 0x00000000009cd30e in analysis.redundant_attributes.RedundantAttributesCheck.__mixin20.visit() (this=0x7ffff7fbed00, n=0x24fabc0) at src/analysis/redundant_attributes.d:160
#35 0x00000000008d35ee in dparse.ast.StatementNoCaseNoDefault.accept() (this=0x24faac8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2291:2292
#36 0x00000000008c8715 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, statementNoCaseNoDefault=0x24faac8) at libdparse/src/dparse/ast.d:290
#37 0x00000000008d5ad9 in dparse.ast.Statement.accept() (this=0x24faa90, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2650:2650
#38 0x00000000008c86f5 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, statement=0x24faa90) at libdparse/src/dparse/ast.d:289
#39 0x00000000008d6962 in dparse.ast.SwitchStatement.accept() (this=0x24fa560, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2830:2831
#40 0x00000000008c8895 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, switchStatement=0x24fa560) at libdparse/src/dparse/ast.d:302
#41 0x00000000008cfb29 in dparse.ast.FinalSwitchStatement.accept() (this=0x24fa540, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1601:1601
#42 0x00000000008c7e55 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, finalSwitchStatement=0x24fa540) at libdparse/src/dparse/ast.d:220
#43 0x00000000008d36d5 in dparse.ast.StatementNoCaseNoDefault.accept() (this=0x24fa448, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2291:2299
#44 0x00000000008c8715 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, statementNoCaseNoDefault=0x24fa448) at libdparse/src/dparse/ast.d:290
#45 0x00000000008d5ad9 in dparse.ast.Statement.accept() (this=0x24fa410, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2650:2650
#46 0x00000000008c86f5 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, statement=0x24fa410) at libdparse/src/dparse/ast.d:289
#47 0x00000000008ceaee in dparse.ast.DeclarationOrStatement.accept() (this=0x24fa3e8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1383:1384
#48 0x00000000008c7c55 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsOrStatement=0x24fa3e8) at libdparse/src/dparse/ast.d:204
#49 0x00000000008ce98d in dparse.ast.DeclarationsAndStatements.accept() (this=0x48b4258, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1370:1370
#50 0x00000000008c7c75 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsAndStatements=0x48b4258) at libdparse/src/dparse/ast.d:205
#51 0x00000000008cbae1 in dparse.ast.BlockStatement.accept() (this=0x48b4228, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-973:973
#52 0x00000000009cd30e in analysis.redundant_attributes.RedundantAttributesCheck.__mixin20.visit() (this=0x7ffff7fbed00, n=0x48b4228) at src/analysis/redundant_attributes.d:160
#53 0x00000000008d02ec in dparse.ast.FunctionBody.accept() (this=0x48b41f0, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1728:1731
#54 0x00000000008c7f75 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, functionBody=0x48b41f0) at libdparse/src/dparse/ast.d:229
#55 0x00000000008d06bd in dparse.ast.FunctionDeclaration.accept() (this=0x48b3cf8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1760:1767
#56 0x00000000008c7fb5 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, functionDeclaration=0x48b3cf8) at libdparse/src/dparse/ast.d:231
#57 0x00000000008cd754 in dparse.ast.Declaration.accept() (this=0x48b3c20, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d:1292
#58 0x00000000009cd2a9 in analysis.redundant_attributes.RedundantAttributesCheck.visit() (this=0x7ffff7fbed00, decl=0x48b3c20) at src/analysis/redundant_attributes.d:64
#59 0x00000000008ceacd in dparse.ast.DeclarationOrStatement.accept() (this=0x48b3bf8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1383:1383
#60 0x00000000008c7c55 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsOrStatement=0x48b3bf8) at libdparse/src/dparse/ast.d:204
#61 0x00000000008ce98d in dparse.ast.DeclarationsAndStatements.accept() (this=0x1047930, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1370:1370
#62 0x00000000008c7c75 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, declarationsAndStatements=0x1047930) at libdparse/src/dparse/ast.d:205
#63 0x00000000008cbae1 in dparse.ast.BlockStatement.accept() (this=0x1047900, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-973:973
#64 0x00000000009cd30e in analysis.redundant_attributes.RedundantAttributesCheck.__mixin20.visit() (this=0x7ffff7fbed00, n=0x1047900) at src/analysis/redundant_attributes.d:160
#65 0x00000000008d02ec in dparse.ast.FunctionBody.accept() (this=0x10478c8, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1728:1731
#66 0x00000000008c7f75 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, functionBody=0x10478c8) at libdparse/src/dparse/ast.d:229
#67 0x00000000008d06bd in dparse.ast.FunctionDeclaration.accept() (this=0x1046a20, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-1760:1767
#68 0x00000000008c7fb5 in dparse.ast.ASTVisitor.visit() (this=0x7ffff7fbed00, functionDeclaration=0x1046a20) at libdparse/src/dparse/ast.d:231
#69 0x00000000008cd754 in dparse.ast.Declaration.accept() (this=0x1046880, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d:1292
#70 0x00000000009cd2a9 in analysis.redundant_attributes.RedundantAttributesCheck.visit() (this=0x7ffff7fbed00, decl=0x1046880) at src/analysis/redundant_attributes.d:64
#71 0x00000000008d2feb in dparse.ast.Module.accept() (this=0x119cea0, visitor=0x7ffff7fbed00) at libdparse/src/dparse/ast.d-mixin-2214:2216
#72 0x00000000009cd2da in analysis.redundant_attributes.RedundantAttributesCheck.__mixin19.visit() (this=0x7ffff7fbed00, n=0x119cea0) at src/analysis/redundant_attributes.d:160
#73 0x00000000009d2233 in analysis.run.analyze() (staticAnalyze=true, tokens=..., moduleCache=..., analysisConfig=..., m=0x119cea0, fileName=...) at src/analysis/run.d:498
#74 0x00000000009cf412 in analysis.run.analyze() (staticAnalyze=true, moduleCache=..., cache=..., config=..., fileNames=...) at src/analysis/run.d:182
#75 0x0000000000a3b397 in D main (args=...) at src/main.d:237
#76 0x0000000000a5073c in rt.dmain2._d_run_main() (this=0x7fffffffdda0) at src/rt/dmain2.d:500
#77 0x0000000000a50584 in rt.dmain2._d_run_main() (this=0x7fffffffdda0, dg=...) at src/rt/dmain2.d:461
#78 0x0000000000a50691 in rt.dmain2._d_run_main() (this=0x7fffffffdda0) at src/rt/dmain2.d:500
#79 0x0000000000a50584 in rt.dmain2._d_run_main() (this=0x7fffffffdda0, dg=...) at src/rt/dmain2.d:461
#80 0x0000000000a504e8 in _d_run_main (argc=7, argv=0x7fffffffdea8, mainFunc=0xa3a42c <D main>) at src/rt/dmain2.d:520
#81 0x0000000000a47e6a in main ()

Or:

gdb -q -ex run -ex bt -batch --args ../dscanner-bb32e9f1e3e5206deb11b3dbc43ad44e23fabf96/dsc --config .dscanner.ini --styleCheck etc std -I.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

<snip my write[f|ln](..) output>

[Inferior 1 (process 14863) exited with code 01]
No stack.

@wilzbach
Copy link
Contributor Author

wilzbach commented Feb 9, 2018

I opened a Bugzilla issue for this, s.t. it's not forgotten: https://issues.dlang.org/show_bug.cgi?id=18409

@JackStouffer
Copy link
Contributor

@rainers @wilzbach I got the GC log

gcx.log

Steps to reproduce

  1. make sure the make file var DSCANNER_HASH=6ba40537532fd795e83db0a7c8609512decdefc5
  2. uncomment the debug flags in druntime debug=PRINTF, debug=COLLECT_PRINTF, debug=PRINTF_TO_FILE in gc.impl.conservative.gc
  3. make -f posix.mak dscanner
  4. ../dscanner-6ba40537532fd795e83db0a7c8609512decdefc5/dsc -S --skipTests etc std

@JackStouffer
Copy link
Contributor

Where in the code it's failing

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
    frame #0: 0x00000001001da560 dsc`D8analysis24incorrect_infinite_range27IncorrectInfiniteRangeCheck5visitMFxC6dparse3ast14BlockStatementZv at incorrect_infinite_range.d:53
   50
   51  		override void visit(const BlockStatement bs)
   52  		{
-> 53  			if (bs.declarationsAndStatements is null)
   54  				return;
   55  			if (bs.declarationsAndStatements.declarationsAndStatements is null)
   56  				return;
Target 0: (dsc) stopped.

Assembly

dsc`D8analysis24incorrect_infinite_range27IncorrectInfiniteRangeCheck5visitMFxC6dparse3ast14BlockStatementZv:
    0x1001da548 <+0>:   pushq  %rbp
    0x1001da549 <+1>:   movq   %rsp, %rbp
    0x1001da54c <+4>:   subq   $0x20, %rsp
    0x1001da550 <+8>:   movq   %rbx, -0x18(%rbp)
    0x1001da554 <+12>:  movq   %rdi, -0x10(%rbp)
    0x1001da558 <+16>:  movq   %rsi, -0x8(%rbp)
    0x1001da55c <+20>:  movq   -0x8(%rbp), %rax
->  0x1001da560 <+24>:  cmpq   $0x0, 0x28(%rax)
    0x1001da565 <+29>:  jne    0x1001da56d               ; <+37>
    0x1001da567 <+31>:  movq   -0x18(%rbp), %rbx
    0x1001da56b <+35>:  leave
    0x1001da56c <+36>:  retq
    0x1001da56d <+37>:  movq   -0x8(%rbp), %rcx
    0x1001da571 <+41>:  movq   0x28(%rcx), %rdx
    0x1001da575 <+45>:  movq   0x20(%rdx), %rsi
    0x1001da579 <+49>:  orq    0x18(%rdx), %rsi
    0x1001da57d <+53>:  jne    0x1001da585               ; <+61>
    0x1001da57f <+55>:  movq   -0x18(%rbp), %rbx
    0x1001da583 <+59>:  leave
    0x1001da584 <+60>:  retq
    0x1001da585 <+61>:  movq   -0x8(%rbp), %rbx
    0x1001da589 <+65>:  movq   0x28(%rbx), %rax
    0x1001da58d <+69>:  cmpq   $0x1, 0x18(%rax)
    0x1001da592 <+74>:  je     0x1001da59a               ; <+82>
    0x1001da594 <+76>:  movq   -0x18(%rbp), %rbx
    0x1001da598 <+80>:  leave
    0x1001da599 <+81>:  retq
    0x1001da59a <+82>:  movq   -0x8(%rbp), %rcx
    0x1001da59e <+86>:  movq   0x28(%rcx), %rsi
    0x1001da5a2 <+90>:  movq   -0x10(%rbp), %rdi
    0x1001da5a6 <+94>:  movq   (%rdi), %rdx
    0x1001da5a9 <+97>:  callq  *0x220(%rdx)
    0x1001da5b0 <+104>: movq   -0x18(%rbp), %rbx
    0x1001da5b4 <+108>: leave
    0x1001da5b5 <+109>: retq
    0x1001da5b6 <+110>: nop

Backtrace

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x28)
  * frame #0: 0x00000001001da560 dsc`D8analysis24incorrect_infinite_range27IncorrectInfiniteRangeCheck5visitMFxC6dparse3ast14BlockStatementZv at incorrect_infinite_range.d:53
    frame #1: 0x00000001001da545 dsc`D8analysis24incorrect_infinite_range27IncorrectInfiniteRangeCheck5visitMFxC6dparse3ast12FunctionBodyZv at incorrect_infinite_range.d:48
    frame #2: 0x00000001000a738c dsc`D6dparse3ast10Destructor6acceptMxFCQBiQBe10ASTVisitorZv at ast.d:1462
    frame #3: 0x000000010009fe95 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd10DestructorZv at ast.d:211
    frame #4: 0x00000001000a58ea dsc`D6dparse3ast11Declaration6acceptMxFCQBjQBf10ASTVisitorZv + 794
    frame #5: 0x000000010009fd95 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd11DeclarationZv at ast.d:203
    frame #6: 0x00000001000ae895 dsc`D6dparse3ast10StructBody6acceptMxFCQBiQBe10ASTVisitorZv at ast.d-mixin-2753:2753
    frame #7: 0x00000001001da472 dsc`D8analysis24incorrect_infinite_range27IncorrectInfiniteRangeCheck5visitMFxC6dparse3ast10StructBodyZv at incorrect_infinite_range.d:29
    frame #8: 0x00000001000aea1b dsc`D6dparse3ast17StructDeclaration6acceptMxFCQBpQBl10ASTVisitorZv at ast.d-mixin-2775:2777
    frame #9: 0x00000001000a0975 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd17StructDeclarationZv at ast.d:298
    frame #10: 0x00000001000a5c8b dsc`D6dparse3ast11Declaration6acceptMxFCQBjQBf10ASTVisitorZv + 1723
    frame #11: 0x000000010009fd95 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd11DeclarationZv at ast.d:203
    frame #12: 0x00000001000a4c7e dsc`D6dparse3ast22ConditionalDeclaration6acceptMxFCQBuQBq10ASTVisitorZv at ast.d-mixin-1177:1178
    frame #13: 0x000000010009fcb5 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd22ConditionalDeclarationZv at ast.d:196
    frame #14: 0x00000001000a5830 dsc`D6dparse3ast11Declaration6acceptMxFCQBjQBf10ASTVisitorZv + 608
    frame #15: 0x000000010009fd95 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd11DeclarationZv at ast.d:203
    frame #16: 0x00000001000ab43d dsc`D6dparse3ast6Module6acceptMxFCQBdQz10ASTVisitorZv at ast.d-mixin-2214:2216
    frame #17: 0x00000001000a04f5 dsc`D6dparse3ast10ASTVisitor5visitMFxCQBhQBd6ModuleZv at ast.d:262
    frame #18: 0x000000010019ac53 dsc`D8analysis3run7analyzeFAyaxC6dparse3ast6ModulexSQBv6config20StaticAnalysisConfigKS7dsymbol11modulecache11ModuleCacheAxS3std12experimental5lexer__T14TokenStructureThVQFma305_0a20202020737472696e6720636f6d6d656e743b0a20202020737472696e6720747261696c696e67436f6d6d656e743b0a0a20202020696e74206f70436d702873697a655f7420692920636f6e73742070757265206e6f7468726f77204073616665207b0a202020202020202069662028696e646578203c2069292072657475726e202d313b0a202020202020202069662028696e646578203e2069292072657475726e20313b0a202020202020202072657475726e20303b0a202020207d0a0a20202020696e74206f70436d702872656620636f6e737420747970656f66287468697329206f746865722920636f6e73742070757265206e6f7468726f77204073616665207b0a202020202020202072657475726e206f70436d70286f746865722e696e646578293b0a202020207d0aZQYobZCQZv9container6rbtree__T12RedBlackTreeTSQBFw4base7MessageVQBFsa62_20612e6c696e65203c20622e6c696e65207c7c2028612e6c696e65203d3d20622e6c696e6520262620612e636f6c756d6e203c20622e636f6c756d6e2920Vbi1ZQGp at run.d:498
    frame #19: 0x0000000100197e24 dsc`D8analysis3run7analyzeFAAyaxSQBc6config20StaticAnalysisConfigKS6dparse5lexer11StringCacheKS7dsymbol11modulecache11ModuleCachebZb at run.d:182
    frame #20: 0x0000000100003136 dsc`_Dmain at main.d:237
    frame #21: 0x000000010024672c dsc`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv + 40
    frame #22: 0x00000001002465bc dsc`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ7tryExecMFMDFZvZv + 32
    frame #23: 0x0000000100246697 dsc`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZv + 139
    frame #24: 0x00000001002465bc dsc`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ7tryExecMFMDFZvZv + 32
    frame #25: 0x000000010024652a dsc`_d_run_main + 486
    frame #26: 0x0000000100003a26 dsc`main + 34
    frame #27: 0x0000000100001194 dsc`start + 52

@wilzbach
Copy link
Contributor Author

wilzbach commented Feb 9, 2018

Weird, now I can reproduce it locally too.
@JackStouffer does this fix everything for you locally?

diff --git a/src/analysis/incorrect_infinite_range.d b/src/analysis/incorrect_infinite_range.d
index cb3b4c8..bac0a7a 100644
--- a/src/analysis/incorrect_infinite_range.d
+++ b/src/analysis/incorrect_infinite_range.d
@@ -50,6 +50,8 @@ class IncorrectInfiniteRangeCheck : BaseAnalyzer
 
 	override void visit(const BlockStatement bs)
 	{
+		if (bs is null)
+			return;
 		if (bs.declarationsAndStatements is null)
 			return;
 		if (bs.declarationsAndStatements.declarationsAndStatements is null)

@wilzbach
Copy link
Contributor Author

wilzbach commented Feb 9, 2018

Oh so the current failure is a DScanner bug (-> #6148), but it's not the GC Segfault we have observed initially :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants