Split-up regex tests to please auto-tester

[wilzbach]

This error appears more and more frequently on auto-tester:

make[1]: *** [generated/linux/release/64/unittest/std/regex/internal/tests.o] Killed

See also: braddr/d-tester#66

Valgrind:

I guess - in lack of fixing DMD being easy - the only "solution" that we can do for now, is to split-up the tests.

After the split-up:

=== tests.d ====

=== tests2.d ===

Not ideal, but should probably buy us more air on the auto-tester.

[dlang-bot]

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

[quickfur]

Dmd's memory usage is really getting out of hand. I used to be able to compile my D projects on a low-memory system without any glitches, but these days even trivial programs frequently invoke the kernel OOM killer. Now I'm unable to compile anything but the most trivial of programs on the same low-memory system. Dmd's memory allocation strategy may have been chosen to improve compilation speed, but what good is compilation speed when it doesn't even finish compiling before it gets killed by the kernel?

Recompiling dmd with version=GC doesn't help, because it just causes a segfault somewhere. If this state of things continue to hold, I'm gonna have to give up using dmd altogether. :-(

[JackStouffer]

@quickfur The new ctfe engine by @UplinkCoder is supposed to fix the CTFE problem, but I don't know what happened to it :/

[UplinkCoder]

@quickfur I'd really like to hear more details about this.
I am going to build a little memory profiler into dmd which will be able to tell you what is allocated and why.

[quickfur]

@UplinkCoder I'm very interested in a tool that will explain why/where dmd is allocating memory, so that I know where to look. Right now it's just an unhelpful Killed from the OOM killer.

I did try to (very crudely) add memory usage output lines to main (basically a crude read of /proc/<pid>/meminfo, since Linux "helpfully" doesn't bother filling out the Posix resource usage structs with memory usage info), and it appears that a lot of memory is allocated somewhere between semantic2() and semantic3(). Very broad, I know, but I haven't had the time to narrow it down further than that just yet.

[UplinkCoder]

@JackStouffer It's still in progress :)
Though time is scarce, I have a few things on my plate :

• full time job
• convincing people that we spent too much compile-time on templates.
• working on a proper solution for minimizing time spent on templates.

this leaves me with very little time/energy to dedicate to newCTFE.
which has less appeal now that I am fighting a much bigger timewaster (templates).

[quickfur]

@UplinkCoder I can't tell you what you should do, but it would be nice if we got newCTFE into a releasable (even if incomplete) state, before we start tackling the big monster that is the template code. I may very well be wrong, but I suspect improving templates may take more effort than finishing newCTFE, but that's just my uninformed opinion.

Or maybe the way to go forward is for some of us to start submitting PRs for the newCTFE branch? Would that help? I haven't looked at the code yet, but I am interested enough to dig into it if it will help prod progress along.

[UplinkCoder]

@quickfur could you join the #d on irc.freenode.net, I'd like to discuss this if you have a minute.

[quickfur]

Sorry, can we do that tomorrow? I'm trying to narrow down another dmd bug right now, and want to stay focused.

[UplinkCoder]

sure, I just didn't want to abuse this github pr for a longer conversation.
I'll be in #d tomarrow, my nick is usually uplinkSOMETHING

[wilzbach]

@UplinkCoder I'm very interested in a tool that will explain why/where dmd is allocating memory, so that I know where to look. Right now it's just an unhelpful Killed from the OOM killer.

You don't need to modify DMD for this, simply use existing tools:

valgrind --tool=massif <command>
massif-visualizer massif.out.<pid>

A simple preview:

[UplinkCoder]

@wilzbach this will not tell you which symbols/modules you are compiling are causing this.
though thanks for posting the tool it's a nice ui.