Merge 2.074 front-end + libs by kinke · Pull Request #2076 · ldc-developers/ldc

kinke · 2017-04-22T01:56:46Z

No description provided.

JohanEngelen · 2017-04-22T10:07:11Z

ddmd/target.d

            realalignsize = 2;
            reverseCppOverloads = true;
            c_longsize = 4;
+            if (ptrsize == 4)


Let's version this out.

This code is only used by DMD, we have our own Target struct at the top of the file.

kinke · 2017-04-22T12:31:38Z

ddmd/aggregate.d

-                        return 0;
-                    }
-                }
-            }


This will need to be checked, i.e., whether a proper fix preventing the same field from being added multiple times to the fields list has made it into upstream 2.074.

kinke · 2017-04-22T14:42:54Z

First upstream PR: dlang/druntime#1815

Fixes compilable/b17111.d, see dlang/dmd@84fcc60.

kinke · 2017-04-22T20:36:28Z

Linux: 4 distinct failing unittest modules left, the rest (dmd-testsuite, lit-tests, LDC unittests) passes.

dnadlinger · 2017-04-22T20:38:35Z

gen/statements.cpp

+        // skip over casts
+        auto ce = cs->exp;
+        while (ce->op == TOKcast)
+          ce = static_cast<CastExp *>(ce)->e1;


Is there any chance that this drops a narrowing conversion? (Sorry if this is totally off, haven't studied the code closely.)

Nope. It's only used to check for a nested VarExp; cs->exp, the original expression, is evaluated in all cases (variable/constant).

kinke · 2017-04-23T02:30:37Z

2nd upstream PR: dlang/druntime#1816

kinke · 2017-04-23T02:56:24Z

Next one: dlang/phobos#5345

Required for OSX, where version `CheckFiberMigration` is set.

kinke · 2017-04-23T22:15:19Z

Another one: dlang/phobos#5346

kinke · 2017-04-23T22:29:54Z

Update:

There are mangling issues when using LDC 0.17.2 as host compiler (Travis LLVM 3.6 job).
OSX isn't tested anymore due to Travis OSX doesn't like LDC 1.2 as host compiler #2077, but the previous druntime compile error should be fixed (nogc fiber migration check).
Linux and Windows only have 2 failing unittest modules left:
- A new core.memory unittest fails in release only (-O2 and higher), apparently because pureMalloc(size_t.max - 2) doesn't return null. I quickly checked the IR for a (working) stand-alone test, and both non-optimized and optimized IR looked fine.
- std.math fails a bunch of new ieeeFlags checks. We already disabled a few of those (right above) due to potential instruction reordering (see std.math ieeeFlags cannot be reliably tested #888), but these new tests are more elaborate (using delegates), so I don't think LLVM interferes here even for debug builds, but I haven't digged deeper.

Conflicts: runtime/druntime runtime/phobos

kinke · 2017-05-11T23:54:08Z

With ldc-developers/phobos@2220a3a, all std.math-debug tests work on my Win64 box except for a rounding test.

kinke · 2017-05-15T23:02:19Z

Alright, first green runs - all but the 2 Travis -DBUILD_SHARED_LIBS=ON jobs should pass now (they seem to be using wrong filenames for the shared stdlibs or something like that).

Conflicts: runtime/druntime runtime/phobos

kinke · 2017-06-10T01:40:12Z

Upgraded to 2.074.1 and synced with master.

In case the host real_t is double (e.g., for MSVC hosts).

Conflicts: runtime/druntime

Conflicts: tests/d2/dmd-testsuite

kinke · 2017-07-13T19:16:15Z

As to the final issue: not even a Phobos hello-world program can be linked when using the shared runtime libs due to undefined members of this nested struct:

echo "import std.stdio; void main() { writefln(\"Hello LDC2 %d\", (void*).sizeof); }" > hello.d
bin/ldc2 hello.d -link-debuglib
/home/martin/build-ldc/lib/libphobos2-ldc-debug.so: undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb'
/home/martin/build-ldc/lib/libphobos2-ldc-debug.so: undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa'
/home/martin/build-ldc/lib/libphobos2-ldc-debug.so: undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm'
/home/martin/build-ldc/lib/libphobos2-ldc-debug.so: undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv'
/home/martin/build-ldc/lib/libphobos2-ldc-debug.so: undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result'
collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc failed with status: 1

Any ideas why only the shared libs are affected? No public visibility or something like that? I know nothing about shared libs on Linux, but the template seems to be instantiated in Phobos (=> #2168).

Conflicts: runtime/druntime

Conflicts: runtime/druntime tests/d2/dmd-testsuite

kinke · 2017-07-21T17:14:30Z

Static:

$ readelf -Ws build-ldc/lib/libphobos2-ldc.a | grep _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result
   886: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb
   887: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa
   888: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm
   889: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv

vs. shared:

$ readelf -W --dyn-syms build-ldc/lib/libphobos2-ldc-shared.so | grep _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result
    57: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb
   233: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa
   255: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm
   298: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv
$ readelf -Ws build-ldc/lib/libphobos2-ldc-shared.so | grep _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result
    57: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb
   233: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa
   255: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm
   298: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv
  3318: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb
  9136: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa
  9696: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm
 11076: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND _D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv

Edit: readelf -s ...-shared.so displays both .dynsym and .symtab tables, that's why the symbols are listed twice.

kinke · 2017-07-21T18:28:53Z

Oh and btw, both shared and static Phobos libs consist of the same D object files.
Any suggestions? I'm stuck. Happens on OSX as well, not just Linux.

JohanEngelen · 2017-07-21T18:47:29Z

So the function is not codegenned, I've run into similar issues with Weka's codebase where some templates are culled too aggressively. But also because of failing to deduce attributes in certain cases but not in others: https://issues.dlang.org/show_bug.cgi?id=17541.
Can you check if there isn't another symbol defined for that function but without the attributes?

kinke · 2017-07-21T19:03:46Z

So the function is not codegenned

But then why is the linker not complaining when feeding it the static lib with exactly the same D objects?

JohanEngelen · 2017-07-21T19:13:56Z

Conjecture: in the static case, the linker sees all symbols and knows it can strip the symbol in which the undef thing is referenced. In the sharedlib case, perhaps the linker cannot make as many assumptions, because the shared lib could be replaced with another and then behavior could be different when certain symbols are available or not? Really, I don't know.

kinke · 2017-07-21T19:33:44Z

Thanks anyway - I tested the needsCodegen() hack from #2168, commenting out this, and hello-world can then be linked successfully.

JohanEngelen · 2017-07-21T19:38:30Z

This is the first time I see that happening in Phobos stuff. That's very good news: see if it happens with dmd too, dustmite and submit bugreport. The bug with Weka source has been a bit elusive so far. (or I've lost track in the confusion of the weird complex bugs I submitted...)

kinke · 2017-07-21T20:08:55Z

DMD 2.074.0 has no problems with linking the above hello-world with its combined (druntime + Phobos) libphobos2.so on Linux x64. That would have been shocking. ;)

JohanEngelen · 2017-07-21T20:20:32Z

lol heh true. (thought they are doing static linking) Very strange that our front-end culls, and theirs doesn't though....

kinke · 2017-07-21T22:05:21Z

Very strange that our front-end culls, and theirs doesn't though

I have a strong assumption - they seem to build druntime & Phobos in a single command line, all D files at once: https://github.com/dlang/phobos/blob/master/posix.mak#L321
Whereas we compile each object separately. And that culling basically checks whether a template is (also) instantiated in some imported non-root module. So if the complete importable source is specified on the command line, this problem can't occur - everything is in a root module.

kinke · 2017-07-21T22:07:40Z

So this should basically be due to (supposedly fixed) https://issues.dlang.org/show_bug.cgi?id=2500 or some more sophisticated variant of it.

kinke · 2017-07-21T22:10:35Z

thought they are doing static linking

They do by default but ship with static and shared libs, so I caught the linker line via -v and linked against the shared lib manually.

kinke · 2017-07-22T00:55:30Z

All green now after the template culling hack.

dnadlinger · 2017-07-26T21:46:59Z

CMakeLists.txt


 install(PROGRAMS ${LDC_EXE_FULL} DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
 install(PROGRAMS ${LDMD_EXE_FULL} DESTINATION ${CMAKE_INSTALL_PREFIX}/bin)
+if(NOT CMAKE_INSTALL_PREFIX STREQUAL "/usr")


Huh? This smells massively.

I didn't want to install the bash script system-wide. It's more of an internal script by itself, useful for local builds and release packages, but not system-wide availability IMO.
Correct context: #2198

kinke · 2017-07-27T11:28:29Z

I created a test branch merge-2.074-2 with the template culling hack reverted and #2231 merged. As expected, this also fixes the linking issues with shared Phobos. The Phobos unittest runners can also be linked successfully although the runtime modules are compiled separately. See https://travis-ci.org/ldc-developers/ldc/builds/257913800. And both Semaphore jobs successfully compiled Phobos; that may have been a fluke or the #2231 Semaphore issue has been fixed in 2.074.

I'd prefer using #2231 (maybe merged as part of 2.074 if the Semaphore issue reproducibly vanishes) and expose the weakened template culling hack as command-line option. Thoughts?

kinke · 2017-07-27T21:02:33Z

if the Semaphore issue reproducibly vanishes

Well, no Phobos compilation failures after 10 jobs, so I think it's safe to assume 2.074 gets rid of that last issue of #2231.
[The random failures when linking the testrunners are due to parallel ninja invokations when running the build-*-test-runner* CTest tests in parallel. Unfortunately, Semaphore doesn't support an in-source config file, so I can't switch the global config to ninja all all-test-runners as long as #2231 isn't merged.]

Conflicts: runtime/druntime runtime/phobos

kinke added 3 commits April 22, 2017 03:52

Merge 2.074.0 front-end

f70a51a

druntime: Remove obsolete OSX code in rt.sections_ldc

83c9d66

druntime: Fix rt.sections_elf_shared regression

0a11aa8

JohanEngelen reviewed Apr 22, 2017

View reviewed changes

kinke commented Apr 22, 2017

View reviewed changes

druntime: Fix __cmp(void[], void[])

708486d

kinke added 2 commits April 22, 2017 21:09

dmd-testsuite: Fix compile error for runnable/ldc_enum

1e11275

Support cast runtime variables as case expressions

15e565e

Fixes compilable/b17111.d, see dlang/dmd@84fcc60.

dnadlinger reviewed Apr 22, 2017

View reviewed changes

druntime: Fix rt.arrayint (RAX instead of EAX for 64-bit inline asm)

fe3d36a

Phobos: Fix missing alignment for kernighan_ritchie allocator unittests

a5fe1ab

kinke added 4 commits April 23, 2017 15:55

Merge branch 'master' into merge-2.074

6ac98e3

dmd-testsuite: Disable Windows-specific runnable/testptrref

bcec3d2

druntime: Make LDC-specific fiber migration check @nogc

a966f22

Required for OSX, where version `CheckFiberMigration` is set.

Phobos: Relax a few unittests for double-precision reals

0637caa

Merge branch 'master' into merge-2.074

291c2a6

Conflicts: runtime/druntime runtime/phobos

kinke added 6 commits May 12, 2017 18:00

Phobos: Apply upstream SSE control register fixes

60285b7

Phobos: Tiny std.math inline assembly constraints fixes

0ae2a95

druntime: Mark dladdr() as @nogc for OSX

51f1b13

Support DMD's new undocumented -vcg-ast switch

6a44d57

druntime: Adapt core.memory to LLVM optimizer cleverness

a3025b0

Phobos: Disable some new std.math tests affected by issue #888

ff33356

Merge branch 'master' into merge-2.074

45eacd0

Conflicts: runtime/druntime runtime/phobos

Fix Target.RealProperties potentially aliasing Target.DoubleProperties

03d8034

In case the host real_t is double (e.g., for MSVC hosts).

PetarKirov mentioned this pull request Jun 25, 2017

Run DScanner on itself dlang-community/D-Scanner#479

Merged

Merge branch 'master' into merge-2.074

6036769

Conflicts: runtime/druntime

kinke mentioned this pull request Jul 2, 2017

Performance regressions without LTO in LDC 1.3.0-beta[1|2] #2168

Open

Merge branch 'master' into merge-2.074

a0a20cd

Conflicts: tests/d2/dmd-testsuite

kinke added 2 commits July 15, 2017 23:53

Merge branch 'master' into merge-2.074

8a3188d

Conflicts: runtime/druntime

Merge branch 'master' into merge-2.074

cd45257

Conflicts: runtime/druntime tests/d2/dmd-testsuite

dnadlinger reviewed Jul 26, 2017

View reviewed changes

Merge branch 'master' into merge-2.074

66241ab

Conflicts: runtime/druntime runtime/phobos

kinke force-pushed the merge-2.074 branch from a9dac06 to 66241ab Compare July 27, 2017 22:15

kinke merged commit 7a9960f into master Jul 27, 2017

kinke deleted the merge-2.074 branch July 27, 2017 22:16

Uh oh!

Comments

Conversation

kinke commented Apr 22, 2017

Uh oh!

JohanEngelen Apr 22, 2017

Choose a reason for hiding this comment

Uh oh!

kinke Apr 22, 2017

Choose a reason for hiding this comment

Uh oh!

kinke Apr 22, 2017

Choose a reason for hiding this comment

Uh oh!

kinke commented Apr 22, 2017

Uh oh!

kinke commented Apr 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dnadlinger Apr 22, 2017

Choose a reason for hiding this comment

Uh oh!

kinke Apr 22, 2017

Choose a reason for hiding this comment

Uh oh!

kinke commented Apr 23, 2017

Uh oh!

kinke commented Apr 23, 2017

Uh oh!

kinke commented Apr 23, 2017

Uh oh!

kinke commented Apr 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented May 11, 2017

Uh oh!

kinke commented May 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jun 10, 2017

Uh oh!

kinke commented Jul 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohanEngelen commented Jul 21, 2017

Uh oh!

kinke commented Jul 21, 2017

Uh oh!

JohanEngelen commented Jul 21, 2017

Uh oh!

kinke commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohanEngelen commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jul 21, 2017

Uh oh!

JohanEngelen commented Jul 21, 2017

Uh oh!

kinke commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jul 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kinke commented Jul 21, 2017

Uh oh!

kinke commented Jul 22, 2017

Uh oh!

dnadlinger Jul 26, 2017

Choose a reason for hiding this comment

Uh oh!

kinke Jul 26, 2017

kinke commented Apr 22, 2017 •

edited

Loading

kinke commented Apr 23, 2017 •

edited

Loading

kinke commented May 15, 2017 •

edited

Loading

kinke commented Jul 13, 2017 •

edited

Loading

kinke commented Jul 21, 2017 •

edited

Loading

kinke commented Jul 21, 2017 •

edited

Loading

kinke commented Jul 21, 2017 •

edited

Loading

JohanEngelen commented Jul 21, 2017 •

edited

Loading

kinke commented Jul 21, 2017 •

edited

Loading

kinke commented Jul 21, 2017 •

edited

Loading