Add support for LLVM 4 #1592

ilovezfs · 2017-02-16T10:45:33Z

In anticipation of the imminent LLVM 4.0.0 release, I'm testing whether each of the reverse dependencies of LLVM in Homebrew core are able to build. Unfortunately, the ponyc build (28397da) fails with

/usr/bin/clang -MMD -MP -march=native -Werror -Wconversion -Wno-sign-conversion -Wextra -Wall -mcx16 -O3 -DNDEBUG -flto -DPONY_USE_LTO -std=gnu11 -fexceptions -DPONY_VERSION=\"0.10.0-28397da\" -DLLVM_VERSION=\"4.0.0\" -DPONY_COMPILER=\"/usr/bin/clang\" -DPONY_ARCH=\"native\" -DBUILD_COMPILER=\""Apple LLVM version 8.0.0 (clang-800.0.42.1)"\" -DPONY_BUILD_CONFIG=\"release\" -mmacosx-version-min=10.8 -Os -w -pipe -march=native -mmacosx-version-min=10.11 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -c -o build/release/obj/libponyc/codegen/gencontrol.o src/libponyc/codegen/gencontrol.c  -I src/common/ -I src/libponyrt/ -isystem /usr/local/Cellar/llvm/4.0.0-rc2/include
gendebug.cc
/usr/bin/clang++ -std=c++11 -stdlib=libc++ -MMD -MP -march=native -Werror -Wconversion -Wno-sign-conversion -Wextra -Wall -mcx16 -O3 -DNDEBUG -flto -DPONY_USE_LTO -std=gnu++11 -fno-rtti -stdlib=libc++ -mmacosx-version-min=10.8 -Os -w -pipe -march=native -mmacosx-version-min=10.11 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -c -o build/release/obj/libponyc/codegen/gendebug.o src/libponyc/codegen/gendebug.cc  -I src/common/ -I src/libponyrt/ -isystem /usr/local/Cellar/llvm/4.0.0-rc2/include
src/libponyc/codegen/gendebug.cc:75:43: error: cannot initialize a parameter of type 'llvm::DIFile *' with an lvalue of type 'const char *'
  return wrap(pd->createCompileUnit(lang, file, dir, producer, optimized,
                                          ^~~~
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:109:46: note: passing argument to parameter 'File' here
    createCompileUnit(unsigned Lang, DIFile *File, StringRef Producer,
                                             ^
src/libponyc/codegen/gendebug.cc:121:8: error: cannot initialize a parameter of type 'DINode::DIFlags' with an lvalue of type 'int'
    0, optimized);
       ^~~~~~~~~
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:625:25: note: passing argument to parameter 'Flags' here
        DINode::DIFlags Flags = DINode::FlagZero, bool isOptimized = false,
                        ^
src/libponyc/codegen/gendebug.cc:143:61: error: cannot initialize a parameter of type 'DINode::DIFlags' with an rvalue of type 'int'
    unwrap<DIFile>(file), line, unwrap<DIType>(type), true, 0));
                                                            ^
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:522:40: note: passing argument to parameter 'Flags' here
                       DINode::DIFlags Flags = DINode::FlagZero,
                                       ^
src/libponyc/codegen/gendebug.cc:162:33: error: cannot initialize a parameter of type 'DINode::DIFlags' with an rvalue of type 'int'
    unwrap<DIType>(type), true, 0));
                                ^
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:540:45: note: passing argument to parameter 'Flags' here
                            DINode::DIFlags Flags = DINode::FlagZero);
                                            ^
src/libponyc/codegen/gendebug.cc:198:64: error: too many arguments to function call, expected 3, have 4
  return wrap(pd->createBasicType(name, size_bits, align_bits, encoding));
              ~~~~~~~~~~~~~~~~~~~                              ^~~~~~~~
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:157:5: note: 'createBasicType' declared here
    DIBasicType *createBasicType(StringRef Name, uint64_t SizeInBits,
    ^
src/libponyc/codegen/gendebug.cc:235:56: error: cannot initialize a parameter of type 'DINode::DIFlags' with an rvalue of type 'int'
    unwrap<DIFile>(file), line, size_bits, align_bits, 0, nullptr,
                                                       ^
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:329:68: note: passing argument to parameter 'Flags' here
        uint64_t SizeInBits, uint32_t AlignInBits, DINode::DIFlags Flags,
                                                                   ^
src/libponyc/codegen/gendebug.cc:257:18: error: cannot initialize a parameter of type 'DINode::DIFlags' with an lvalue of type 'unsigned int'
    offset_bits, flags, unwrap<DIType>(type)));
                 ^~~~~
/usr/local/Cellar/llvm/4.0.0-rc2/include/llvm/IR/DIBuilder.h:229:53: note: passing argument to parameter 'Flags' here
                                    DINode::DIFlags Flags, DIType *Ty);
                                                    ^
7 errors generated.
make: *** [build/release/obj/libponyc/codegen/gendebug.o] Error 1

Full build log: https://gist.github.com/ilovezfs/166d8b2494a5ea1dbf73d66cc08114cf

The text was updated successfully, but these errors were encountered:

Praetonus · 2017-02-16T12:30:37Z

We haven't added LLVM 4.0 support yet. Unfortunately I'm unable to find the changelog for that version so I don't know what kind of changes would be required. Has anyone been able to find it?

ilovezfs · 2017-02-16T12:31:50Z

http://www.llvm.org/pre-releases/4.0.0/rc2/docs/ReleaseNotes.html

Praetonus · 2017-02-16T15:01:16Z

Thanks for the link. Apparently the changes to the DI API aren't described yet. I think we'll have to wait until there is enough information in the changelog to work on adding the support of that version.

rurban · 2017-03-11T18:17:21Z

The current dev version llvm-5.0.0svn requires the same minor changes.
Only llvm::DIFile * createFile and some minor type adjustments. Nothing serious though.

In my other projects clang-4.0 is about 20% faster at run-time, and with zapcc (based on that) compile-times are drastically faster. (like 5x).

I fixed the llvm 4.0 and 5.0 compilation issues, but 2 got test regressions: CodegenTest.JitRun, CodegenTest.CCallback. ~~And with debug EnableABIBreakingChecks is missing.~~
See my llvm-4 branch below.
The old CodegenTest.LoadCombineDarwinError is also still present.

WIP: [ FAILED ] CodegenTest.JitRun [ FAILED ] CodegenTest.CCallback Closes ponylang#1592

WIP: [ FAILED ] CodegenTest.JitRun [ FAILED ] CodegenTest.CCallback Closes ponylang#1592 ABIBreakingChecks policy: Disable ABI checks, as LLVM_ENABLE_ABI_BREAKING_CHECKS is undefined.

ilovezfs · 2017-03-14T11:00:45Z

@rurban it builds with your patch. However, there's a segfault in the test block:

iMac-TMP:~ joe$ brew test ponyc -vd
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/ponyc.rb
Testing ponyc
==> Using the sandbox
/usr/bin/sandbox-exec -f /tmp/homebrew20170314-16039-1sa9ptf.sb /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/ponyc.rb -vd --HEAD
/usr/local/Homebrew/Library/Homebrew/test.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/ponyc.rb
==> /usr/local/Cellar/ponyc/HEAD-4951d96_1/bin/ponyc -rexpr /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/stdlib
Building builtin -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/builtin
Building /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/stdlib -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/stdlib
Building ponytest -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/ponytest
Building time -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/time
Building collections -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/collections
Building assert -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/assert
Building encode/base64 -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/encode/base64
Building builtin_test -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/builtin_test
Building buffered -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/buffered
Building bureaucracy -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/bureaucracy
Building promises -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/promises
Building capsicum -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/capsicum
Building files -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/files
Building collections/persistent -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/collections/persistent
Building random -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/random
Building crypto -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/crypto
Building format -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/format
Building debug -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/debug
Building glob -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/glob
Building regex -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/regex
Building net/http -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/net/http
Building net -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/net
Building net/ssl -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/net/ssl
Building ini -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/ini
Building json -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/json
Building logger -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/logger
Building math -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/math
Building options -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/options
Building ponybench -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/ponybench
Building term -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/term
Building strings -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/strings
Building signals -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/signals
Building process -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/process
Building itertools -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/itertools
Building serialise -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/serialise
==> /usr/local/Cellar/ponyc/HEAD-4951d96_1/bin/ponyc test
Building builtin -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/builtin
Building test -> /private/tmp/ponyc-test-20170314-16040-2sq925/test
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Stack dump:
0.	Running pass 'CallGraph Pass Manager' on module 'test'.
1.	Running pass 'Loop Pass Manager' on function '@Stdin_ref__read_b'
2.	Running pass 'Loop Invariant Code Motion' on basic block '%while_body'
/usr/local/Homebrew/Library/Homebrew/debrew.rb:11:in `raise'
BuildError: Failed executing: /usr/local/Cellar/ponyc/HEAD-4951d96_1/bin/ponyc test
1. raise
2. ignore
3. backtrace
4. irb
5. shell
Choose an action: 5
When you exit this shell, you will return to the menu.
bash-3.2$ /usr/local/Cellar/ponyc/HEAD-4951d96_1/bin/ponyc test
Building builtin -> /usr/local/Cellar/ponyc/HEAD-4951d96_1/packages/builtin
Building test -> /private/tmp/ponyc-test-20170314-16040-2sq925/test
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Stack dump:
0.	Running pass 'CallGraph Pass Manager' on module 'test'.
1.	Running pass 'Loop Pass Manager' on function '@Stdin_ref__read_b'
2.	Running pass 'Loop Invariant Code Motion' on basic block '%while_body'
Segmentation fault: 11
bash-3.2$

rurban · 2017-03-14T11:44:54Z

Yes, that's why it's still WIP
I looked at how crystal updated from llvm 3.9 to 4.0 and they had the very same changes. So it's a bit weird.

killerswan · 2017-03-22T22:51:51Z

When you're ready, @rurban, I can help you get the CI bits going for LLVM 4. (Travis changes will look something like my partial master...killerswan:llvm4-ci branch.)

ilovezfs · 2017-03-31T02:08:24Z

Any progress here?

WIP: [ FAILED ] CodegenTest.JitRun [ FAILED ] CodegenTest.CCallback Closes ponylang#1592 ABIBreakingChecks policy: Disable ABI checks, as LLVM_ENABLE_ABI_BREAKING_CHECKS is undefined.

rurban · 2017-04-02T08:45:58Z

nope, busy with other stuff

killerswan · 2017-04-02T12:29:21Z

I've renamed the ticket because this is a new feature, more or less, not a bug.

SeanTAllen · 2017-04-05T20:26:56Z

@kulibali reports during sync he has this compiling on Windows but it crashes.

@jemc suggested trying to disable some of our custom optimizations and see if that helps.

chalcolith · 2017-04-07T00:03:47Z

LLVM changed the way of doing TBAA a while back, and have finally deprecated the old way: http://llvm.org/viewvc/llvm-project?view=revision&revision=286291

This will require some changes in code generation.

Praetonus · 2017-04-28T23:14:12Z

@kulibali I believe that the LLVM internals can automatically update metadatas in the old TBAA format to the new one. Of course, it doesn't magically add the more precise information that the new format supports, but at least it shouldn't be required to update that part to get Pony running on LLVM 4. I think it would be better as a separate change later.

Have you made any progress on the Windows crashes?

chalcolith · 2017-04-28T23:38:17Z

The JIT tests are failing because they validate the LLVM code, which fails because the TBAA format is old.

chalcolith · 2017-04-28T23:39:01Z

I have been working very slowly on the TBAA changes -- you can see in my branch https://github.com/kulibali/ponyc/tree/llvm400

This change includes the following: - Compiler code updates for changes to the LLVM 4.0.0 API. - Updates to Type-Based Alias Analysis metadata format for LLVM 4.0.0. - Builds in Release mode on Windows to conform to the Unix makefile. - Some efficiency tweaks for the Windows build.

ilovezfs · 2017-09-12T05:05:33Z

LLVM 5 is now out. I'm not sure if that adds any new challenges for ponyc but that may need to be factored into this issue.

chalcolith · 2017-09-12T15:17:27Z

I have a branch that compiles with 5.0.0 (https://github.com/kulibali/ponyc/tree/llvm500). It works on Windows; unfortunately it has the same problem on Linux as 4.0.1; certain generated code segfaults if the HeapToStack pass is enabled (see discussion in #2061).

This change includes the following: - Compiler code updates for changes to the LLVM 4.0.0 API. - Updates to Type-Based Alias Analysis metadata format for LLVM 4.0.0. - Builds in Release mode on Windows to conform to the Unix makefile. - Some efficiency tweaks for the Windows build.

… and 5.0.0. This change includes the following: - Compiler code updates for changes to the LLVM 4.0.0 and 5.0.0 API. - Updates to Type-Based Alias Analysis metadata format for LLVM 4.0.0 and up.

This change includes the following: - Compiler code updates for changes to the LLVM 4.0.0 and 5.0.0 API. - Updates to Type-Based Alias Analysis metadata format for LLVM 4.0.0 and up. Note that this code contains a workaround for a problem with hoisting loads of stack-allocated references (see ponylang#2303, ponylang#2061, ponylang#1592), and should be considered experimental. Do not use LLVM 4 or 5 for critical production builds.

SeanTAllen · 2017-12-21T00:49:11Z

Closing in favor of #2371.

Praetonus added enhancement: 1 - needs discussion labels Feb 16, 2017

rurban added a commit to rurban/ponyc that referenced this issue Mar 11, 2017

Add LLVM 4.0 support

6c04546

WIP: [ FAILED ] CodegenTest.JitRun [ FAILED ] CodegenTest.CCallback Closes ponylang#1592

fxcoudert mentioned this issue Mar 14, 2017

llvm 4.0.0 Homebrew/homebrew-core#11044

Closed

This was referenced Mar 15, 2017

LLVM 3.9? Homebrew/homebrew-core#11110

Closed

EXC_BAD_ACCESS crash in try_gc when under heavy load #1483

Closed

killerswan changed the title ~~Fails to build with LLVM 4.0.0-rc2~~ Add support for LLVM 4 Apr 2, 2017

jemc assigned chalcolith Apr 5, 2017

SeanTAllen added triggers release Major issue that when fixed, results in an "emergency" release and removed priority: 2 - medium labels Apr 5, 2017

jemc mentioned this issue Apr 7, 2017

Pony will not build on Fedora Rawhide #1821

Closed

jemc mentioned this issue Jul 15, 2017

Can't compile ponyc on Fedora 26 #2046

Closed

Praetonus assigned chalcolith and unassigned Praetonus Jul 21, 2017

Praetonus added enhancement: 4 - in progress and removed enhancement: 1 - needs discussion labels Jul 21, 2017

SeanTAllen mentioned this issue Jul 31, 2017

Can't link on Arch Linux, even with --pic #2115

Closed

chalcolith mentioned this issue Nov 24, 2017

Fix building with LLVM 5 and 6 so that the stack allocation workaround is no longer needed. #2371

Closed

SeanTAllen closed this as completed Dec 21, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for LLVM 4 #1592

Add support for LLVM 4 #1592

ilovezfs commented Feb 16, 2017

Praetonus commented Feb 16, 2017

ilovezfs commented Feb 16, 2017

Praetonus commented Feb 16, 2017

rurban commented Mar 11, 2017 •

edited

Loading

ilovezfs commented Mar 14, 2017

rurban commented Mar 14, 2017 •

edited

Loading

killerswan commented Mar 22, 2017

ilovezfs commented Mar 31, 2017

rurban commented Apr 2, 2017

killerswan commented Apr 2, 2017

SeanTAllen commented Apr 5, 2017

chalcolith commented Apr 7, 2017

Praetonus commented Apr 28, 2017

chalcolith commented Apr 28, 2017

chalcolith commented Apr 28, 2017

ilovezfs commented Sep 12, 2017

chalcolith commented Sep 12, 2017

SeanTAllen commented Dec 21, 2017

Add support for LLVM 4 #1592

Add support for LLVM 4 #1592

Comments

ilovezfs commented Feb 16, 2017

Praetonus commented Feb 16, 2017

ilovezfs commented Feb 16, 2017

Praetonus commented Feb 16, 2017

rurban commented Mar 11, 2017 • edited Loading

ilovezfs commented Mar 14, 2017

rurban commented Mar 14, 2017 • edited Loading

killerswan commented Mar 22, 2017

ilovezfs commented Mar 31, 2017

rurban commented Apr 2, 2017

killerswan commented Apr 2, 2017

SeanTAllen commented Apr 5, 2017

chalcolith commented Apr 7, 2017

Praetonus commented Apr 28, 2017

chalcolith commented Apr 28, 2017

chalcolith commented Apr 28, 2017

ilovezfs commented Sep 12, 2017

chalcolith commented Sep 12, 2017

SeanTAllen commented Dec 21, 2017

rurban commented Mar 11, 2017 •

edited

Loading

rurban commented Mar 14, 2017 •

edited

Loading