Unresolved differences between gdc/dmd front ends #2194

ibuclaw · 2013-06-17T10:44:54Z

I recall mentioning this in one of the DDMD pulls. FYI @yebblies

This covers all current unresolved differences between gdc and dmd front ends (and acts as a TODO list). An internal goal is to have them 100% shared code bases by release 2.065. This makes a transition from the D front-end implement from C++ to D more of a 'drop in' (and is arguably a blocker and hindrance for adoption of the new implementation), and also relieves a HUGE maintenance burden when it comes round to merge-time. However such a goal requires a bit of tug and pull from both sides of the fence so I'm raising this here to allow discussion with any core devs to iron these out in a way that we can say good riddance to the IN_GCC macro.

The diff can be summarised into the following points:

Internal backend types differ
Differing _argptr and _arguments front end implementation.
Always use Itanium ABI for C++ mangling
Support for GDC Extended Assembly in parser.
Rejected/not yet pulled fixes for things done in the front end that send invalid/garbage code to the glue part of the front-end.

@yebblies - or anyone else - if you have any questions or suggestions, anything is welcome. Also let me know if you'd rather this broken down... :o)

Will be sending separate pulls for things that can be folded in immediately...

Thanks
Iain.

yebblies · 2013-06-17T14:11:45Z

Hmm...

Internal backend types differ

This is ugly, but that is more due to dmd's design than anything else.

Use of fprintf(stderr) instead of printf

Are people relying on the -v output coming over stdout? I seem to remember something about this, for rdmd or something...

Built-in functions are instead CTFE'd in glue part of front-end.

They... what? How does that work?

real_t requires special handling as is not host float.

Can we cram all of the affected code into Port somehow?

PowExp does not depend on std.math.

Hmm.

Differing _argptr, _arguments and va_list front end implementation.

shudder

Always use cppmangle.c for C++ mangling

Will this be fixed by the new mangling stuff?

Support for GDC Extended Assembly in parser.

Urrgh, I guess we don't have much choice here.

mars.c is only used for verror and friends (front end entry point is in glue)

I think we should move these into another file, along with everything else you need from mars.c (global and Loc?)

ibuclaw · 2013-06-17T14:32:16Z

Use of fprintf(stderr) instead of printf

Are people relying on the -v output coming over stdout? I seem to remember something about this, for rdmd or something...

Yes, I think rdmd might depend on it. I think rdmd specifically doesn't work with gdc though... that or they've worked around it without telling me. The rationale behind sending all output to stderr is that the gcc backend already uses stdout to pipe assembly to as.

Built-in functions are instead CTFE'd in glue part of front-end.

They... what? How does that work?

There are approximately 580 built-ins that may be subject to being const-folded - without being restricted by host native precision - by the gcc backend. As CTFE has guaranteed that all parameters are known at compile time, I build the call in GCC's IR, and if it gets folded into a constant literal, a new D frontend expression is generated and returned back to CTFE.

real_t requires special handling as is not host float.

Can we cram all of the affected code into Port somehow?

Yes, that is my plan.

PowExp does not depend on std.math.

Hmm.

foo ^^ bar -> __builtin_pow{fl} (foo, bar)

Always use cppmangle.c for C++ mangling

Will this be fixed by the new mangling stuff?

Once I'm convinced that it has done properly, yes removing the CPP_MANGLE macro will sort it just fine.

Support for GDC Extended Assembly in parser.

Urrgh, I guess we don't have much choice here.

This can be the last thing to worry about...

mars.c is only used for verror and friends (front end entry point is in glue)

I think we should move these into another file, along with everything else you need from mars.c (global and Loc?)

Yes please. Although I thought you objected to this last time I mentioned? :o)

yebblies · 2013-06-17T14:39:35Z

Yes, I think rdmd might depend on it. I think rdmd specifically doesn't work with gdc though... that or they've worked around it without telling me. The rationale behind sending all output to stderr is that the gcc backend already uses stdout to pipe assembly to as.

Sigh, this probably means we need to bring back stdmsg.

Built-in functions are instead CTFE'd in glue part of front-end.

They... what? How does that work?

There are approximately 580 built-ins that may be subject to being const-folded - without being restricted by host native precision - by the gcc backend. As CTFE has guaranteed that all parameters are known at compile time, I build the call in GCC's IR, and if it gets folded into a constant literal, a new D frontend expression is generated and returned back to CTFE.

Yikes, that's a little scary. I guess there's no more direct way invoke the builtin folders?

PowExp does not depend on std.math.

Hmm.

foo ^^ bar -> __builtin_pow{fl} (foo, bar)

Yeah, but this will need druntime support before dmd can do it properly.

mars.c is only used for verror and friends (front end entry point is in glue)

I think we should move these into another file, along with everything else you need from mars.c (global and Loc?)

Yes please. Although I thought you objected to this last time I mentioned? :o)

I don't think so... or at least, if I did I was wrong. Not sure where to put them exactly...

ibuclaw · 2013-06-17T14:52:32Z

Yes, I think rdmd might depend on it. I think rdmd specifically doesn't work with gdc though... that or they've worked around it without telling me. The rationale behind sending all output to stderr is that the gcc backend already uses stdout to pipe assembly to as.

Sigh, this probably means we need to bring back stdmsg.

Need not have it defined globally though, eg: add a new function message(...) and vmessage(...) to use instead of printf().

Built-in functions are instead CTFE'd in glue part of front-end.

They... what? How does that work?

There are approximately 580 built-ins that may be subject to being const-folded - without being restricted by host native precision - by the gcc backend. As CTFE has guaranteed that all parameters are known at compile time, I build the call in GCC's IR, and if it gets folded into a constant literal, a new D frontend expression is generated and returned back to CTFE.

Yikes, that's a little scary. I guess there's no more direct way invoke the builtin folders?

Currently it's done just by implementing our own d_gcc_eval_builtin routine - renamed because it takes slightly different arguments than eval_builtin (a FuncDeclaration instead of an enum BUILTIN)

mars.c is only used for verror and friends (front end entry point is in glue)

I think we should move these into another file, along with everything else you need from mars.c (global and Loc?)

Yes please. Although I thought you objected to this last time I mentioned? :o)

I don't think so... or at least, if I did I was wrong. Not sure where to put them exactly...

Might have been in reference to me mentioning that I was going to drop mars.c from gdc's copy of the D front end.

ghost · 2013-06-17T15:42:36Z

W.r.t. erorring lets just be careful not to re-introduce this regression: http://d.puremagic.com/issues/show_bug.cgi?id=10050

yebblies · 2013-06-19T02:54:21Z

Need not have it defined globally though, eg: add a new function message(...) and vmessage(...) to use instead of printf().

This is much better.

Currently it's done just by implementing our own d_gcc_eval_builtin routine - renamed because it takes slightly different arguments than eval_builtin (a FuncDeclaration instead of an enum BUILTIN)

Interesting. Does DMD really need the BUILTIN enum, or could it use the FuncDeclaration approach too?

Might have been in reference to me mentioning that I was going to drop mars.c from gdc's copy of the D front end.

I actually thought you'd done this already.

ibuclaw · 2013-06-19T11:29:55Z

Currently it's done just by implementing our own d_gcc_eval_builtin routine - renamed because it takes slightly different arguments than eval_builtin (a FuncDeclaration instead of an enum BUILTIN)

Interesting. Does DMD really need the BUILTIN enum, or could it use the FuncDeclaration approach too?

Shouldn't make a difference if dmd was to switch... all that you'd use it for is to switch (fd->builtin) - though I think a more pluggable way of adding built-in functions to the front end might be in order...

Might have been in reference to me mentioning that I was going to drop mars.c from gdc's copy of the D front end.

I actually thought you'd done this already.

Not yet... :o)

ibuclaw · 2013-07-18T13:15:56Z

Rebased to most recent head. Round about half of the initial list of differences raised in this pull have now been resolved.

yebblies · 2013-07-18T14:07:14Z

Half way!

ibuclaw · 2014-01-30T19:51:42Z

Gone from - 25 changed files with 387 additions and 70 deletions.
To - 23 files changed, 353 insertions(+), 59 deletions(-)

At the very least, builtin.c, mars.c, and root/async.c have been removed from gdc's frontend which have in part accounted for the small drop over the last 7 months.

ibuclaw · 2014-01-30T19:55:54Z

On the bright side, conflicts of interest between implementations are small, and most of the differences are now gdc ehancements.

ibuclaw · 2014-05-26T21:15:16Z

@yebblies - btw how much of this do you feel has been sorted out in DMD-HEAD?

yebblies · 2014-05-27T00:05:46Z

The diff looks up-to-date. The iasm and mem stuff should probably go into the frontend unconditionally, and the paint_float_int stuff should be in port. I have a patch somewhere to remove the verror workaround.

ibuclaw · 2014-05-27T01:51:56Z

The mem changes are only needed to maintain the c++ frontend. The glue doesn't allocate through Mem directly itself.

yebblies · 2014-05-27T11:21:14Z

The mem changes are only needed to maintain the c++ frontend. The glue doesn't allocate through Mem directly itself.

Ok, but for now we support the C++ frontend so it needs to be fixed.

andralex · 2015-01-27T05:51:48Z

Well so should we close this for now?

ibuclaw · 2015-01-27T08:32:52Z

If only to stop the auto-tester trying to build it.

The idea here is that there should be as close to zero changes as possible. And I would hope that a common shared frontend is a prerequesite for ddmd.

ibuclaw · 2015-01-27T08:34:37Z

I will need to update this anyway for 2.066/2.067 - doing so with also remind me to raise new PRs for other things (and cross items off the list in the OP)

Orvid · 2015-01-27T08:41:52Z

src/declaration.h

@@ -580,10 +580,10 @@ class FuncDeclaration : public Declaration
                                        // scopes from having the same name
    VarDeclaration *vthis;              // 'this' parameter (member and nested)
    VarDeclaration *v_arguments;        // '_arguments' parameter
-#ifdef IN_GCC
+#ifdef IN_GCC // %%


Any particular reason for this change? This same change made it's way in in multiple other places as well.

To make them show up in the diff.

Yeah - upstream need to be aware of any IN_GCC macros - whether or not we can remove these in future is up for discussion.

Btw, how do you deal with _argptr, if you don't declare it in the frontend?

@yebblies - We use v_argptr to stash the _argptr var, then generate the code in the backend. (va_start and va_end are gcc builtins).

Oh right, I misread this block as a #ifndef IN_GCC block. I'm considering doing the same in dmd.

yebblies · 2015-01-27T10:34:29Z

I wouldn't worry about the autotester, it will only run this after all the more recently updated ones and will fail very quickly.

ibuclaw · 2015-02-09T16:45:34Z

@yebblies @andralex - By the way, I intend on releasing a GDC stubs repository sometime this week, which takes the current dmd-master sources, then provides the minimum required stubs in order to tie the frontend to GCC. So this will include parsing and semantic processing, but no codegen.

This is for a few reasons:

Trialling DDMD
Catch problems in the front-end before 2.067 release.
Allows a possible clean-room rewrite of GDC in a manner that takes advantage of GCC's new coding style for C++, and lowers the entrypoint for potential contributors.

yebblies · 2015-02-09T18:32:06Z

I like the sound of that!

yebblies · 2015-03-10T05:36:38Z

A lot of these differences seem to be fixed already. I'd be very interested to see a current diff.

I've opened #4472 to look at merging in the ExtAsmStatement code.

ibuclaw · 2015-04-12T10:36:35Z

@yebblies rebased: (edit: rebased again)
Was 23 files changed 353 insertions(+) 59 deletions(-)
Now 19 files changed, 360 insertions(+), 56 deletions(-)

Some progress, hidden by the fact that parseExtAsm has been re-implemented / grown in size.

yebblies · 2015-04-12T11:35:39Z

What do you think about the approach in #4569 for getting rid of the backend type references?

ibuclaw · 2017-09-27T23:11:19Z

Granted that most comments here are just me saying "bump, bump, bump..." - I certainly wouldn't miss them. :-)

dlang-bot · 2017-09-27T23:58:23Z