Skip to content

Conversation

@whitequark
Copy link
Member

See the commit message. cc @gasche

@whitequark
Copy link
Member Author

This PR will need an update once #159 is merged, I think.

@gasche
Copy link
Member

gasche commented Feb 26, 2017

I have bad gut feelings about the idea of removing the ability to use -ocamlc etc. to tweak how the plugin gets built. Currently what you want is expressible, albeit inconveniently, by using -plugin-option -ocamlc -plugin-option '...' (or -toolchain or whatever). After the proposed change, what you want will be easier to express, but what is currently the default will be impossible to express anymore. It is true that the options that are related to cross-compilation are not meaningful for the plugin, but there plenty of other things one may want to control there (I don't know, code coverage of plugin code, specific debug modes, tweaking the plugin build in way that are not expressible with -plugin-tag) that would be lost.

I thought of proposing also having -plugin-build-ocaml{c,opt,dep...} options to specify which commands to use at plugin-build-time, or having -plugin-build-option OPT as a way to specify that OPT should be "applied" during the plugin build (for certain categories of options that are, by default, only effectful when the plugin runs). I am not satisfied with either proposals (for the first, I am not sure how -use-ocamlfind is suppose to behave, and it is painful to have to change its implementation for this seemingly-orthogonal concept) (and the second is a hack that makes the system irregular and I don't see how to implement easily).

@gasche
Copy link
Member

gasche commented Feb 26, 2017

(Making plugin-option less painful to use, for example with a plugin-options that takes a string and splice it in the plugin-time arguments, could also make this less necessary? But in principle I agree that using default options for both plugin build and project build is sort of iffy.)

@whitequark
Copy link
Member Author

(I don't know, code coverage of plugin code, specific debug modes, tweaking the plugin build in way that are not expressible with -plugin-tag) that would be lost.

I don't believe anyone in the entire history of OCaml development has wanted the first two. "I don't know" gives a good hint. Therefore I do not wish to waste my time implementing this feature you suggest.

@gasche
Copy link
Member

gasche commented Feb 26, 2017

I have certainly seen people in the wild use -ocamlc 'ocamlc -linkall foo.cmo' as a way to hot-link behavior into their myocamlbuild.ml without changing its code -- not that I recommend this practice in any way, and I don't know if it was adopted in any actually released build system.

@whitequark
Copy link
Member Author

Sigh. Nevermind. I'll just instruct the users of opam-cross-* repositories to pin my fork of ocamlbuild instead.

@whitequark whitequark closed this Feb 26, 2017
@gasche
Copy link
Member

gasche commented Feb 26, 2017

So maybe users with specific plugin-build needs could actually run the more verbose two-step process where they first ocamlbuild -just-plugin <weird plugin build options>, and then ocamlbuild <options not intended for the plugin>. I wonder whether this would actually work.

@gasche
Copy link
Member

gasche commented Feb 26, 2017

Sigh. Nevermind. I'll just instruct the users of opam-cross-* repositories to pin my fork of ocamlbuild instead.

You may be tired or angry about something else, but pouting because I am thinking out loud about your feature before integrating it is not particularly nice. I want to do the right thing and I appreciate your work, but I also have to make sure that I understand the pros and cons and proposed changes -- many of the current silent users of ocamlbuild don't particularly ask for new features and just want the shit to keep working, which is in tension with the newly proposed features (as the .cmxs breakage precisely demonstrated).

@whitequark
Copy link
Member Author

many of the current silent users of ocamlbuild don't particularly ask for new features and just want the shit to keep working

I recall you've mentioned on many occasions that splitting ocamlbuild from the compiler was going to enable it to evolve faster. I don't see that happening. In fact over the last many months, every time I came back to the OCaml ecosystem, there was simultaneously more complexity and more brokenness in the build systems, and not that much interest in improving the situation.

But sure, let's try this again.

@whitequark whitequark reopened this Feb 27, 2017
@whitequark
Copy link
Member Author

So maybe users with specific plugin-build needs could actually run the more verbose two-step process where they first ocamlbuild -just-plugin , and then ocamlbuild . I wonder whether this would actually work.

I consider this unacceptable. This is broken in principle (sharing the options for building host and target software should never happen) and in practice (e.g. the users of topkg that override build_cmd i.e. the function that, in the default case, would have the -just-plugin/-no-plugin invocation, are unlikely to care about cross-compilation workloads themselves, and so any such override would needlessly break them). You are effectively proposing to take something that can and should be fixed in ocamlbuild, and force downstream users and/or anyone cross-compiling packages to work around that, often going through multiple layers of software (opam -> topkg -> ocamlbuild...) and maintaining patches indefinitely.

@whitequark
Copy link
Member Author

I thought of proposing also having -plugin-build-ocaml{c,opt,dep...} options to specify which commands to use at plugin-build-time

I think this is acceptable. I'm now implementing support for -plugin-ocamlc, -plugin-ocamlopt, -plugin-{use,no}-ocamlfind options. I don't think any others are necessary as they are never used during compilation of the plugin. Similarly, -cflag, -lflag, etc, are not used.

@whitequark
Copy link
Member Author

@gasche Please review the updated PR.

@gasche
Copy link
Member

gasche commented Feb 27, 2017

I recall you've mentioned on many occasions that splitting ocamlbuild from the compiler was going to enable it to evolve faster.

The reasoning, I think, was that having it an independent, less-scary project may remove some psychological barriers that people would have to contribute. (There were no strong practical barriers at the time, in the sense that it is not too hard today to get code integrated in the language implementation.) Of course, I could not tell whether this would actually work or not (I was never a big proponent of the move, it's just that I did not oppose it and I think the ability to have desynchronized releases, in principle, makes sense). But the people that told us that they would be willing to contribute in fact do not, I suppose that they don't actually have the time to do it, so we have exactly as much contributions now than before the split, that is not very much -- you would be the notable exception.

Note that if you were willing to take a more active role in the maintenance of ocamlbuild, I would of course be happy to let you. (I believe that you have full commit rights, so technically there is nothing more to be done.) I am not very good at being a maintainer. But if nobody steps up, well that's how it is.

And I think that our duty to the OCaml community is mostly to ensure that ocamlbuild keeps working for the people for which it works today, and of course let people that decide to build interesting things off it have some control over its improvement. One reason why I'm slow to work on ocamlbuild is that there is always something higher in my priority queue, even whithin the OCaml community: right now I have been working on making sure the future 4.05 release works well, and that, in fact, does not require any ocamlbuild-side change (actually, before we interacted this week-end I was thinking of porting ppx_deriving to use ocaml-migrate-parsetree; I did not decide otherwise, but just didn't have the time to do it).

@whitequark
Copy link
Member Author

whitequark commented Feb 27, 2017

Note that if you were willing to take a more active role in the maintenance of ocamlbuild, I would of course be happy to let you.

I did. Assuming you're watching the repo, you've got the notification about me adding Travis support, and you've seen my build system changes already. Once that lands there won't be another mtime fiasco, at least.

Incidentally what do you think about removing support for platforms without the Unix module? Does this still really matter for anybody? There's so much poorly exercised code lying around... The current buildsystem is not even able to build it.

actually, before we interacted this week-end I was thinking of porting ppx_deriving to use ocaml-migrate-parsetree; I did not decide otherwise, but just didn't have the time to do it

That would be warmly welcome.

@whitequark
Copy link
Member Author

whitequark commented Feb 27, 2017

This fix might interact with (solve?) #66.

A plugin is always built for the same platform ocamlbuild is.
It is therefore always a mistake to pass the -toolchain option
to ocamlfind when building a plugin. More generally, the compiler
that is used for building the plugin and the user code should be
possible to set separately.

This necessarily results in four new command-line options:
  * -plugin-ocaml{c,opt}, which are like -ocaml{c,opt} but only
    for building the plugin;
  * -plugin-{no,use}-ocamlfind, which are like -{no,use}-ocamlfind,
    but only for building the plugin.

To preserve compatibility, the old -{use,no}-ocamlfind options imply
the new -plugin-{use,no}-ocamlfind. It is still possible to have any
combination of using ocamlfind for build or host code by ordering
the -plugin-{use,no}-ocamlfind options after -{use,no}-ocamlfind
ones.

Note that the -cflag, -lflag, etc., options were never used
when building the plugin and are therefore not touched by this
commit.
@gasche
Copy link
Member

gasche commented Feb 28, 2017

The new patch lifted my concerns, thanks.

I am still a bit unsure about the -plugin-ocamlc naming scheme. As a user, how do I know whether this means "the ocamlc command used when the plugin is run" or "when the plugin is built"? This is why I suggested -plugin-build-ocamlc to avoid the ambiguity. Sure, we have -plugin-option and -plugin-tag and they use the two different meanings, but maybe we should avoid increasing the confusion? Any second opinion?

@gasche gasche mentioned this pull request Feb 28, 2017
@whitequark
Copy link
Member Author

whitequark commented Feb 28, 2017

As a user, how do I know whether this means "the ocamlc command used when the plugin is run" or "when the plugin is built"?

By reading the --help text (or the manpage... which should be probably updated). It also seems very counterintuitive to use one set of options when we are building with the plugin, as opposed to building the plugin, so I'm not sure how common would this interpretation be. It took me a few moments to understand the semantics of your other option.

@gasche gasche merged commit b30b31f into ocaml:master Feb 28, 2017
@gasche
Copy link
Member

gasche commented Feb 28, 2017

You seem to hold your position strong :-) I merged.

@whitequark whitequark deleted the no-toolchain-for-plugin branch February 28, 2017 22:03
@whitequark
Copy link
Member Author

According to my testing there seems to be a bug in the -toolchain option handling, when used in the wild. Investigating.

@whitequark
Copy link
Member Author

Oh, sorry, nevermind. I forgot to update my local master branch after the merge...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants