Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows bootstrap #3155

Merged
merged 10 commits into from
Jan 12, 2018
Merged

Windows bootstrap #3155

merged 10 commits into from
Jan 12, 2018

Conversation

dra27
Copy link
Member

@dra27 dra27 commented Jan 3, 2018

This pull request primarily adds support for Windows to the opam bootstrap scripts and uses these to extend AppVeyor testing to all six Windows ports. The Cygwin ports are switched to use the bootstrap compiler rather than the Cygwin-installed ocaml package.

There are a few minor tweaks included on the way:

  • Since beta15, jbuild-ignore files can no longer contain subdirectories (see Display a warning for bad jbuild-ignore lines dune#389). As it happens, generating src_ext/jbuild-ignore is helpful for another pull request due to land...
  • I think that when I originally changed MCCS_DISABLED to MCCS_ENABLED there was a definite need for it, but either way - "if not disabled then do stuff" conditions do read oddly in the Makefile versus "if enabled then do stuff", so I've left it!
  • There's a small tweak to shell/check_linker (for MSVC building) as I discovered during AppVeyor testing that you get some irritating stderr junk from Git's link command.

The major stuff...

Automatic detection of Microsoft Visual Studio

This utilises my MSVS Tools package and means that you don't need to start Cygwin from within a Visual Studio Command Prompt to build an MSVC OCaml. The motivation for this becomes apparent in the full Windows port, since opam needs both a 32-bit and a 64-bit compiler and sadly no one ever told Microsoft about compiler triples...

make cold updates

Windows support is added for make cold - it's necessary to be able to specify which port you want built, which is either inferred automatically or specified using the OCAML_PORT variable passed to make. The script becomes quite a bit longer since it also supports automatic detection of the Microsoft C compiler, and the Windows ports of OCaml lack a configure script and need FlexDLL to be extracted.

The configure script is enhanced so that if a bootstrap compiler has been built then configure always uses it. This means you can develop opam using a bootstrap compiler, which has been quite handy for doing the Windows ports.

The final tweak to the make cold process is that it also supports being run "offline" which meant moving the location of the OCaml and FlexDLL tarballs to src_ext/Makefile

AppVeyor

I've transferred over some of the scripts I did for OCaml's AppVeyor runs which reduces the time spent in Cygwin setup. Various notes on this:

  • All six ports are built, but the limited test for opam init and installation of findlib is only done on Cygwin, since it won't work on native Windows without more patches (which will follow very, very soon...)
  • make cold (or, rather, the new make compiler) is used to get OCaml (for uniformity, this is done for the Cygwin ports, rather than using Cygwin-packaged OCaml). The bootstrap folder (and also the src_ext archives) are cached, so each build only takes a couple of minutes, unless the OCaml version has been bumped.

Compilation on native Windows is dependent on three external patches which are at present applied using appveyor.patch:

@dra27 dra27 force-pushed the windows-bootstrap branch from 212da1b to c32255a Compare January 3, 2018 14:55
@AltGr
Copy link
Member

AltGr commented Jan 9, 2018

Very nice, thanks! Do you think we should merge as-is, or wait for the releases of mccs and "dune" (and possibly FlexDLL) ?

@dra27
Copy link
Member Author

dra27 commented Jan 10, 2018

The 4.06 changes should go in first and I'll then rebase this for 4.06 (I've already tested it with 4.06). I would say this can then be merged - the jbuilder and ocaml-mccs changes are just awaiting releases and the flexdll change only affects one set of Windows ports, not all three sets, so it's all usable work even without the patch.

dra27 added 8 commits January 11, 2018 10:33
Since beta15, jbuilder no longer permits subdirectories in jbuild-ignore
files so the sub-directory ones are split off to separate files.

Signed-off-by: David Allsopp <david.allsopp@metastack.com>
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
Causing innocuous error message on AppVeyor (conflict with Git's link
command)
Allow configure to detect cl from a Windows SDK or Visual Studio
installation, even if cl.exe isn't in the PATH.

Simplifies the launching of bash for multiple compiler architectures as
PATH doesn't require manual editing.
Updates the bootstrap script to build any of the four Windows ports. On
non-Windows OSes, no change in behaviour. On Windows, the variable
OCAML_PORT will control which native Windows port is built - mingw64,
mingw, msvc64 and msvc. If Cygwin's gcc is installed (not recommended
when building for native Windows) then OCAML_PORT must be specified, as
make cold will build the Cygwin version of OPAM. OCAML_PORT=auto causes
the script to auto-detect available compilers in the order given above.

Bootstrap compiler also bumped to 4.04.2
Split the cold Makefile target into compiler and cold. Allows a compiler
to be bootstrapped before running configure. configure updated so that
if the boot strap has been built, PATH is altered to use it. The change
is persisted in Makefile.config, along with OCAMLLIB.
Archives downloaded by shell/bootstrap-ocaml.sh may be placed in
src_ext/archives and will be cp'd if they exist (as for make lib-ext).
@dra27 dra27 force-pushed the windows-bootstrap branch from c32255a to b62a7d5 Compare January 11, 2018 10:46
@dra27
Copy link
Member Author

dra27 commented Jan 11, 2018

Hopefully correctly rebased for 4.06 - let's see what AppVeyor thinks!

@dra27 dra27 force-pushed the windows-bootstrap branch from b62a7d5 to 91b10b2 Compare January 11, 2018 11:09
dra27 added 2 commits January 11, 2018 11:10
Signed-off-by: David Allsopp <david.allsopp@metastack.com>
 - Need unreleased ocaml-mccs 1.1+5 to build mccs on Windows.
 - Need unreleased jbuilder beta17 to build mccs on MSVC.
 - Need unreleased flexdll 0.38 to build mccs on mingw.

Signed-off-by: David Allsopp <david.allsopp@metastack.com>
@dra27 dra27 force-pushed the windows-bootstrap branch from 91b10b2 to 2a54a90 Compare January 11, 2018 11:11
@AltGr AltGr merged commit 268f073 into ocaml:master Jan 12, 2018
@XVilka XVilka mentioned this pull request Jan 15, 2018
@dra27 dra27 deleted the windows-bootstrap branch April 4, 2018 15:29
@dra27 dra27 mentioned this pull request Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants