Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake: Enable multi-processor builds for MSVC toolchain #70

Closed
wants to merge 1 commit into from

Conversation

rcdailey
Copy link
Contributor

@rcdailey rcdailey commented Dec 21, 2018

When MSVC is detected, add the /MP option to the compile flags for
all targets. This enables multi-processor (multi-core) builds per
translation unit being compiled.

Fixes #69

When MSVC is detected, add the `/MP` option to the compile flags for
all targets. This enables multi-processor (multi-core) builds per
translation unit being compiled.

Fixes Orphis#69

Signed-off-by: Robert Dailey <rcdailey@gmail.com>
@Orphis
Copy link
Owner

Orphis commented Dec 21, 2018

I'm not sure about this. If a project wants it, they can provide it in their toolchain file.
Also, the test in this change is wrong, it should only target Visual Studio generators, not MSVC itself (for example, it won't help with Ninja builds).

@rcdailey
Copy link
Contributor Author

First: MSVC is the compiler, and this is an option for that compiler. Doesn't matter what the makefile generator is, it will still work. To be fair, I am not aware of any official support for Ninja + MSVC. I've personally never gotten it to work, nor have I seen it working. Not saying it's not possible or not important, but Ninja should have nothing to do with the /MP option.

Second, this flag is important when folks build this project outside of their own. I personally use your project to build without doing add_subdirectory(). I realize that's the main use case you've designed for, but that's not a limitation. And in those scenarios, if this flag is not defined by your own CMake scripts, there's no other mechanism to turn it on.

So based on all of this, I personally still find this change acceptable.

@rcdailey
Copy link
Contributor Author

I should also add that the -j option for Ninja may compete with the compiler-native parallelization that MSVC provides. Ninja was obviously designed around the way GCC and Clang work, which do not implement their own parallelization features. So really this is a problem outside the scope of this issue, I think.

@Orphis
Copy link
Owner

Orphis commented Dec 22, 2018

The Visual Studio generators are the only ones passing multiple files to compile to the compiler.
Ninja + MSVC is supported, documented elsewhere and works great. It's even the default choice now for the CMake support in Visual Studio. Check the CI for the project for some example!

The parallelisation is done at the build system instead of the compiler (since it will cause lots of issues) with Ninja, Make or any other. You could make the check for MSVC and Visual Studio generator.

Admittedly, if you integrate the project, you should already have the flag in your toolchain file. Also, Boost is so small, if you already have project based parallelisation too, it should have minimal impact too.

@rcdailey
Copy link
Contributor Author

I'm not sure why /MP is a compiler option and not an option for msbuild itself. That to me would make more sense. You're likely right about the VS generator semantics, but I haven't found any documentation on that nor do I have any personal observations to prove it. But you're right, the intent is for this to be for Visual Studio only. If Ninja's -j option already works for MSVC, then we don't need /MP. I'll adjust the condition to handle this case.

@Orphis
Copy link
Owner

Orphis commented Dec 22, 2018

msbuild has a /M option to build projects in parallel. The problem is that if you don't have global parallelisation, so it is possible to have N (cpucount) msbuild instances building projects in parallel and then /MP for each cl process compiling N compilation units.
You end up with N^2 processes. It's bad.

That's why usually, Ninja builds are globally faster than msbuild (with tuned /M and /MP options for the right cpucount) despite spawning new processes being slower on Windows: you just have the right amount of parallel processing.

@Orphis
Copy link
Owner

Orphis commented Jun 12, 2019

Closing as this should come from a toolchain file as I said before. Ninja is the preferred build tool for MSVC now (it's the default) so this shouldn't be needed most of the time.

@Orphis Orphis closed this Jun 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable /MP for parallel builds with MSVC
2 participants