Improve MSVC and clang modules compilation #2664

Arthapz · 2022-08-09T10:03:43Z

#2641
#2623

waruqi · 2022-08-09T14:22:51Z

Now tests/dependence work for clang, but tests/user_headerunits and tests/cpp_with_modules are broken for msvc.

Arthapz · 2022-08-09T14:23:24Z

yeah i'm working on it

xmake/rules/c++/modules/modules_support/common.lua

xmake/rules/c++/modules/modules_support/msvc.lua

xmake/rules/c++/modules/modules_support/clang.lua

xmake/rules/c++/modules/modules_support/msvc.lua

Arthapz · 2022-08-09T14:53:29Z

i'm trying to have only needed flags when building a .cpp, is there a way to add cxxflags to a specific cpp file from build_modules_for_* ?

waruqi · 2022-08-09T15:10:37Z

i'm trying to have only needed flags when building a .cpp, is there a way to add cxxflags to a specific cpp file from build_modules_for_* ?

no way now, we can only specify them when adding files. add_files("src/xxx.cc", {cxxflags = "x"})

waruqi · 2022-08-09T15:30:01Z

i'm trying to have only needed flags when building a .cpp, is there a way to add cxxflags to a specific cpp file from build_modules_for_* ?

I added it on cxxmodule branch. you can rebase it and try target:fileconfig_add

target("test2")
    set_kind("binary")
    add_files("src/*.cpp", {cxxflags = "-DTEST"})
    before_build(function (target)
        target:fileconfig_add("src/main.cpp", {cxxflags = {"-DTEST2", "-DTEST3"}, defines = "TEST4"})
    end)

... -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DTEST -DTEST2 -DTEST3 -DTEST4

Arthapz · 2022-08-09T15:35:26Z

nice thanks

Arthapz · 2022-08-10T14:54:51Z

i need to do the same for clang and gcc now

waruqi · 2022-08-10T15:22:27Z

i need to do the same for clang and gcc now

gcc uses mapper files, and I don't think it would be a problem even if it's added globally to cxxflags. it works fine for gcc now.

waruqi · 2022-08-10T15:28:27Z

I tested new patches for msvc, it works fine for more tests, but user_headerunits will still be broken for msvc.

xmake/rules/c++/modules/modules_support/msvc.lua

waruqi · 2022-08-11T13:08:41Z

Does it work now? I try it, but it's still broken for user_headerunits/msvc.

Z:\personal\xmake\tests\projects\c++\modules\user_headerunit>xmake clean -a && x
make f -c && xmake -r
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.32.31332
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
[ 15%]: generating.cxx.headerunit.bmi header.hpp
[ 53%]: generating.cxx.module.bmi hello
[ 76%]: ccache compiling.release src\main.cpp
error: main.cpp
src\main.cpp(2): error C7612: could not find header unit for 'Z:\personal\xmake\
tests\projects\c++\modules\user_headerunit\src\header.hpp'
src\main.cpp(5): error C2039: 'FOO': is not a member of 'hello'
Z:\personal\xmake\tests\projects\c++\modules\user_headerunit\src\hello.mpp(6): n
ote: see declaration of 'hello'
src\main.cpp(5): error C2065: 'FOO': undeclared identifier

  > in src\main.cpp

Arthapz · 2022-08-11T16:21:57Z

Weird, all the tests work on my machine.

https://asciinema.org/a/0At3VCdO9rQxjcdqaqcMlfFVM

Can you show me the command line of src\main.cpp ?

waruqi · 2022-08-12T00:44:58Z

Weird, all the tests work on my machine.

https://asciinema.org/a/0At3VCdO9rQxjcdqaqcMlfFVM

Can you show me the command line of src\main.cpp ?

checking for flags (cl_scan_dependencies) ... ok
[  0%]: generating.cxx.module.deps src\main.cpp
checking for flags (-std:c++20) ... ok
checking for flags (-experimental:module) ... ok
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++20 -experimental:module -TP -scanDependencies build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\src\main.cpp.json src\main.cpp -Fobuild\.objs\user_headerunit\windows\x64\release\src\main.cpp.obj
[  0%]: generating.cxx.module.deps src\hello.mpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++20 -experimental:module -TP -scanDependencies build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\src\hello.mpp.json src\hello.mpp -Fobuild\.objs\user_headerunit\windows\x64\release\src\hello.mpp.obj
checking for flags (cl_ifc_output) ... ok
checking for flags (cl_interface) ... ok
checking for flags (cl_reference) ... ok
checking for flags (cl_export_header) ... ok
checking for flags (cl_header_unit_quote) ... ok
checking for flags (cl_header_unit_angle) ... ok
checking for flags (cl_header_name_quote) ... ok
checking for flags (cl_header_name_angle) ... ok
[ 15%]: generating.cxx.headerunit.bmi header.hpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++20 -experimental:module -TP -exportHeader -c -headerName:quote Z:\personal\xmake\tests\projects\c++\modules\user_headerunit\src\header.hpp -ifcOutput build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\src /Fobuild\.objs\user_headerunit\windows\x64\release\src\header.hpp.obj
header.hpp
[ 53%]: generating.cxx.module.bmi hello
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++20 -experimental:module -TP -c -Fobuild\.objs\user_headerunit\windows\x64\release\src\hello.mpp.obj -interface -ifcOutput build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\hello.ifc src\hello.mpp
hello.mpp
[ 76%]: ccache compiling.release src\main.cpp
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.32.31326\\bin\\HostX64\\x64\\cl.exe" -c /EHsc -nologo -std:c++20 -experimental:module -headerUnit:quote header.hpp=build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\src\header.hpp.ifc -reference hello=build\.gens\user_headerunit\windows\x64\release\rules\modules\cache\hello.ifc -Fobuild\.objs\user_headerunit\windows\x64\release\src\main.cpp.obj src\main.cpp
checking for flags (cl_sourceDependencies) ... ok
error: main.cpp
src\main.cpp(2): error C7612: could not find header unit for 'Z:\personal\xmake\tests\projects\c++\modules\user_headerunit\src\header.hpp'
src\main.cpp(5): error C2039: 'FOO': is not a member of 'hello'
Z:\personal\xmake\tests\projects\c++\modules\user_headerunit\src\hello.mpp(6): note: see declaration of 'hello'
src\main.cpp(5): error C2065: 'FOO': undeclared identifier

waruqi · 2022-08-12T00:46:37Z

Weird, all the tests work on my machine.

Can you remove build and .xmake first and try it again? or run xmake clean -a

waruqi · 2022-08-12T00:49:28Z

And tests/class example is also broken on ci (msvc), but it works for me. wired.

>> testing tests\projects\c++\modules\class ...
>>     running test_main ...
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2019
checking for Microsoft C/C++ Compiler (x64) version ... 19.29.30146
[  0%]: generating.cxx.module.deps src\hello_impl.cpp
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
[ 41%]: generating.cxx.module.bmi hello
[ 66%]: ccache compiling.release src\hello_impl.cpp
[ 66%]: ccache compiling.release src\main.cpp
error: hello_impl.cpp
src\hello_impl.cpp(4): error C2230: could not find module 'hello'
src\hello_impl.cpp(8): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(8): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
src\hello_impl.cpp(8): error C2550: 'hello::say': constructor initializer lists are only allowed on constructor definitions
src\hello_impl.cpp(10): warning C4508: 'hello::say': function should return a value; 'void' return type assumed
src\hello_impl.cpp(11): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(12): error C2065: 'data_': undeclared identifier

ohmm, I tested it on my vs2019, it will be broken for vs2019.

and tests/dependence example has same issue.

Z:\personal\xmake\tests\projects\c++\modules\class>xmake clean -a && xmake f -c && xmake -rv -j1
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2019
checking for Microsoft C/C++ Compiler (x64) version ... 19.28.29334
checking for flags (cl_scan_dependencies) ... no
[  0%]: generating.cxx.module.deps src\hello_impl.cpp
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
checking for flags (cl_ifc_output) ... ok
checking for flags (cl_interface) ... ok
checking for flags (cl_reference) ... ok
[ 41%]: generating.cxx.module.bmi hello
checking for flags (-std:c++20) ... no
checking for flags (-std:c++latest) ... ok
checking for flags (-experimental:module) ... ok
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++latest -experimental:module -TP -c -Fobuild\.objs\class\windows\x64\release\src\hello.mpp.obj -interface -ifcOutput build\.gens\class\windows\x64\release\rules\modules\cache\hello.ifc src\hello.mpp
hello.mpp
[ 66%]: ccache compiling.release src\hello_impl.cpp
"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX64\\x64\\cl.exe" -c /EHsc -nologo -std:c++latest -experimental:module -Fobuild\.objs\class\windows\x64\release\src\hello_impl.cpp.obj src\hello_impl.cpp
checking for flags (cl_sourceDependencies) ... ok
error: hello_impl.cpp
src\hello_impl.cpp(4): error C2230: could not find module 'hello'
src\hello_impl.cpp(8): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(8): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
src\hello_impl.cpp(8): error C2550: 'hello::say': constructor initializer lists are only allowed on constructor definitions
src\hello_impl.cpp(10): warning C4508: 'hello::say': function should return a value; 'void' return type assumed
src\hello_impl.cpp(11): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(12): error C2065: 'data_': undeclared identifier

waruqi · 2022-08-12T14:10:07Z

Weird, all the tests work on my machine.

https://asciinema.org/a/0At3VCdO9rQxjcdqaqcMlfFVM

Can you show me the command line of src\main.cpp ?

The user_headerunits problem is probably related to the virtualbox shared disk partition, I copied it to my local c drive and it works fine.

waruqi · 2022-08-12T14:10:39Z

But the problem with impl_unit/class in vs2019 is still there

waruqi · 2022-08-12T14:24:01Z

And tests/class example is also broken on ci (msvc), but it works for me. wired.

>> testing tests\projects\c++\modules\class ...
>>     running test_main ...
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2019
checking for Microsoft C/C++ Compiler (x64) version ... 19.29.30146
[  0%]: generating.cxx.module.deps src\hello_impl.cpp
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
[ 41%]: generating.cxx.module.bmi hello
[ 66%]: ccache compiling.release src\hello_impl.cpp
[ 66%]: ccache compiling.release src\main.cpp
error: hello_impl.cpp
src\hello_impl.cpp(4): error C2230: could not find module 'hello'
src\hello_impl.cpp(8): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(8): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
src\hello_impl.cpp(8): error C2550: 'hello::say': constructor initializer lists are only allowed on constructor definitions
src\hello_impl.cpp(10): warning C4508: 'hello::say': function should return a value; 'void' return type assumed
src\hello_impl.cpp(11): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(12): error C2065: 'data_': undeclared identifier

ohmm, I tested it on my vs2019, it will be broken for vs2019.

and tests/dependence example has same issue.

Z:\personal\xmake\tests\projects\c++\modules\class>xmake clean -a && xmake f -c && xmake -rv -j1
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2019
checking for Microsoft C/C++ Compiler (x64) version ... 19.28.29334
checking for flags (cl_scan_dependencies) ... no
[  0%]: generating.cxx.module.deps src\hello_impl.cpp
[  0%]: generating.cxx.module.deps src\main.cpp
[  0%]: generating.cxx.module.deps src\hello.mpp
checking for flags (cl_ifc_output) ... ok
checking for flags (cl_interface) ... ok
checking for flags (cl_reference) ... ok
[ 41%]: generating.cxx.module.bmi hello
checking for flags (-std:c++20) ... no
checking for flags (-std:c++latest) ... ok
checking for flags (-experimental:module) ... ok
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.28.29333\bin\HostX64\x64\cl.exe /EHsc -nologo -std:c++latest -experimental:module -TP -c -Fobuild\.objs\class\windows\x64\release\src\hello.mpp.obj -interface -ifcOutput build\.gens\class\windows\x64\release\rules\modules\cache\hello.ifc src\hello.mpp
hello.mpp
[ 66%]: ccache compiling.release src\hello_impl.cpp
"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX64\\x64\\cl.exe" -c /EHsc -nologo -std:c++latest -experimental:module -Fobuild\.objs\class\windows\x64\release\src\hello_impl.cpp.obj src\hello_impl.cpp
checking for flags (cl_sourceDependencies) ... ok
error: hello_impl.cpp
src\hello_impl.cpp(4): error C2230: could not find module 'hello'
src\hello_impl.cpp(8): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(8): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
src\hello_impl.cpp(8): error C2550: 'hello::say': constructor initializer lists are only allowed on constructor definitions
src\hello_impl.cpp(10): warning C4508: 'hello::say': function should return a value; 'void' return type assumed
src\hello_impl.cpp(11): error C2653: 'say': is not a class or namespace name
src\hello_impl.cpp(12): error C2065: 'data_': undeclared identifier

About vs2019, because of missing /scandependence flag, fallback_generate_dependencies() call generate all requires for bmi provide.

hello_impl.cpp has not been added -reference hello flags

add referenceflag -reference hello build\.gens\impl_unit\windows\x64\release\rules\modules\cache\hello.ifc
get_requiresflags src\main.cpp {
  {
    "-reference",
    "hello=build\.gens\impl_unit\windows\x64\release\rules\modules\cache\hello.ifc"
  }
}

[ 66%]: ccache compiling.release src\hello_impl.cpp
"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX64\\x64\\cl.exe" -c /EHsc -nologo -std:c++latest -experimental:module -Fobuild\.objs\impl_unit\windows\x64\release\src\hello_impl.cpp.obj src\hello_impl.cpp
checking for flags (cl_sourceDependencies) ... ok
error: hello_impl.cpp
src\hello_impl.cpp(4): error C2230: could not find module 'hello'

only main.cpp get -reference hello for fileconfig_add

waruqi · 2022-08-12T14:26:02Z

maybe we cannot use fileconfig_add mode for vs2019, because of the generated dependences, some cpp files may be missing.

we need add mapper flags for all cpp file if vs <= 2019, or we need improve fallback_generate_dependencies to scan all requires

waruqi · 2022-08-12T14:52:56Z

I have improved fallback_generate_dependencies, it should work for 2019.

Arthapz force-pushed the cxxmodules branch from 3e4b734 to f065533 Compare August 9, 2022 10:43

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/common.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/clang.lua Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 9, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

Arthapz added 8 commits August 10, 2022 16:52

Avoid building a same header unit at the same time

172d2c7

Fix clang module compilation

6d0a8f9

Improve MSVC and clang module build

b3ed354

don't rebuild headerunits build by other targets

eb4dcb3

fix headerunits on clang

4dbf51b

fix clang

12b6d0c

fix clang

fa4ada8

Improve MSVC

f4fda8f

Arthapz force-pushed the cxxmodules branch from a734c9c to f4fda8f Compare August 10, 2022 14:53

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

waruqi reviewed Aug 10, 2022

View reviewed changes

xmake/rules/c++/modules/modules_support/msvc.lua Outdated Show resolved Hide resolved

Arthapz added 2 commits August 10, 2022 20:40

fix msvc

e0ef300

remove unused vars

5619e35

Arthapz force-pushed the cxxmodules branch 4 times, most recently from e007c60 to c6b465a Compare August 10, 2022 18:55

improve clang

1cba4c5

Arthapz force-pushed the cxxmodules branch from c6b465a to 1cba4c5 Compare August 10, 2022 18:56

improve code

81eda42

Arthapz force-pushed the cxxmodules branch from dff1959 to 81eda42 Compare August 10, 2022 21:07

code style

9d0aee8

fix user_headerunit link test

5e0358b

Update common.lua

8dccc2d

waruqi marked this pull request as ready for review August 13, 2022 00:47

waruqi merged commit b48b639 into xmake-io:cxxmodules Aug 13, 2022

waruqi mentioned this pull request Aug 15, 2022

Improve C++20 modules support #2641

Merged

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve MSVC and clang modules compilation #2664

Improve MSVC and clang modules compilation #2664

Arthapz commented Aug 9, 2022 •

edited by waruqi

Loading

waruqi commented Aug 9, 2022

Arthapz commented Aug 9, 2022

Arthapz commented Aug 9, 2022 •

edited

Loading

waruqi commented Aug 9, 2022

waruqi commented Aug 9, 2022 •

edited

Loading

Arthapz commented Aug 9, 2022

Arthapz commented Aug 10, 2022

waruqi commented Aug 10, 2022

waruqi commented Aug 10, 2022

waruqi commented Aug 11, 2022

Arthapz commented Aug 11, 2022 •

edited

Loading

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022 •

edited

Loading

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022 •

edited

Loading

waruqi commented Aug 12, 2022

Improve MSVC and clang modules compilation #2664

Improve MSVC and clang modules compilation #2664

Conversation

Arthapz commented Aug 9, 2022 • edited by waruqi Loading

waruqi commented Aug 9, 2022

Arthapz commented Aug 9, 2022

Arthapz commented Aug 9, 2022 • edited Loading

waruqi commented Aug 9, 2022

waruqi commented Aug 9, 2022 • edited Loading

Arthapz commented Aug 9, 2022

Arthapz commented Aug 10, 2022

waruqi commented Aug 10, 2022

waruqi commented Aug 10, 2022

waruqi commented Aug 11, 2022

Arthapz commented Aug 11, 2022 • edited Loading

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022 • edited Loading

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022

waruqi commented Aug 12, 2022 • edited Loading

waruqi commented Aug 12, 2022

Arthapz commented Aug 9, 2022 •

edited by waruqi

Loading

Arthapz commented Aug 9, 2022 •

edited

Loading

waruqi commented Aug 9, 2022 •

edited

Loading

Arthapz commented Aug 11, 2022 •

edited

Loading

waruqi commented Aug 12, 2022 •

edited

Loading

waruqi commented Aug 12, 2022 •

edited

Loading