Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm] Clean emsdk setup #106403

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e064ec6
Remove embedded scripts, fix build on Linux.
ilonatommy Aug 14, 2024
7bc1637
Merge branch 'main' into fix-105439-changing-runtime-only - fix froze…
ilonatommy Aug 14, 2024
60bfafc
Windows - almost fixed, missing native build access to EMSDK_PATH.
ilonatommy Aug 16, 2024
07ce65d
Fix `$(EMSDK_PATH)` is missing is missing some paths.
ilonatommy Aug 16, 2024
5636d09
Fix wasi.
ilonatommy Aug 16, 2024
ea2d900
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 16, 2024
4a3e122
Cleanup - move the pre-env script logic to emscripten provisioning.
ilonatommy Aug 16, 2024
bf2c1bb
Move remaining emsdk-provision bits to provision target.
ilonatommy Aug 19, 2024
97a5513
Fix: it's an Item, not a property.
ilonatommy Aug 19, 2024
c9852a5
More cleanup on Linux.
ilonatommy Aug 19, 2024
5f96265
Fix rebuild.
ilonatommy Aug 19, 2024
0335335
Fix "NODE_JS is set to empty value in D:\a\_work\1\s\src\mono\browser…
ilonatommy Aug 19, 2024
89148e8
Fix "BINARYEN_ROOT is set to empty value in \src\mono\browser\emsdk\e…
ilonatommy Aug 19, 2024
bd9179b
Workaround - set `EMSKD_PATH` for native build separately.
ilonatommy Aug 19, 2024
7da3a6c
Cleanup.
ilonatommy Aug 19, 2024
1a94c15
Fix "Specified Emscripten sdk at $(EMSDK_PATH) is missing some paths:…
ilonatommy Aug 19, 2024
d8b8a7c
Fix: path to python "is not recognized as an internal or external com…
ilonatommy Aug 19, 2024
6246fed
Missing change for the fix in the previous commit.
ilonatommy Aug 19, 2024
6fd5e6f
Fix wbt: add missing env vars
ilonatommy Aug 20, 2024
fbb71be
Missing "bin" part of node path.
ilonatommy Aug 21, 2024
caf5379
Lib tests have "bin" in the end of node path but wbt do not, satisfy …
ilonatommy Aug 21, 2024
2a34403
Wrong condition, c&p error.
ilonatommy Aug 21, 2024
fbd9744
Fix WBT: "binaryen executable not found (%HostBinaryenDir%\bin\wasm-o…
ilonatommy Aug 21, 2024
8ed999c
"EndsWith" did not detect the need to append "bin" correctly.
ilonatommy Aug 21, 2024
383e5cd
Revert redundant changes to user-app targets.
ilonatommy Aug 22, 2024
2dd0adb
Missing change revert
ilonatommy Aug 22, 2024
65e574e
Feedback.
ilonatommy Aug 22, 2024
7d70eb6
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 23, 2024
1b7befc
Import after `SetupPackagePaths` finishes.
ilonatommy Aug 23, 2024
5830355
Fix
ilonatommy Aug 23, 2024
fb1738b
Remove redundant comments.
ilonatommy Aug 27, 2024
5f8908b
`WasmApp.InTree.props` are loaded too late. Move the props back to de…
ilonatommy Aug 27, 2024
8c9a3f0
Fix missing slash in the end of the path.
ilonatommy Aug 27, 2024
b14d7a9
Remove whitespace change.
ilonatommy Aug 28, 2024
755c343
Merge branch 'main' into fix-105439-changing-runtime-only
ilonatommy Aug 28, 2024
6daf79e
Make sure properties to build nuget paths on helix are not empty.
ilonatommy Sep 3, 2024
08f86f3
Debugging commit, see where are packages.
ilonatommy Sep 3, 2024
4744846
More debugging.
ilonatommy Sep 3, 2024
574548b
Fallback for helix.
ilonatommy Sep 5, 2024
f9acf9f
Cleanup needed when we have the fallback.
ilonatommy Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/native/gen-buildsys.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ if not "%__ConfigureOnly%" == "1" (
)

if /i "%__UseEmcmake%" == "1" (
call "!EMSDK_PATH!/emsdk_env.cmd" > nul 2>&1 && emcmake "%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
call "!EMSDK_PATH!/pre_emsdk_env.cmd" > nul 2>&1 && call "!EMSDK_PATH!/emsdk_env.cmd" > nul 2>&1 && emcmake "%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
) else (
"%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
)
Expand Down
8 changes: 4 additions & 4 deletions src/mono/browser/browser.proj
Original file line number Diff line number Diff line change
Expand Up @@ -415,13 +415,13 @@
<CMakeBuildRuntimeConfigureCmd Condition="'$(WasmEnableJsInteropByValue)' == 'true'">$(CMakeBuildRuntimeConfigureCmd) -DENABLE_JS_INTEROP_BY_VALUE=1</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd>$(CMakeBuildRuntimeConfigureCmd) $(CMakeConfigurationEmsdkPath)</CMakeBuildRuntimeConfigureCmd>

<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)'</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'pre_emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)</CMakeBuildRuntimeConfigureCmd>
<CMakeBuildRuntimeConfigureCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/pre_emsdk_env.sh &amp;&amp; source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeConfigureCmd)'</CMakeBuildRuntimeConfigureCmd>

<CMakeOptions Condition="'$(MonoVerboseBuild)' != ''">-v</CMakeOptions>
<CMakeBuildRuntimeCmd>cmake --build . --config $(Configuration) $(CmakeOptions)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeCmd)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeCmd)'</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' == 'Windows_NT'">call &quot;$(RepositoryEngineeringDir)native\init-vs-env.cmd&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'pre_emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; call &quot;$([MSBuild]::NormalizePath('$(EMSDK_PATH)', 'emsdk_env.cmd'))&quot; &amp;&amp; $(CMakeBuildRuntimeCmd)</CMakeBuildRuntimeCmd>
<CMakeBuildRuntimeCmd Condition="'$(OS)' != 'Windows_NT'">bash -c 'source $(EMSDK_PATH)/pre_emsdk_env.sh &amp;&amp; source $(EMSDK_PATH)/emsdk_env.sh 2>&amp;1 &amp;&amp; $(CMakeBuildRuntimeCmd)'</CMakeBuildRuntimeCmd>
</PropertyGroup>
<ItemGroup>
<_CmakeEnvironmentVariable Include="ENABLE_JS_INTEROP_BY_VALUE=1" Condition="'$(WasmEnableJsInteropByValue)' != 'false'"/>
Expand Down
20 changes: 19 additions & 1 deletion src/mono/browser/build/BrowserWasmApp.targets
Original file line number Diff line number Diff line change
Expand Up @@ -175,12 +175,25 @@

<Import Project="$(MSBuildThisFileDirectory)EmSdkRepo.Defaults.props" Condition="'$(WasmUseEMSDK_PATH)' == 'true'" />
<Target Name="_SetupEmscripten" BeforeTargets="_SetupToolchain">
<Exec Command="dotnet nuget locals all -l" ConsoleToMSBuild="true">
<Output TaskParameter="ConsoleOutput" PropertyName="NuGetLocalsOutput" />
</Exec>
<PropertyGroup>
<GlobalPackagesPath>$([System.Text.RegularExpressions.Regex]::Match($(NuGetLocalsOutput), 'global-packages:\s*([^;]+)').Groups[1].Value)</GlobalPackagesPath>
</PropertyGroup>
<Message Text="Global Packages Path: $(GlobalPackagesPath)" Importance="High" />
<PropertyGroup>
<NodeTransportDir>$([System.IO.Path]::Combine($(GlobalPackagesPath), 'runtime.$(_portableHostOS)-$(BuildArchitecture).microsoft.netcore.runtime.wasm.node.transport'))</NodeTransportDir>
<_EMSDKMissingPaths Condition="'$(_EMSDKMissingPaths)' == '' and ('$(EmscriptenSdkToolsPath)' == '' or !Exists('$(EmscriptenSdkToolsPath)'))">%24(EmscriptenSdkToolsPath)=$(EmscriptenSdkToolsPath) </_EMSDKMissingPaths>
<_EMSDKMissingPaths Condition="'$(_EMSDKMissingPaths)' == '' and ('$(EmscriptenNodeToolsPath)' == '' or !Exists('$(EmscriptenNodeToolsPath)'))">%24(EmscriptenNodeToolsPath)=$(EmscriptenNodeToolsPath) </_EMSDKMissingPaths>
<_EMSDKMissingPaths Condition="'$(_EMSDKMissingPaths)' == '' and ('$(EmscriptenUpstreamBinPath)' == '' or !Exists('$(EmscriptenUpstreamBinPath)'))">%24(EmscriptenUpstreamBinPath)=$(EmscriptenUpstreamBinPath) </_EMSDKMissingPaths>
</PropertyGroup>

<ItemGroup>
<NodeTransportDirContents Include="$(NodeTransportDir)\*" />
<PackagesDirContents Include="$(GlobalPackagesPath)\*" />
</ItemGroup>

<PropertyGroup>
<_ToolchainMissingErrorMessage Condition="'$(EMSDK_PATH)' == '' and '$(EmscriptenSdkToolsPath)' == ''">Could not find emscripten sdk. Either set %24(EMSDK_PATH), or use workloads to get the sdk.</_ToolchainMissingErrorMessage>

Expand All @@ -189,19 +202,24 @@
<_ToolchainMissingErrorMessage Condition="'$(_ToolchainMissingErrorMessage)' == '' and '$(_UsingEMSDK_PATH)' == 'true' and '$(_EMSDKMissingPaths)' != ''">Specified Emscripten sdk at %24(EMSDK_PATH)=$(EMSDK_PATH) is missing some paths: $(_EMSDKMissingPaths).</_ToolchainMissingErrorMessage>

<_IsToolchainMissing Condition="'$(_ToolchainMissingErrorMessage)' != ''">true</_IsToolchainMissing>
<_ToolchainMissingErrorMessage>$(_ToolchainMissingErrorMessage); [DEBUG] Directories at $(NodeTransportDir): %(NodeTransportDirContents.Identity); Directories at $(PackagesDir): %(PackagesDirContents.Identity);</_ToolchainMissingErrorMessage>
</PropertyGroup>

<PropertyGroup>
<EmscriptenSdkToolsPath Condition="'$(EmscriptenSdkToolsPath)' != ''" >$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath)))</EmscriptenSdkToolsPath>
<EmscriptenNodeToolsPath Condition="'$(EmscriptenNodeToolsPath)' != ''" >$([MSBuild]::NormalizeDirectory($(EmscriptenNodeToolsPath)))</EmscriptenNodeToolsPath>
<EmscriptenUpstreamBinPath Condition="'$(EmscriptenUpstreamBinPath)' != ''">$([MSBuild]::NormalizeDirectory($(EmscriptenUpstreamBinPath)))</EmscriptenUpstreamBinPath>
<_EmscriptenNodePath>$([MSBuild]::NormalizePath($(EmscriptenNodeToolsPath), 'bin', 'node$(_ExeExt)'))</_EmscriptenNodePath>
</PropertyGroup>

<!-- Environment variables required for running emsdk commands like `emcc` -->
<ItemGroup Condition="'$(EmscriptenSdkToolsPath)' != ''">
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_LLVM_ROOT=$(EmscriptenSdkToolsPath)bin" />
<EmscriptenEnvVars Include="EM_LLVM_ROOT=$(EmscriptenSdkToolsPath)bin" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_BINARYEN_ROOT=$(EmscriptenSdkToolsPath)" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_NODE_JS=$([MSBuild]::NormalizePath($(EmscriptenNodeToolsPath), 'bin', 'node$(_ExeExt)'))" />
<EmscriptenEnvVars Condition="'$(OS)' == 'Windows_NT'" Include="EM_BINARYEN_ROOT=$(EmscriptenSdkToolsPath)" />
<EmscriptenEnvVars Include="DOTNET_EMSCRIPTEN_NODE_JS=$(_EmscriptenNodePath)" />
<EmscriptenEnvVars Include="EM_NODE_JS=$(_EmscriptenNodePath)" />
</ItemGroup>

<ItemGroup>
Expand Down
9 changes: 7 additions & 2 deletions src/mono/browser/build/EmSdkRepo.Defaults.props
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,20 @@
<EmscriptenUpstreamBinPath>$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath), 'bin'))</EmscriptenUpstreamBinPath>
<EmscriptenUpstreamEmscriptenPath>$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath), 'emscripten'))</EmscriptenUpstreamEmscriptenPath>

<_PathToNugetsAvailable Condition="'$(NuGetPackageRoot)' != '' and '$(_portableHostOS)' != '' and '$(BuildArchitecture)' != '' and '$(NodePackageVersion)' != ''">true</_PathToNugetsAvailable>

<!-- gets the path like emsdk/python/3.7.4-2_64bit -->
<EmscriptenNodeToolsPath Condition="'$(EmscriptenNodeToolsPath)' == ''">$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath), 'node', 'bin'))</EmscriptenNodeToolsPath>
<EmscriptenNodeToolsPath Condition="'$(EmscriptenNodeToolsPath)' == '' and '$(_PathToNugetsAvailable)' != 'true'">$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath), 'node', 'bin'))</EmscriptenNodeToolsPath>
<EmscriptenNodeToolsPath Condition="'$(EmscriptenNodeToolsPath)' == ''">$([System.IO.Path]::Combine($(NuGetPackageRoot), 'runtime.$(_portableHostOS)-$(BuildArchitecture).microsoft.netcore.runtime.wasm.node.transport', '$(NodePackageVersion)', 'tools', '$(_portableHostOS)-$(BuildArchitecture)'))</EmscriptenNodeToolsPath>

<_UsingEMSDK_PATH>true</_UsingEMSDK_PATH>
</PropertyGroup>

<!-- Emscripten uses system python on Linux and macos, so we don't need $(EmscriptenPythonToolsPath) -->
<PropertyGroup Condition="!$([MSBuild]::IsOSPlatform('linux')) and !$([MSBuild]::IsOSPlatform('OSX'))">
<EmscriptenPythonToolsPath>$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath)python))</EmscriptenPythonToolsPath>
<_PathToNugetsAvailable Condition="'$(NuGetPackageRoot)' != '' and '$(EmsdkVersion)' != '' and '$(BuildArchitecture)' != '' and '$(EmsdkPackageVersion)' != ''">true</_PathToNugetsAvailable>
<EmscriptenPythonToolsPath Condition="'$(_PathToNugetsAvailable)' != 'true'" >$([MSBuild]::NormalizeDirectory($(EmscriptenSdkToolsPath)python))</EmscriptenPythonToolsPath>
<EmscriptenPythonToolsPath Condition="'$(EmscriptenPythonToolsPath)' == ''">$([MSBuild]::NormalizeDirectory($([System.IO.Path]::Combine($(NuGetPackageRoot), 'Microsoft.NET.Runtime.Emscripten.$(EmsdkVersion).Python.win-$(BuildArchitecture)', '$(EmsdkPackageVersion)', 'tools'))))</EmscriptenPythonToolsPath>

<_EMSDKMissingPaths Condition="'$(_EMSDKMissingPaths)' == '' and ('$(EmscriptenPythonToolsPath)' == '' or !Exists('$(EmscriptenPythonToolsPath)'))">%24(EmscriptenPythonToolsPath)=$(EmscriptenPythonToolsPath) </_EMSDKMissingPaths>
</PropertyGroup>
Expand Down
Loading
Loading