Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use lib prefix in all native library names #33236

Merged
merged 1 commit into from
Mar 7, 2020
Merged

Use lib prefix in all native library names #33236

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions eng/native/naming.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<Project>
<!-- Add path globs specific to native binaries to exclude unnecessary files from packages. -->
<Choose>
<When Condition=" '$(_runtimeOSFamily)' == 'win' OR $(RuntimeIdentifier.StartsWith('win')) ">
<PropertyGroup>
<ApplicationFileExtension>.exe</ApplicationFileExtension>
<LibraryFilePrefix></LibraryFilePrefix>
<LibraryFileExtension>.dll</LibraryFileExtension>
<SymbolFileExtension>.pdb</SymbolFileExtension>
</PropertyGroup>
</When>
<When Condition=" '$(_runtimeOSFamily)' == 'osx' OR $(RuntimeIdentifier.StartsWith('osx')) ">
<PropertyGroup>
<LibraryFilePrefix Condition=" '$(SkipLibraryPrefixFromUnix)' == '' ">lib</LibraryFilePrefix>
<LibraryFileExtension>.dylib</LibraryFileExtension>
<SymbolFileExtension>.dwarf</SymbolFileExtension>
</PropertyGroup>
</When>
<When Condition=" '$(_runtimeOSFamily)' == 'android' ">
<PropertyGroup>
<LibraryFilePrefix Condition=" '$(SkipLibraryPrefixFromUnix)' == '' ">lib</LibraryFilePrefix>
<LibraryFileExtension>.so</LibraryFileExtension>
<!--symbols included in .so, like Linux, but can be generated externally and if so, uses .debug ext-->
<SymbolFileExtension>.debug</SymbolFileExtension>
</PropertyGroup>
</When>
<Otherwise>
<PropertyGroup>
<LibraryFilePrefix Condition=" '$(SkipLibraryPrefixFromUnix)' == '' ">lib</LibraryFilePrefix>
<LibraryFileExtension>.so</LibraryFileExtension>
<SymbolFileExtension>.dbg</SymbolFileExtension>
</PropertyGroup>
</Otherwise>
</Choose>

<ItemGroup>
<AdditionalLibPackageExcludes Condition="'$(SymbolFileExtension)' != ''" Include="%2A%2A\%2A$(SymbolFileExtension)" />
<AdditionalSymbolPackageExcludes Condition="'$(LibraryFileExtension)' != ''" Include="%2A%2A\%2A.a;%2A%2A\%2A$(LibraryFileExtension)" />
</ItemGroup>

</Project>
27 changes: 4 additions & 23 deletions src/coreclr/src/.nuget/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
<!-- build the transport package which includes product and symbols in addition to standard packages -->
<CreatePackedPackage Condition="'$(CreatePackedPackage)' == ''">true</CreatePackedPackage>

<!-- indicates that lib prefix is not added to library names for Unix-like operating systems -->
<SkipLibraryPrefixFromUnix>true</SkipLibraryPrefixFromUnix>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the reason for this property? why would we want to skip the prefix?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of coreclr, the prefixes were not added in the original version. Apparently, there is a job on CI that crosscompile clrjit.so (Linux version) on Windows. That one was failing, so I restored it by skipping the prefix here. It will require a bit more investigation and it can be avoided (at least in theory) 😅


<!-- Distro rid is passed as runtimeos-arch-->
<_parseDistroRid>$(__DistroRid)</_parseDistroRid>
<_parseDistroRid Condition="'$(_parseDistroRid)' == '' and '$(__BuildOS)' == 'OSX'">osx.10.12-x64</_parseDistroRid>
Expand Down Expand Up @@ -129,29 +132,7 @@
</Otherwise>
</Choose>

<!-- Determine per-platform native binary extensions. -->
<Choose>
<When Condition="'$(_runtimeOSFamily)' == 'win'" />
<When Condition="'$(_runtimeOSFamily)' == 'osx'">
<PropertyGroup>
<LibraryFileExtension>.dylib</LibraryFileExtension>
<SymbolFileExtension>.dwarf</SymbolFileExtension>
</PropertyGroup>
</When>
<When Condition="'$(_runtimeOSFamily)' == 'android'">
<PropertyGroup>
<LibraryFileExtension>.so</LibraryFileExtension>
<!--symbols included in .so, like Linux, but can be generated externally and if so, uses .debug ext-->
<SymbolFileExtension>.debug</SymbolFileExtension>
</PropertyGroup>
</When>
<Otherwise>
<PropertyGroup>
<LibraryFileExtension>.so</LibraryFileExtension>
<SymbolFileExtension>.dbg</SymbolFileExtension>
</PropertyGroup>
</Otherwise>
</Choose>
<Import Project="$(RepoRoot)eng\native\naming.props" />

<ItemGroup Condition="$(SupportedPackageOSGroups.Contains(';Linux;'))">
<OfficialBuildRID Include="linux-x64" />
Expand Down
34 changes: 1 addition & 33 deletions src/installer/pkg/projects/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
<Project>
<Import Project="$([MSBuild]::GetPathOfFileAbove(Directory.Build.props, $(MSBuildThisFileDirectory)..))" />

<Import Project="Sdk.props" Sdk="Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk" />

<PropertyGroup>
Expand Down Expand Up @@ -169,36 +168,5 @@
<RuntimeIdentifier>$(PackageRID)</RuntimeIdentifier>
</PropertyGroup>

<Choose>
<When Condition="$(RuntimeIdentifier.StartsWith('win'))">
<PropertyGroup>
<ApplicationFileExtension>.exe</ApplicationFileExtension>
<LibraryFilePrefix></LibraryFilePrefix>
<LibraryFileExtension>.dll</LibraryFileExtension>
<SymbolFileExtension>.pdb</SymbolFileExtension>
</PropertyGroup>
</When>
<When Condition="$(RuntimeIdentifier.StartsWith('osx'))">
<PropertyGroup>
<ApplicationFileExtension></ApplicationFileExtension>
<LibraryFilePrefix>lib</LibraryFilePrefix>
<LibraryFileExtension>.dylib</LibraryFileExtension>
<SymbolFileExtension>.dwarf</SymbolFileExtension>
</PropertyGroup>
</When>
<Otherwise>
<PropertyGroup>
<ApplicationFileExtension></ApplicationFileExtension>
<LibraryFilePrefix>lib</LibraryFilePrefix>
<LibraryFileExtension>.so</LibraryFileExtension>
<SymbolFileExtension>.dbg</SymbolFileExtension>
</PropertyGroup>
</Otherwise>
</Choose>

<ItemGroup>
<AdditionalLibPackageExcludes Condition="'$(SymbolFileExtension)' != ''" Include="%2A%2A\%2A$(SymbolFileExtension)" />
<AdditionalSymbolPackageExcludes Condition="'$(LibraryFileExtension)' != ''" Include="%2A%2A\%2A.a;%2A%2A\%2A$(LibraryFileExtension)" />
</ItemGroup>

<Import Project="$(RepoRoot)eng\native\naming.props" />
</Project>
12 changes: 6 additions & 6 deletions src/libraries/Common/src/Interop/Unix/Interop.Libraries.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ internal static partial class Interop
internal static partial class Libraries
{
// Shims
internal const string SystemNative = "System.Native";
akoeplinger marked this conversation as resolved.
Show resolved Hide resolved
internal const string GlobalizationNative = "System.Globalization.Native";
internal const string NetSecurityNative = "System.Net.Security.Native";
internal const string CryptoNative = "System.Security.Cryptography.Native.OpenSsl";
internal const string CompressionNative = "System.IO.Compression.Native";
internal const string IOPortsNative = "System.IO.Ports.Native";
internal const string SystemNative = "libSystem.Native";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would lib.System.Native (and so forth) be better? more readable than libSystem.Native perhaps

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, this is not the Linux convention. usually `lib' is prefixed without dots.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plus adding the . might make it such that the loader won't find the library automatically as for anybody PInvoking to System.Native, I believe the runtime will try either System.Native or preappending lib, but it won't find it if it also needs to pre-append a dot.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense

internal const string GlobalizationNative = "libSystem.Globalization.Native";
internal const string NetSecurityNative = "libSystem.Net.Security.Native";
internal const string CryptoNative = "libSystem.Security.Cryptography.Native.OpenSsl";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change, overall, makes me sad. We intentionally stripped off the lib prefix as part of "these aren't general purpose libraries, don't look here". But if we need to, we need to, I guess.

Copy link
Member

@jkotas jkotas Mar 5, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative may be to statically link these libraries into the runtime for Android where it is causing problems. We are going to have this option for single file flavor of CoreCLR anyway, so doing it for Android as well (or even everywhere) should not be a big deal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CC @akoeplinger and @marek-safar to be aware of the suggestion.

Copy link
Contributor

@marek-safar marek-safar Mar 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

statically link these libraries into the runtime

Xamarin.Android does not support this mode. I think this would be nice but we won't be in a position to deliver this for .net5. This looks more like .net6 enhancement because we would need to do native linking during publish phase (this depends on Android NDK, etc).

/cc @jonpryor

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For CoreCLR, the plan is to statically link the libraries as part of the dotnet/runtime build (see #32823). We are not planning to do native linking during publish phase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That won't work for us. The thinking right now is to copy the dynamic libs which are used (during publish) and extend that to full static linking for Android as we plan to do for iOS (and wasm) during publish

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious - why it does not work for you?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Primary because of size. If an app uses no crypto we should not link libSystem.Net.Security.Native and you can apply that to all native libs where System.IO.Compression.Native is probably the biggest one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The moment someone brings in any managed HTTP implementation, client or server, it's going to end up requiring both of those. Presumably then the size concern would only be if they were using a platform-specific implementation (e.g. AndroidHttpHandler) where the platform provided all of that support?

internal const string CompressionNative = "libSystem.IO.Compression.Native";
internal const string IOPortsNative = "libSystem.IO.Ports.Native";
internal const string Libdl = "libdl";
}
}
1 change: 0 additions & 1 deletion src/libraries/Native/Unix/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ set(CMAKE_MACOSX_RPATH ON)
set(CMAKE_INSTALL_PREFIX $ENV{__CMakeBinDir})
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
set(CMAKE_SHARED_LIBRARY_PREFIX "")
set(VERSION_FILE_PATH "${CMAKE_BINARY_DIR}/version.c")

# We mark the function which needs exporting with PALEXPORT
Expand Down
2 changes: 0 additions & 2 deletions src/libraries/Native/Unix/System.Globalization.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,6 @@ add_library(System.Globalization.Native-Static
${NATIVEGLOBALIZATION_SOURCES}
)

# Disable the "lib" prefix and override default name
set_target_properties(System.Globalization.Native-Static PROPERTIES PREFIX "")
set_target_properties(System.Globalization.Native-Static PROPERTIES OUTPUT_NAME System.Globalization.Native CLEAN_DIRECT_OUTPUT 1)

install (TARGETS System.Globalization.Native-Static DESTINATION .)
Expand Down
2 changes: 0 additions & 2 deletions src/libraries/Native/Unix/System.IO.Compression.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,6 @@ add_library(System.IO.Compression.Native-Static
${NATIVECOMPRESSION_SOURCES}
)

# Disable the "lib" prefix and override default name
set_target_properties(System.IO.Compression.Native-Static PROPERTIES PREFIX "")
set_target_properties(System.IO.Compression.Native-Static PROPERTIES OUTPUT_NAME System.IO.Compression.Native CLEAN_DIRECT_OUTPUT 1)

install (TARGETS System.IO.Compression.Native-Static DESTINATION .)
2 changes: 0 additions & 2 deletions src/libraries/Native/Unix/System.IO.Ports.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ add_library(System.IO.Ports.Native-Static
pal_serial.c
)

# Disable the "lib" prefix and override default name
set_target_properties(System.IO.Ports.Native-Static PROPERTIES PREFIX "")
set_target_properties(System.IO.Ports.Native-Static PROPERTIES OUTPUT_NAME System.IO.Ports.Native CLEAN_DIRECT_OUTPUT 1)

install_library_and_symbols (System.IO.Ports.Native)
Expand Down
2 changes: 0 additions & 2 deletions src/libraries/Native/Unix/System.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,6 @@ add_library(System.Native-Static
${NATIVE_SOURCES}
)

# Disable the "lib" prefix and override default name
set_target_properties(System.Native-Static PROPERTIES PREFIX "")
set_target_properties(System.Native-Static PROPERTIES OUTPUT_NAME System.Native CLEAN_DIRECT_OUTPUT 1)

install (TARGETS System.Native-Static DESTINATION .)
2 changes: 0 additions & 2 deletions src/libraries/Native/Unix/System.Net.Security.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ add_library(System.Net.Security.Native-Static
${NATIVEGSS_SOURCES}
)

# Disable the "lib" prefix and override default name
set_target_properties(System.Net.Security.Native-Static PROPERTIES PREFIX "")
set_target_properties(System.Net.Security.Native-Static PROPERTIES OUTPUT_NAME System.Net.Security.Native CLEAN_DIRECT_OUTPUT 1)

target_link_libraries(System.Net.Security.Native
Expand Down
5 changes: 0 additions & 5 deletions src/libraries/Native/Unix/System.Security.Cryptography.Native.Apple/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,6 @@ add_library(System.Security.Cryptography.Native.Apple-Static
${NATIVECRYPTO_SOURCES}
)

# Disable the "lib" prefix.
set_target_properties(System.Security.Cryptography.Native.Apple PROPERTIES PREFIX "")

# Disable the "lib" prefix and override default name
set_target_properties(System.Security.Cryptography.Native.Apple-Static PROPERTIES PREFIX "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here, this is not only adding lib as a prefix, it is removing our override

set_target_properties(System.Security.Cryptography.Native.Apple-Static PROPERTIES OUTPUT_NAME System.Security.Cryptography.Native.Apple CLEAN_DIRECT_OUTPUT 1)

target_link_libraries(System.Security.Cryptography.Native.Apple
Expand Down
5 changes: 0 additions & 5 deletions src/libraries/Native/Unix/System.Security.Cryptography.Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,6 @@ add_library(System.Security.Cryptography.Native.OpenSsl-Static
$<TARGET_OBJECTS:objlib>
)

# Disable the "lib" prefix.
set_target_properties(System.Security.Cryptography.Native.OpenSsl PROPERTIES PREFIX "")

# Disable the "lib" prefix and override default name
set_target_properties(System.Security.Cryptography.Native.OpenSsl-Static PROPERTIES PREFIX "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too familiar with this so probably @bartonjs should take a look, but looks like in here you are also changing the override of the default name

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and override default name

this part of the comment refers to the line below. There are two variants of this library produced by this script: shared and static. We have prefix override for each which this delta is removing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this change is correct. We could however preserve the override default name part of the comment for the next line which sets the OUTPUT_NAME on the static lib.

set_target_properties(System.Security.Cryptography.Native.OpenSsl-Static PROPERTIES OUTPUT_NAME System.Security.Cryptography.Native.OpenSsl CLEAN_DIRECT_OUTPUT 1)

if (FEATURE_DISTRO_AGNOSTIC_SSL)
Expand Down
23 changes: 2 additions & 21 deletions src/libraries/pkg/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,30 +35,11 @@
<Project Include="@(_project->'$(MSBuildProjectName).pkgproj')" />
</ItemGroup>

<!-- Add path globs specific to native binaries to exclude unnecessary files from packages. -->
<Choose>
<When Condition="$(PackageTargetRuntime.StartsWith('win'))"/>
<When Condition="$(PackageTargetRuntime.StartsWith('osx'))">
<PropertyGroup>
<LibraryFileExtension>.dylib</LibraryFileExtension>
<SymbolFileExtension>.dwarf</SymbolFileExtension>
</PropertyGroup>
</When>
<Otherwise>
<PropertyGroup>
<LibraryFileExtension>.so</LibraryFileExtension>
<SymbolFileExtension>.dbg</SymbolFileExtension>
</PropertyGroup>
</Otherwise>
</Choose>
<ItemGroup>
<AdditionalLibPackageExcludes Condition="'$(SymbolFileExtension)' != ''" Include="%2A%2A\%2A$(SymbolFileExtension)" />
<AdditionalSymbolPackageExcludes Condition="'$(LibraryFileExtension)' != ''" Include="%2A%2A\%2A.a;%2A%2A\%2A$(LibraryFileExtension)" />
</ItemGroup>

<PropertyGroup>
<!-- SuppressFinalPackageVersion on private packages by default -->
<SuppressFinalPackageVersion Condition="'$(SuppressFinalPackageVersion)' == '' and $(MSBuildProjectName.Contains('Private'))">true</SuppressFinalPackageVersion>
</PropertyGroup>

<Import Project="$(RepoRoot)eng\native\naming.props" />

</Project>
2 changes: 1 addition & 1 deletion src/libraries/pkg/runtime.native.System.IO.Ports/runtime.native.System.IO.Ports.pkgproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<SkipValidatePackage>true</SkipValidatePackage>
</PropertyGroup>
<ItemGroup Condition="'$(PackageTargetRuntime)' == 'linux-arm' or '$(PackageTargetRuntime)' == 'linux-arm64' or '$(PackageTargetRuntime)' == 'linux-x64' or '$(PackageTargetRuntime)' == 'osx-x64' or '$(PackageTargetRuntime)' == 'freebsd-x64'">
<File Include="$(NativeBinDir)\System.IO.Ports.Native$(LibraryFileExtension)" >
<File Include="$(NativeBinDir)\$(LibraryFilePrefix)System.IO.Ports.Native$(LibraryFileExtension)" >
<TargetPath>runtimes/$(PackageRID)/native</TargetPath>
</File>
</ItemGroup>
Expand Down
25 changes: 2 additions & 23 deletions src/mono/netcore/nuget/Directory.Build.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
<_runtimeOSVersionIndex>$(RuntimeOS.IndexOfAny(".-0123456789"))</_runtimeOSVersionIndex>
<_runtimeOSFamily Condition="'$(_runtimeOSVersionIndex)' != '-1'">$(RuntimeOS.SubString(0, $(_runtimeOSVersionIndex)))</_runtimeOSFamily>
<_runtimeOSFamily Condition="'$(_runtimeOSVersionIndex)' == '-1'">$(RuntimeOS)</_runtimeOSFamily>
<PackageTargetRuntime>$(_runtimeOSFamily)</PackageTargetRuntime>
<_isSupportedOSGroup>true</_isSupportedOSGroup>
</PropertyGroup>

Expand Down Expand Up @@ -114,29 +115,7 @@
</Otherwise>
</Choose>

<!-- Determine per-platform native binary extensions. -->
<Choose>
<When Condition="'$(_runtimeOSFamily)' == 'win'" />
<When Condition="'$(_runtimeOSFamily)' == 'osx'">
<PropertyGroup>
<LibraryFileExtension>.dylib</LibraryFileExtension>
<SymbolFileExtension>.dwarf</SymbolFileExtension>
</PropertyGroup>
</When>
<When Condition="'$(_runtimeOSFamily)' == 'android'">
<PropertyGroup>
<LibraryFileExtension>.so</LibraryFileExtension>
<!--symbols included in .so, like Linux, but can be generated externally and if so, uses .debug ext-->
<SymbolFileExtension>.debug</SymbolFileExtension>
</PropertyGroup>
</When>
<Otherwise>
<PropertyGroup>
<LibraryFileExtension>.so</LibraryFileExtension>
<SymbolFileExtension>.dbg</SymbolFileExtension>
</PropertyGroup>
</Otherwise>
</Choose>
<Import Project="$(RepoRoot)eng\native\naming.props" />

<ItemGroup Condition="$(SupportedPackageOSGroups.Contains(';Linux;'))">
<OfficialBuildRID Include="linux-x64" />
Expand Down