Add Linking the .NET Libraries doc #1
Conversation
* .NET Runtime Form Factors * Update accepted/2020/form-factors.md Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com> * Update accepted/2020/form-factors.md Co-Authored-By: Sven Boemer <sbomer@gmail.com> * Feedback * Update accepted/2020/form-factors.md Co-Authored-By: Ben Adams <thundercat@illyriad.co.uk> * Update accepted/2020/form-factors.md Co-Authored-By: Dan <dan.quist@gmail.com> * Fix link * Reordered AOT sections Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com> Co-authored-by: Sven Boemer <sbomer@gmail.com> Co-authored-by: Ben Adams <thundercat@illyriad.co.uk> Co-authored-by: Dan <dan.quist@gmail.com>
There was a problem hiding this comment.
Really good writeup !!! (really good!)
I don't see anything important missing.
Maybe we could explicitly mention that some parts of the framework might be really problematic and might never work with linker well (unlike serializers which are problematic, but we want to make them work) - example could be DI (without redesign of the APIs), Expression.Linq, .... But I'm not sure this has place in the doc - it's more about setting expectations... so maybe we should not put it here. On the other hand in theory we can make anything work with enough changes.
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
accepted/2020/linking-libraries.md
Outdated
| In the context of trimming applications, there are two main concerns: | ||
|
|
||
| 1. Any code that is used by the application must be preserved in the application. | ||
| 2. Any code that is not used by the application can be removed from the application. |
There was a problem hiding this comment.
| 2. Any code that is not used by the application can be removed from the application. | |
| 2. Any code or dependencies that are not used by the application can be removed from the self-contained application. |
There was a problem hiding this comment.
I'm not sure I follow the or dependencies part. To me dependencies are code as well. I'd also like to keep these two sentences symmetrical, to help drive the point of must and can below.
There was a problem hiding this comment.
The dependencies in this context are for example "facade" assemblies, unused Android resources or unmanaged native libraries.
|
|
||
| A high-level goal is that the .NET Libraries are clean of linker warnings. This is one level of confidence we can have that .NET applications will be linker safe. | ||
|
|
||
| #### Serialization |
There was a problem hiding this comment.
Why to explicitly call out serialization only? What about other large framework features should they be described in the same level of detail?
There was a problem hiding this comment.
There was a problem hiding this comment.
My reasoning was that serialization seemed to be a common enough pattern used across many features, and in some cases it isn't immediately obvious that a feature is serializing objects - e.g. EventSource. It isn't a single feature, but instead a class/category of features that is problematic for the linker.
There was a problem hiding this comment.
If you meant to include serialization only as example then I think that's fine but it should be more obvious from the text
|
|
||
| In general, using unit tests to ensure a linked application still works isn't a great testing strategy. Unit tests, by definition, try exercising all the code in the library. If all the code in the library is being exercised, the linker won't be able to trim any of it. Thus if nothing is trimmed, you aren't confirming that the library still works when not all of the code is present. | ||
|
|
||
| Instead of using unit tests, we should create dedicated applications to test that the libraries are linker safe. One set of applications will be higher-level applications that we want to measure the size over time. Another set will target specific APIs that needed to be annotated to ensure the annotations and the linker are working correctly. |
There was a problem hiding this comment.
Could this be rewritten to be more actionable?
- How should target specific APIs test look like?
- Where they will be located?
- How they will be run?
- Who will be responsible for writing the test(s)?
There was a problem hiding this comment.
Who will be responsible for writing the test(s)?
The .NET Library developers will write the target specific APIs test, as it is API specific.
As for the higher-level applications, I think that is up for debate whether it is the perf team, the application-model team (Xamarin, Blazor, etc), the Libraries team, or someone else.
Where they will be located?
I think a decent place for the higher-level applications would be https://github.com/dotnet/performance. It has the ability to do the "measure over time" capabilities that I mention in the doc. It would also keep application-model specific dependencies out of dotnet/runtime.
As for the targeted API tests, I was assuming they would be in dotnet/runtime, since they are tied to the APIs.
There was a problem hiding this comment.
As for the targeted API tests
Where and how they will be written when they cannot be xunit based or use any testing framework?
I'd focus on unit test only in this doc but define them well (see the questions I asked they were all about unit tests)
|
|
||
| Instead of using unit tests, we should create dedicated applications to test that the libraries are linker safe. One set of applications will be higher-level applications that we want to measure the size over time. Another set will target specific APIs that needed to be annotated to ensure the annotations and the linker are working correctly. | ||
|
|
||
| As we are measuring size over time, we would not like to get "noise" in the sizes to cause false positives. For example, if an application size gets larger by ~10 KB, we don't necessarily want the test to fail. And conversely, if an application size suddenly get 50% larger (or smaller), we probably do want to alert someone - as an anomaly was detected and it is possibly a bug somewhere. |
|
|
||
| This is especially important for applications that need to be downloaded over the internet, for example Xamarin and Blazor applications. End users on a slow or limited network will be less likely to install and use applications that are too large. | ||
|
|
||
| The [ILLinker](https://github.com/mono/linker) is a tool that is able to take an application and trim the classes and methods that are not used by the application. In many scenarios, this results in a significantly smaller packaged application. |
There was a problem hiding this comment.
Complete aside, but are there any plans to move https://github.com/mono/linker to be in the dotnet org as https://github.com/dotnet/linker?
There was a problem hiding this comment.
@marek-safar @vitek-karas - thoughts?
There was a problem hiding this comment.
No, I don't see any benefit of that work. Why?
There was a problem hiding this comment.
It would be for the same reason why we have moved e.g. ASP.NET from the aspnet org to dotnet org. The primary reasons are consistency and public perception.
Other more technical reasons include permission management or complications moving issues between repos when they are in different orgs.
accepted/2020/linking-libraries.md
Outdated
|
|
||
| In both of these cases, the code is sometimes needed, and sometimes isn't. If you are never going to debug your application (for example, the application is deployed to a device that can't be debugged), it isn't necessary to keep these types in the application. A similar situation exists with COM. Since COM is not IL, the linker isn't able to see if some code in the application is going to use COM or not. | ||
|
|
||
| To solve these, we can define a [`feature switch`](#Adding-appropriate-feature-switches) for each of these scenarios. An application (or SDK) can default these to on or off, as appropriate. For example, a Xamarin iOS application can safely turn COM support off. |
There was a problem hiding this comment.
Are the COM-related APIs kept around today even on Unix because of the S.P.Corelib ILLinkTrim.xml file? If yes, will that matter for the goaled-application types anyway, since they'll be targeting the mono runtime and its Corelib?
Separately, if we are thinking about coreclr as part of this effort, have we given any thought to how we might untangle the dependencies introduced by the native runtime pieces? We generate an ILLinkTrim.xml file that roots anything that might ever be used from native code, regardless of whether that code could actually end up being used.
There was a problem hiding this comment.
There are COM-only types outside of CoreLib:
have we given any thought to how we might untangle the dependencies introduced by the native runtime pieces?
My only thoughts have been the feature switch in the ILLinkTrim.xml file idea above that would allow application to remove them dynamically. Maybe @vitek-karas or @marek-safar have more thoughts here?
There was a problem hiding this comment.
any thought to how we might untangle the dependencies introduced by the native runtime pieces?
Lift more of the unmanaged logic into C# as necessary, use explicitly passed in callbacks instead of calling functions by name from VM.
There was a problem hiding this comment.
have we given any thought to how we might untangle the dependencies introduced by the native runtime pieces?
It depends. The XML descriptors are actually a good fit for this. If we know the runtime code will be 99% time present we just add it to XML (e.g. string constructor) for the rest it's a combination of using DynamicDependencyAttribute (aka PreserveDependency) and for large chunks of native code moving them to manage for better likability.
There was a problem hiding this comment.
The XML descriptors are actually a good fit for this
I'm surprised by that. From what I've seen, the XML isn't a great fit because a non-trivial amount of what's automatically kept isn't actually used by any given app. I could be wrong, of course.
There was a problem hiding this comment.
by the native runtime pieces
I was specifically referring to core runtime dependencies. There is a certain amount of core types/methods which runtime always requires (e.g. System.Object, System.Exception, etc) to even start. I don't think we need to make that conditional and listing such core dependencies in S.P.C XML descriptor is fine.
There was a problem hiding this comment.
Sure, there's a subset that's always used.
eerhardt
force-pushed
the
AddLinkingLibrariesDoc
branch
from
25ce0d6
to
d295fec
Compare
No description provided.