Commit 77800dd
authored
[Java.Interop.Tools.Expressions] Add Java.Interop.Tools.Expressions (#1046)
Fixes: #616
Context: #14
Context: ff4053c
Context: da5d1b8
Context: 4787e01
Context: 41ba348
Remember `jnimarshalmethod-gen` (176240d)? And it's crazy idea to
use the `System.Linq.Expressions`-based custom marshaling
infrastructure (ff4053c, da5d1b8) to generate JNI marshal methods
at build/packaging time. And how we had to back burner it because
it depended upon `System.Reflection.Emit` being able to write
assemblies to disk, which is a feature that never made it to .NET Core,
and is still lacking as of .NET 7 (#616)?
Add `src/Java.Interop.Tools.Expressions`, which contains code which
uses Mono.Cecil to compile `Expression<T>` expressions to IL.
Then update `jnimarshalmethod-gen` to use it!
~~ Usage ~~
% dotnet bin/Debug-net7.0/jnimarshalmethod-gen.dll \
bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll \
-v -v --keeptemp \
--jvm /Library/Java/JavaVirtualMachines/microsoft-11.jdk/Contents/Home/lib/jli/libjli.dylib \
-o _x \
-L bin/TestDebug-net7.0 \
-L /usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.0
First param is assembly to process; `Java.Interop.Export-Tests.dll`
is handy because that's what the `run-test-jnimarshal` target in
`Makefile` processed.
* `-v -v` is *really* verbose output
* `--keeptemp` is keep temporary files, in this case
`_x/Java.Interop.Export-Tests.dll.cecil`.
* `--jvm PATH` is the path to the JVM library to load+use.
* `-o DIR` is where to place output files; this will create
`_x/Java.Interop.Export-Tests.dll`.
* `-L DIR` adds `DIR` to library resolution paths; this adds
`bin/TestDebug/net7.0` (dependencies of
`Java.Interop.Export-Tests.dll`) and
`Microsoft.NETCore.App/7.0.0-rc.1.22422.12` (net7 libs).
By default the directories containing input assemblies and the
directory containing `System.Private.CoreLib.dll` are part of the
default `-L` list.
When running in-tree, e.g. AzDO pipeline execution, when
`--jvm PATH` isn't specified, `jnimarshalmethod-gen`
will attempt to read the path in `bin/Build*/JdkInfo.props`
a'la `TestJVM` (002dea4). This allows an in-place update in
`core-tests.yaml` which does:
dotnet bin/Debug-net7.0/jnimarshalmethod-gen.dll \
bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll \
-v -v --keeptemp -o bin/TestDebug-net7.0
~~ Using `jnimarshalmethod-gen` output ~~
What does `jnimarshalmethod-gen` *do*?
% ikdasm bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll.orig > beg.il
% ikdasm bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll > end.il
% git diff --no-index beg.il end.il
# https://gist.github.com/jonpryor/b8233444f2e51043732bea922f6afc81
is a ~1KB diff which shows, paraphrasing greatly:
public partial class ExportTest {
partial class '__<$>_jni_marshal_methods' {
static IntPtr funcIJavaObject (IntPtr jnienv, IntPtr this) => …
// …
[JniAddNativeMethodRegistration]
static void __RegisterNativeMembers (JniNativeMethodRegistrationArguments args) => …
}
}
internal delegate long _JniMarshal_PP_L (IntPtr jnienv, IntPtr self);
// …
wherein `ExportTest._<$>_jni_marshal_methods` and the `_JniMarshal*`
delegate types are added to the assembly.
This also unblocks the desire stated in 4787e01:
> For `Java.Base`, @jonpryor wants to support the custom marshaling
> infrastructure introduced in 77a6bf8. This would allow types to
> participate in JNI marshal method ("connector method") generation
> *at runtime*, allowing specialization based on the current set of
> types and assemblies.
What can we do with this `jnimarshalmethod-gen` output? Use it!
First, make sure the tests work:
# do this *before* running above `dotnet jnimarshalmethod-gen.dll` command…
% dotnet test --logger "console;verbosity=detailed" bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll
…
Passed! - Failed: 0, Passed: 17, Skipped: 0, Total: 17, Duration: 103 ms - Java.Interop.Export-Tests.dll (net7.0)
Then after running the above `dotnet jnimarshalmethod-gen.dll` command,
re-run the tests:
% dotnet test --logger "console;verbosity=detailed" bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll
…
Total tests: 17
Passed: 17
`core-tests.yaml` has been updated to do this.
~~ One-Off Tests ~~
One-off tests: ensure that the generated assembly can be decompiled:
% ikdasm bin/TestDebug-net7.0/Java.Interop.Tools.Expressions-Tests-ExpressionAssemblyBuilderTests.dll
% monodis bin/TestDebug-net7.0/Java.Interop.Tools.Expressions-Tests-ExpressionAssemblyBuilderTests.dll
% ikdasm _x/Java.Interop.Export-Tests.dll
% monodis _x/Java.Interop.Export-Tests.dll
# which currently fails :-(
Re-enable most of `Java.Interop.Export-Tests.dll` for .NET 7;
see 41ba348, which disabled those tests.
To verify the generated IL, use the [dotnet-ilverify][0] tool:
dotnet tool install --global dotnet-ilverify
Usage of which is "weird":
$HOME/.dotnet/tools/ilverify bin/TestDebug-net7.0/Java.Interop.Export-Tests.dll \
--tokens --system-module System.Private.CoreLib \
-r 'bin/TestDebug-net7.0/*.dll' \
-r '/usr/local/share/dotnet/shared/Microsoft.NETCore.App/7.0.0/*.dll'
All Classes and Methods in /Volumes/Xamarin-Work/src/xamarin/Java.Interop/_x/Java.Interop.Export-Tests.dll Verified.
# no errors!
where:
* `--tokens`: Include metadata tokens in error messages.
* `--system-module NAME`: set the "System module name". Defaults
to `mscorlib`, which is wrong for .NET 5+, so this must be set to
`System.Private.CoreLib` (no `.dll` suffix!).
* `-r FILE-GLOB`: Where to resolve assembly references for the
input assembly. Fortunately file globs are supported…
~~ Removing `System.Private.CoreLib` ~~
`System.Private.CoreLib.dll` is *private*; it's not part of the
public assembly surface area, so you can't use
`csc -r:System.Private.CoreLib …` and expect it to work. This makes
things interesting because *at runtime* everything "important" is in
`System.Private.CoreLib.dll`, like `System.Object`.
Specifically, if we do the "obvious" thing and do:
newTypeDefinition.BaseType = assemblyDefinition.MainModule
.ImportReference (typeof (object));
you're gonna have a bad type, because the resulting IL for
`newTypeDefinition` will have a base class of
`[System.Private.CoreLib]System.Object`, which isn't usable.
Fix this by:
1. Writing the assembly to a `Stream`.
2. Reading the `Stream` from (1)
3. Fixing all member references and assembly references so that
`System.Private.CoreLib` is not referenced.
If `jnimarshalmethod-gen.dll --keeptemp` is used, then a `.cecil`
file is created with the contents of (1).
Additionally, and unexpectedly -- [jbevain/cecil#895][1] -- Mono.Cecil
adds a reference to the assembly being modified. Remove the declaring
assembly from `AssemblyReferences`.
[0]: https://www.nuget.org/packages/dotnet-ilverify
[1]: jbevain/cecil#8951 parent bbaeda6 commit 77800dd
File tree
18 files changed
+2134
-124
lines changed- build-tools/automation/templates
- src
- Java.Interop.Export
- Java.Interop
- Java.Interop.Tools.Expressions
- Java.Interop.Tools.Expressions
- tests
- Java.Interop.Export-Tests/Java.Interop
- Java.Interop.Tools.Expressions-Tests
- Java.Interop.Tools.ExpressionsTests
- tools/jnimarshalmethod-gen
18 files changed
+2134
-124
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
76 | 75 | | |
77 | 76 | | |
78 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
112 | 116 | | |
113 | 117 | | |
114 | 118 | | |
| |||
308 | 312 | | |
309 | 313 | | |
310 | 314 | | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
311 | 323 | | |
312 | 324 | | |
313 | 325 | | |
| |||
360 | 372 | | |
361 | 373 | | |
362 | 374 | | |
| 375 | + | |
| 376 | + | |
363 | 377 | | |
364 | 378 | | |
365 | 379 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
| |||
33 | 35 | | |
34 | 36 | | |
35 | 37 | | |
36 | | - | |
| 38 | + | |
37 | 39 | | |
38 | 40 | | |
39 | 41 | | |
| |||
43 | 45 | | |
44 | 46 | | |
45 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
46 | 52 | | |
47 | 53 | | |
48 | 54 | | |
| |||
127 | 133 | | |
128 | 134 | | |
129 | 135 | | |
130 | | - | |
| 136 | + | |
131 | 137 | | |
132 | 138 | | |
133 | 139 | | |
| |||
150 | 156 | | |
151 | 157 | | |
152 | 158 | | |
153 | | - | |
| 159 | + | |
154 | 160 | | |
155 | 161 | | |
156 | | - | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
157 | 168 | | |
158 | | - | |
159 | | - | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
160 | 178 | | |
161 | | - | |
| 179 | + | |
| 180 | + | |
162 | 181 | | |
163 | 182 | | |
164 | 183 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
108 | | - | |
| 108 | + | |
109 | 109 | | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
115 | 133 | | |
116 | 134 | | |
117 | 135 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6408 | 6408 | | |
6409 | 6409 | | |
6410 | 6410 | | |
6411 | | - | |
| 6411 | + | |
6412 | 6412 | | |
6413 | 6413 | | |
6414 | 6414 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
Lines changed: 9 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | 87 | | |
102 | 88 | | |
103 | 89 | | |
| |||
242 | 228 | | |
243 | 229 | | |
244 | 230 | | |
| 231 | + | |
245 | 232 | | |
246 | 233 | | |
247 | 234 | | |
| |||
277 | 264 | | |
278 | 265 | | |
279 | 266 | | |
| 267 | + | |
280 | 268 | | |
281 | 269 | | |
282 | 270 | | |
| |||
290 | 278 | | |
291 | 279 | | |
292 | 280 | | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
293 | 285 | | |
294 | 286 | | |
295 | 287 | | |
| |||
307 | 299 | | |
308 | 300 | | |
309 | 301 | | |
310 | | - | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
311 | 305 | | |
312 | 306 | | |
313 | 307 | | |
| |||
Lines changed: 27 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
0 commit comments