-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AndroidAnnotationSupport needs better invalid XML fixup #883
Labels
enhancement
Proposed change to current functionality
generator
Issues binding a Java library (generator, class-parse, etc.)
Comments
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this issue
Sep 20, 2021
Context: dotnet#6300 Context: dotnet/java-interop#883 Changes: http://github.com/xamarin/xamarin-android-tools/compare/9b658b29bd41157151f5515619d0d90dc062563d...a5194e93498e7f12225d87e2811415a45f742116 * dotnet/android-tools@a5194e9: [Xamarin.Android.Tools.AndroidSdk] Downgrade build-tools to API-30 * dotnet/android-tools@440e6be: [Xamarin.Android.Tools.AndroidSdk] Update SDK component for API-31 (dotnet#134) Note: while this xamarin-android-tools bump updates the Android SDK [Platform Tools][0] version to 31.0.3, we are *not* updating `$(XAPlatformToolsVersion)` to 31.0.3. As such, there will be a mismatch between what we build xamarin-android against, vs. the default suggested platform-tools package potentially installed via the `/t:InstallAndroidDependencies` target. The reason to *not* bump `$(XAPlatformToolsVersion)` is that attempting to do so breaks the `src/Mono.Android` build: 1. platform-tools r31.0.3 doesn't contain `platform-tools/api/annotations.zip`, which is used by `generator` to emit certain custom attributes such as `RequiresPermission`. 2. While (1) can be worked around by instead using `$(AndroidSdkDirectory)/platforms/android-*/data/annotations.zip`, this introduces API changes reported by the `_CheckApiCompatibility` target, in particular changes due to custom attribute string changes. These string changes happen because Google ships *invalid XML* in `data/annotations.zip` for API-29+ (?!): <!-- annotations.zip!android/accounts/annotations.xml --> - <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> + <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> `AccountManagerCallback<android.os.Bundle>` is not a valid value within an XML attribute. This change doesn't break the build, but instead causes [`AndroidAnnotationSupport`][1] to hit a "fallback" codepath which uses HtmlAgilityPack, and this causes attribute values to "gain" `quot;`: -[global::Android.Runtime.RequiresPermission ("android.permission.MANAGE_ACCOUNTS")] +[global::Android.Runtime.RequiresPermission ("quot;android.permission.MANAGE_ACCOUNTS"")] See also: dotnet/java-interop#883 After manual review, no other file removals appear to be problematic, so @jonpryor asserts that it should be acceptable for `$(XAPlatformToolsVersion)` and `$(AndroidSdkPlatformToolsVersion)` to be inconsistent with each other. [0]: https://developer.android.com/studio/releases/platform-tools [1]: https://github.com/xamarin/java.interop/blob/1e8f5137345db3160c99265ff3a56c43a132194f/src/Xamarin.Android.Tools.AnnotationSupport/AndroidAnnotationsSupport.cs#L58-L87
jonpryor
added a commit
to dotnet/android
that referenced
this issue
Sep 20, 2021
Context: #6300 Context: dotnet/java-interop#883 Changes: http://github.com/xamarin/xamarin-android-tools/compare/9b658b29bd41157151f5515619d0d90dc062563d...a5194e93498e7f12225d87e2811415a45f742116 * dotnet/android-tools@a5194e9: [Xamarin.Android.Tools.AndroidSdk] Downgrade build-tools to API-30 * dotnet/android-tools@440e6be: [Xamarin.Android.Tools.AndroidSdk] Update SDK component for API-31 (#134) Note: while this xamarin-android-tools bump updates the Android SDK [Platform Tools][0] version to 31.0.3, we are *not* updating `$(XAPlatformToolsVersion)` to 31.0.3. As such, there will be a mismatch between what we build xamarin-android against, vs. the default suggested platform-tools package potentially installed via the `/t:InstallAndroidDependencies` target. The reason to *not* bump `$(XAPlatformToolsVersion)` is that attempting to do so breaks the `src/Mono.Android` build: 1. platform-tools r31.0.3 doesn't contain `platform-tools/api/annotations.zip`, which is used by `generator` to emit certain custom attributes such as `RequiresPermission`. 2. While (1) can be worked around by instead using `$(AndroidSdkDirectory)/platforms/android-*/data/annotations.zip`, this introduces API changes reported by the `_CheckApiCompatibility` target, in particular changes due to custom attribute string changes. These string changes happen because Google ships *invalid XML* in `data/annotations.zip` for API-29+ (?!): <!-- annotations.zip!android/accounts/annotations.xml --> - <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> + <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> `AccountManagerCallback<android.os.Bundle>` is not a valid value within an XML attribute. This change doesn't break the build, but instead causes [`AndroidAnnotationSupport`][1] to hit a "fallback" codepath which uses HtmlAgilityPack, and this causes attribute values to "gain" `quot;`: -[global::Android.Runtime.RequiresPermission ("android.permission.MANAGE_ACCOUNTS")] +[global::Android.Runtime.RequiresPermission ("quot;android.permission.MANAGE_ACCOUNTS"")] See also: dotnet/java-interop#883 After manual review, no other file removals appear to be problematic, so @jonpryor asserts that it should be acceptable for `$(XAPlatformToolsVersion)` and `$(AndroidSdkPlatformToolsVersion)` to be inconsistent with each other. [0]: https://developer.android.com/studio/releases/platform-tools [1]: https://github.com/xamarin/java.interop/blob/1e8f5137345db3160c99265ff3a56c43a132194f/src/Xamarin.Android.Tools.AnnotationSupport/AndroidAnnotationsSupport.cs#L58-L87
jonathanpeppers
pushed a commit
to dotnet/android
that referenced
this issue
Sep 21, 2021
Context: #6300 Context: dotnet/java-interop#883 Changes: http://github.com/xamarin/xamarin-android-tools/compare/9b658b29bd41157151f5515619d0d90dc062563d...a5194e93498e7f12225d87e2811415a45f742116 * dotnet/android-tools@a5194e9: [Xamarin.Android.Tools.AndroidSdk] Downgrade build-tools to API-30 * dotnet/android-tools@440e6be: [Xamarin.Android.Tools.AndroidSdk] Update SDK component for API-31 (#134) Note: while this xamarin-android-tools bump updates the Android SDK [Platform Tools][0] version to 31.0.3, we are *not* updating `$(XAPlatformToolsVersion)` to 31.0.3. As such, there will be a mismatch between what we build xamarin-android against, vs. the default suggested platform-tools package potentially installed via the `/t:InstallAndroidDependencies` target. The reason to *not* bump `$(XAPlatformToolsVersion)` is that attempting to do so breaks the `src/Mono.Android` build: 1. platform-tools r31.0.3 doesn't contain `platform-tools/api/annotations.zip`, which is used by `generator` to emit certain custom attributes such as `RequiresPermission`. 2. While (1) can be worked around by instead using `$(AndroidSdkDirectory)/platforms/android-*/data/annotations.zip`, this introduces API changes reported by the `_CheckApiCompatibility` target, in particular changes due to custom attribute string changes. These string changes happen because Google ships *invalid XML* in `data/annotations.zip` for API-29+ (?!): <!-- annotations.zip!android/accounts/annotations.xml --> - <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> + <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> `AccountManagerCallback<android.os.Bundle>` is not a valid value within an XML attribute. This change doesn't break the build, but instead causes [`AndroidAnnotationSupport`][1] to hit a "fallback" codepath which uses HtmlAgilityPack, and this causes attribute values to "gain" `quot;`: -[global::Android.Runtime.RequiresPermission ("android.permission.MANAGE_ACCOUNTS")] +[global::Android.Runtime.RequiresPermission ("quot;android.permission.MANAGE_ACCOUNTS"")] See also: dotnet/java-interop#883 After manual review, no other file removals appear to be problematic, so @jonpryor asserts that it should be acceptable for `$(XAPlatformToolsVersion)` and `$(AndroidSdkPlatformToolsVersion)` to be inconsistent with each other. [0]: https://developer.android.com/studio/releases/platform-tools [1]: https://github.com/xamarin/java.interop/blob/1e8f5137345db3160c99265ff3a56c43a132194f/src/Xamarin.Android.Tools.AnnotationSupport/AndroidAnnotationsSupport.cs#L58-L87
jpobst
added
enhancement
Proposed change to current functionality
generator
Issues binding a Java library (generator, class-parse, etc.)
labels
Sep 27, 2021
jonpryor
pushed a commit
that referenced
this issue
Oct 26, 2021
Fixes: #883 Due to invalid XML provided in Google's `annotations.zip` file, we run it through the more forgiving `HtmlAgilityPack` (4073f3e) to attempt to fix it to valid XML. However, given this snippet: <item name="android.accounts.AccountManager android.accounts.AccountManagerFuture<android.os.Bundle> addAccount(java.lang.String, java.lang.String, java.lang.String[], android.os.Bundle, android.app.Activity, android.accounts.AccountManagerCallback<android.os.Bundle>, android.os.Handler)"> <annotation name="androidx.annotation.RequiresPermission"> <val name="value" val=""android.permission.MANAGE_ACCOUNTS"" /> <val name="apis" val=""..22"" /> </annotation> </item> The invalid unescaped `<` and `>` characters in the `//item/@name` attribute seem to tell `HtmlAgilityPack` not to expect any attribute strings to be properly escaped. Thus when it gets to the `//val/@val` attributes, it treats `"` as unescaped as well, thinking we want to value to be the literal string `"""`. When it writes out the valid XML, it realizes it needs to escape the ampersand, and writes out `&quot;`, which breaks our usage of these annotations. The fix is to "unescape" every `"` to a `"` so that it will be escaped correctly when saved as valid XML.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
enhancement
Proposed change to current functionality
generator
Issues binding a Java library (generator, class-parse, etc.)
Context: https://issuetracker.google.com/issues/116182838
Context: 4073f3e
In 2018-September, Google started providing various XML files which were not valid XML. We worked around this in commit 4073f3e by using HtmlAgilityPack to "loosely parse" the XML.
The problem with commit 4073f3e is that the resulting "intermediate" XML isn't "the same". Given XML of:
Consider this fragment:
When run, we get:
Note how the
Values
value changes, fromandroid.permission.MANAGE_ACCOUNTS
to"android.permission.MANAGE_ACCOUNTS"
. This value eventually becomes theRequiresPermission
attribute value, resulting in "downstream"Mono.Android.dll
changes:which results in hundreds of "API changes" reported by the
_CheckApiCompatibility
target.For Great Sanity™, we want the "invalid XML" path to produce the same value as the "valid XML" path, especially considering that the
//item/annotation/val/@val
attribute value is the same.The text was updated successfully, but these errors were encountered: