Remove DroidDoc dependencies from enumification helpers.

[atsushieno]

api-*.xml.in can be generated dynamically (up to [1]) but enumification
helper tools still depended on DroidDoc, meaning that we cannot remove
package download dependencies. This fixes the situation by eliminating
the dependencies. Especially it is easier to just parse api-.xml.in to
extract int consts from Java API.

... thought so? It turnd out to not be that easy.

Unlike DroidDoc, api-*.xml.in does not contain "api-since" information,
so we need to parse ALL the API XML. And they cannot be precise
because we only have a subset of the API definitions. For example,
"since API Level 1" now becomes "since API Level 10" because we don't
have api-1.xml.in (not even api-4.xml.in). So they will become non-precise.

(A slightly better possibility to "fix" this is to parse
android-sdk-whatever/platform-tools/api/api-versions.xml, but it won't
contain "already vanished" constants so it will be incomplete either.
I didn't bother to do "better" so far. We don't need that for enumification.)

Therefore, there is a lot of changes in map.csv to reflect those
"insignificant" changes to remap "API since" column (e.g. from "1" to "10").

Regeneration from api-LEVEL.xml.in instead of DroidDoc has some other
side effects; some consts are back to the list so that we don't have to
manually copy existing mapped consts that could not be generated from
the latest DroidDocs (due to disappearance) anymore. Instead we have to
remove manually-mapped constants (either from the sources or map.csv).
Therefore, a handful of heading lines in map.csv were removed, and
some definitions in enum-conversion-mappings.xml are back.

[*1] #1038

[jonpryor]

Therefore, there is a lot of changes in map.csv to reflect those
"insignificant" changes to remap "API since" column (e.g. from "1" to "10").

I don't understand this: generator uses integer < for version comparisons, not string comparisons or equality comparisons when parsing CSV files and the API level for an enum mapping.

Why did these lines need to change at all?

[jonpryor]

Why did this "block" of mappings need to be moved, or otherwise touched? It makes for a noisier diff.

[atsushieno]

Those lines above around line # 100 were manually added because they could not be automatically generated (because, they didn't exist in DroidDoc anymore!).

Now we generate enum const list based on api-*.xml.in they can be automatically generated again and therefore they went below # 100. It is a good thing!

[jonpryor]

As per a previous comment, I don't understand why the first column here actually needed to change.

[jonpryor]

Do you still need this debug spew?

[atsushieno]

It's not a big deal, we (well most likely only I) use it only when we are going to deal with new API for enumification.

[jonpryor]

For example,
"since API Level 1" now becomes "since API Level 10" because we don't
have api-1.xml.in (not even api-4.xml.in). So they will become non-precise.

I believe that this is a non-issue, as we only generate [Register (..., ApiSince=1)] if the type was never seen before. As a consequence, all types introduced in API-10 have ApiSince=1, as we don't actually know anything before API-10.

We don't currently emit [Register (..., ApiSince=4)] anywhere.

In a post PR #1032 world, we'll replace all instances of [Register (..., ApiSince=11) with [Register (..., ApiSince=1)], and that will be fine.

[jonpryor]

It would be nice if the documentation could be updated to explain the use/workflow of the XML files that generate-const-list-2.exe creates.

[atsushieno]

Why did these lines need to change at all?

From the "data" perspective they don't have to change, but for "tools" perspective there is nothing else we CAN do. There is only filename that could give us API level information in api-*.xml.in. And we don't have api-1.xml.in, api-4.xml,in, api-20.xml.in etc.

[atsushieno]

In a post PR #1032 world, we'll replace all instances of [Register (..., ApiSince=11) with [Register (..., ApiSince=1)], and that will be fine.

No, that's not going to happen because we still don't have api-1.xml.in.

[atsushieno]

It would be nice if the documentation could be updated to explain the use/workflow of the XML files that generate-const-list-2.exe creates.

Good call! It will be updated.

[jonpryor]

@atsushieno wrote:

In a post PR #1032 world, we'll replace all instances of [Register (..., ApiSince=11) with [Register (..., ApiSince=1)], and that will be fine.

No, that's not going to happen because we still don't have api-1.xml.in.

That doesn't matter. We currently emit ApiSince=1, and we currently do not have an api-1.xml.in file:

$ grep -r -E '\[Register.*ApiSince = 1\)' src/Mono.Android/obj/Debug/android-27/mcw
src/Mono.Android/obj/Debug/android-27/mcw/Android.App.DatePickerDialog.cs:              [Register ("android/app/DatePickerDialog$OnDateSetListener", "", "Android.App.DatePickerDialog/IOnDateSetListenerInvoker", ApiSince = 1)]
src/Mono.Android/obj/Debug/android-27/mcw/Android.App.KeyguardManager.cs:               [Register ("android/app/KeyguardManager$OnKeyguardExitResult", "", "Android.App.KeyguardManager/IOnKeyguardExitResultInvoker", ApiSince = 1)]
...

There are 464 instances of ApiSince = 1 in the code generated for API-27, and those are obviously not coming from an api-1.xml.in.

They are instead coming from $(AndroidSdkDirectory)/platform-tools/api/api-versions.xml, as our generator invocation uses --apiversions, which parses the above file to extract ApiSince values:

        <class name="android/app/DatePickerDialog$OnDateSetListener" since="1">
                <extends name="java/lang/Object" />
                <method name="onDateSet(Landroid/widget/DatePicker;III)V" />
        </class>

---

Again, I reiterate: I do not believe that src/Mono.Android/map.csv needed to change at all. If it does need to change, I do not understand why.

[atsushieno]

You don't really understand what I have been writing.

map.csv is NOT generated BY generator.

It WAS generated by scraping DroidDoc which HAD "since" section.

api-*.xml.in does NOT have that information.

There is NO SOURCE that retrieves that information.

Note that api-versions.xml does not contain any information about old API.

[jonpryor]

You don't really understand what I have been writing.

Indeed.

map.csv is NOT generated BY generator.

This implies that map.csv is generated. I was not aware it was generated; I thought it was manually maintained.

The first comment in map.csv doesn't help matters, either:

// this cannot be automatically generated, too exceptional...

If map.csv is generated, then could it please follow some semblance of convention and start with a comment containing the command used to generate the file? Or at least an indication that this is a generated file, and where one should go to properly update it, if only so I don't go manually updating that file in the future?

If map.csv isn't generated, then I'm really confused...

[atsushieno]

Everyone can track git history and see how those things are made changes. Did you add "it is automatically generated" comments on EVERYTHING you make it to generate? Which is automatically generated for Configuration.*.props, C# sources and Java sources, timestamp files etc. etc.? No we don't.

If you want to add those lines you can create a PR and ask me for merging permission.

[jonpryor]

Everyone can track git history and see how those things are made changes.

No, I can't. Git history isn't going to contain what commands were used to generate the changes, or when those commands were executed, or what the implications are. That's what documentation is for (previously requested).

I've been under the assumption that everything in src/Mono.Android/map.csv is manually maintained, full stop. I have no idea how the other tools fit together, or what parts are hand-written, and what parts aren't, or how they're merged, or...

All I see from git history and PRs is that map.csv changed. That's all.

Did you add "it is automatically generated" comments on EVERYTHING you make it to generate?

Good point. I should update everything to do so.

[atsushieno]

The updated documentation is in this revised PR.

[atsushieno]

It is going to be unnecessary once PR #1045 gets merged as it contains this change too.

[atsushieno]

Superceded by #1045