Convert Android Telemetry To V2 Data Model

[bleege]

The first iteration of Android Telemetry ( #2421) was built using the original Events data model (aka V1). V2 has now been internally approved and will what is used going forward. Android needs to be converted to use this data model so that it only uses V2 from Telemetry's initial release in 4.0.0..

/cc @mapbox/telemetry

[bleege]

While implementing isPluggedIn() for the pluggedIn attribute I found that the ideal API in Android is BatteryManager.ACTION_CHARGING. However it's API 23+ which means we'll likely have to rely on a combination of BATTERY_PLUGGED_AC, BATTERY_PLUGGED_USB, and BATTERY_STATUS_CHARGING.

[bleege]

I decided to go with reusing the ACTION_BATTERY_CHANGED Intent that's already being used for determining battery level to determine if the device is currently connected or not as described in Google's documentation instead of implementing the more sophisticated ACTION_POWER_CONNECTED and ACTION_POWER_DISCONNECTED as it required using a standalone BroadcastReceiver. This may change in the future however.

I also limited the isPluggedIn() determination to BATTERY_PLUGGED_USB and BATTERY_PLUGGED_AC even though according to the documentation it'd likely be safe to assume anything greater than 0 for this value would indicate being plugged in. The value being "knowing for sure" over "not knowing for sure, but pretty likely". BATTERY_PLUGGED_WIRELESS was not used as it is API 17+ while the SDK currently supports 15+. Again, these things may change in the future.

[bleege]

The basic refactor to V2 is complete after some API changes with @camilleanne @mikemorris @mick this afternoon. @camilleanne is working on updating the validation logic on the server to match our changes. When she's done we'll be able to field test the new Telemetry data collection setup.

[bleege]

I did some testing this morning now that the Staging server's validation logic was updated last night. The first log block shows 2 events successfully being received by the staging server (aka HTTP 204 response). These events are from app launch and will be the appTurnstile and the map.load events. However when I try to confirm their existence in the database they don't appear. I'll debug this with @camilleanne later today.

02-24 09:41:41.575 8012-8055/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 204 for 2 events sent in.
02-24 09:41:50.489 8012-8234/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: No events in the queue to send so returning.

For the second batch of events, 17 in total, they were denied by the server (aka HTTP 422 response) so I don't expect them in the database. I'll work with @camilleanne to get some insight as to what got tripped up in the logs.

02-24 09:43:50.499 8012-9997/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 09:43:51.261 8012-9997/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 422 for 17 events sent in.

Please note the times are US Central timezone.

[bleege]

More test data to help track down the issue.... All going to V2 endpoint on Staging server.

02-24 11:21:26.572 11737-11786/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 204 for 2 events sent in.

App Startup Events

02-24 11:23:35.950 11737-13838/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 11:23:36.449 11737-13838/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 422 for 65 events sent in.

Location and Gestures

02-24 11:25:35.801 11737-15089/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 11:25:36.310 11737-15089/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 422 for 17 events sent in.

Location and Gestures

02-24 11:27:35.801 11737-16463/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 11:27:36.398 11737-16463/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 422 for 34 events sent in.

Location and Gestures

/cc @camilleanne

[bleege]

@camilleanne found some error logs confirming that boolean values aren't being sent in correctly (using yes and no instead of true and false) as well as needing to send in null values instead of Unknown. Will review data model, make adjustments, and test again.

[bleege]

Converted to boolean and sent the following in. Will need log statements to confirm what's next to fix.

02-24 11:45:06.489 17995-18251/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 11:45:07.275 17995-18251/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 422 for 5 events sent in.

map.load, Location, and map.click

[bleege]

Next round with fixed Create Date in gesture events. Looking promising, but need to confirm.

02-24 11:57:55.200 20075-20141/? I/MapboxEventManager: Response Code from Mapbox Events Server: 204 for 2 events sent in.

App Startup

02-24 12:00:04.608 20075-22159/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: url = https://cloudfront-staging.tilestream.net/events/v2?access_token=sk.eyJ1IjoiYmxlZWdlIiwiYSI6InNpcml1c2x5In0.KyT-boMyC_xZYTYojTc8zg
02-24 12:00:05.906 20075-22159/com.mapbox.mapboxsdk.testapp I/MapboxEventManager: Response Code from Mapbox Events Server: 204 for 24 events sent in.

Gestures, Locations, etc

[bleege]

Confirmed appTurnstile and location events with latest code accepted and displayed in database.

[bleege]

Confirmed map.load event with same code as previous comment.

[bleege]

Confirmed map.click and map.dragend with new code that broke them apart. Registered all 8 types of Gestures for map.click.

Will do one final test for all events with code base before officially signing off on this.

[bleege]

Confirmed that all the events minus map.load made it into the server. I think this is a race condition as map.load and appTurnstile are tied together and appTurnstile flushes immediately. This is likely causing the event queue may be being cleared out after appTurnstile is sent but just after the time map.load is being added.

[bleege]

I confirmed with @camilleanne in chat that order is irrelevant for map.load and appTurnstile so I refactored the processing to do map.load first so that when appTurnstile flushes the queue it'll already be there and will get sent.

I think Android is ready to go with V2 now! 🎉

[boundsj]

@bleege @camilleanne thanks for confirming. iOS had already been changed to send the turnstile on map load. In a more recent commit, I've added an api client that make it easy to send the turnstile event immediately whenever a mapview event is about to be enqueued (but still only once per "session")

[zugaldia]

I think Android is ready to go with V2 now! 🎉

💥 💥 💥