RFC: Basic NOBIL implementation #363
Conversation
robho
force-pushed
the
nobil
branch
2 times, most recently
from
47f0131
to
80cdd5e
Compare
|
Very nice, thanks a lot! I'll try to find some time in the weekend to try it out and provide some help with code cleanup.
Yeah, if there are properties that don't fit into EVMap's data structure we can think about adding new fields or expanding existing ones where it is useful. For payment methods, we have the
As previously mentioned, I guess all the filters would have to be implemented locally, as the Nobil API doesn't really have server-side filtering abilities, right? That should be possible (just like the min_connectors filter for GE and OCM), but in the longer term the better solution is probably to first download all chargers into the local DB (which is a capability I'm implementing for #290) and perform filtering there.
Chargeprice currently only support GoingElectric and OpenChargeMap unfortunately. And by now they have also created their own separate database of charging stations aggregated from different sources, so from what I heard they are not so interested in fixing issues with their adapters for GE and OCM, let alone adding new ones 🫤.
Likely yes, I can test that as well.
True, that is probably a general bug - for GoingElectric it's not so relevant as we are only allowed to cache the data for 24h anyway. But yeah, it probably also applies to OpenChargeMap. Let's track that in a separate issue. |
app/src/main/java/net/vonforst/evmap/api/nobil/NobilAdapters.kt
Outdated
Show resolved
Hide resolved
| referenceData: ReferenceData, | ||
| id: Long | ||
| ): Resource<ChargeLocation> { | ||
| // TODO: Nobil ids are "SWE_1234", not Long |
There was a problem hiding this comment.
Hmm, it seems a bit strange that Nobil has two different types of IDs, the id (integer) and the International_id (string with country prefix and same integer). First I thought that the IDs are assigned separately in each country, so only the International_id is guaranteed to be unique, not the id (and in that case, we would have to use the International_id also for storing in our local database - so the Long field would have to be changed into a String with corresponding DB migration).
On the other hand, according to the API docs, the existingids parameter in API requests expects the "non-international" ID - this would be a bit strange if that ID could appear in multiple countries, right?
There was a problem hiding this comment.
I dumped data from Finland, Norway and Sweden and checked the id of all chargers -> the id field is unique across countries. I don't understand the need for two ids either..
I also verified that sending this get-charger-details-request really does require the International_id. No data is returned if the id is sent in the request.
One question I have here is whether this function is used/needed for nobil. All data that nobil returns in responses is "detailed" so is there a need to implement this function?
Thanks for your input and review! I'll go through it all, but could take some time, depending on the time I have to look into it.
| // 24: Open 24h | ||
| if (chargerStationAttributes.st["24"]?.attrTrans == "Yes") OpeningHours(twentyfourSeven = true, null, null) else null, | ||
| // 7: Parking fee | ||
| if (chargerStationAttributes.st["7"]?.attrTrans == "No") Cost(freeparking = true) else null, |
There was a problem hiding this comment.
If the attribute is set to "Yes" (i.e. known to have a parking fee), you could also set it to Cost(freeparking = false). It should only be set to null if the parking fee is not known.
There was a problem hiding this comment.
Other fields from Nobil that might be relevant to the Cost - maybe these could be used to generate a description?
- Time limit
- Accessibility (does
Openmeanfreecharging?) - Payment Method
There was a problem hiding this comment.
Looks like I either add freecharging and freeparking or descriptions. If I set freecharging and freeparking then EVMap creates a text string for me and ignores the descriptions. Right?
There was a problem hiding this comment.
No, you can also set both - then freecharging/freeparking will be shown in the first line and the description as a smaller text below.
| listOfNotNull(chargerStationData.street, chargerStationData.houseNumber).joinToString(" ")), | ||
| chargerStationAttributes.conn.map { createChargepointFromNobilConnection(it.value) }, | ||
| null, | ||
| "", |
There was a problem hiding this comment.
Hmm, Nobil itself doesn't really have a web map intended for the general public, so we can't easily put a link there... Something like https://ladekart.elbil.no/ seems to use different IDs.
For the button at the very bottom ("Source: Nobil") it would probably still be good to have a working link (also to comply with the license terms), but that could just be the home page https://nobil.no. For the share button that of course wouldn't make sense. So maybe we have to add another field called dataSourceUrl (home page of the data source) and make the existing url (direct link to this charger at the data source) optional. Then the "Source" button would use url if available and fall back to dataSourceUrl, while the share button would be disabled if chargerUrl is not available.
There was a problem hiding this comment.
There's a web page to report errors for Swedish chargers (https://www.energimyndigheten.se/klimat/transporter/laddinfrastruktur/registrera-din-laddstation/elbilsagare/), but for nobil in general they want error reports by email.
Do you think it adds any value to add the web page link for Swedish chargers (I don't think the web page accepts any query parameters so it's not possible to pre-fill charger details)? Can something be done to send an email? Create a mailto link?
There was a problem hiding this comment.
Yeah, we could put this link as the editUrl for chargers in Sweden and use a mailto URL for the other countries. I'm not sure if Android will handle a mailto URL out of the box, but if not, we could parse it in the app and convert it into an email intent.
| chargerStationData.zipCode, | ||
| listOfNotNull(chargerStationData.street, chargerStationData.houseNumber).joinToString(" ")), | ||
| chargerStationAttributes.conn.map { createChargepointFromNobilConnection(it.value) }, | ||
| null, |
There was a problem hiding this comment.
Nobil seems to have both Operator and Owned_by fields - I think Nobil's Operator is what GoingElectric and EVMap call network and Nobil's Owned_by is what EVMap calls operator. In many cases they are the same (e.g., Ionity chargers are owned and operated by Ionity), but sometimes they are different (e.g., chargers on the streets of Munich are owned and operated by Stadtwerke München, the municipal energy supplier, but are part of the larger Ladenetz network who are responsible for the backend etc.).
The network field is more useful for filters, as there are less networks than operators and pricing usually depends on the network.
There was a problem hiding this comment.
Yes, your understanding is correct. Nobil's Operator = EVMap network and Nobil's Owned_by = EVMap operator.
There was a problem hiding this comment.
I extracted the network information available in nobil and there are ~1500 different "networks". The field is a text field and users have entered information in many different variations (different capitalization, typos, abbreviations, alternative names, "owned by" information, ...).
Even if I only parse operator from OCPI data I get ~60 "networks" and also here there are name variations for the same actual network and some names don't even correspond to a network.
Maybe we could create a network list manually and try to match charge locations to the list items. Not a great solution since it requires manual work to update the network list and it's hard to know which networks to include in the list, especially in "foreign" countries.
Maybe we have to live without a network filter, for now
There was a problem hiding this comment.
Ah 😟 Wow, strange that that happens even with OCPI data...
Yeah, then we probably have to live without a network filter for now, and hope that Nobil can improve the data quality in the future.
| null, | ||
| Instant.now(), | ||
| true | ||
| ) |
There was a problem hiding this comment.
Other than what we already have, I think the Availability attribute ( Public/Visitors/Employees/By Appointment/Residents) would be great to add and also have as a filter (at least as a binary filter - only publicly accessible or all)
| else -> null | ||
| } | ||
|
|
||
| return Chargepoint(connectionType, connectionPower, 1, null, null) |
There was a problem hiding this comment.
voltage and amperage can also be added (Attributes 12 and 31). It would also be very useful to add a field for the EVSEID (Attribute 28).
|
Regarding filtering..
Is your idea to replace the on-demand-loading with a full data fetch or are you thinking of having some combination of the two? The nobil data is ~130 Mbytes today (up ~6 Mbytes in ~2 months). This is uncompressed data, I haven't checked if the server can compresss data and what the compressed size is. I think I would prefer to use on-demand loading most of the time, but it would be valuable to have the possibility to cache all data when needed (no mobile connection or high mobile roaming prices, ...). Is the problem that you see with local filters that they need to fetch data for chargers that are uninteresting? Ie unnecessary network traffic and data processing? Are those "unnecessary" downloads cached even if they are not used? |
This reverts commit d9be059. It doesn't work to set data dataSource to "nobil.no". I guess that the period in the string confuses EVMap. Maybe related to the db?
nobil has data on biogas and hydrogen "chargers", but we don't want them in EVMap
Gzip usually helps quite a lot with JSON data, and the Nobil server does seem to support it. According to the download size is pretty small at just ~5 MB. Of course it might occupy a bit more space in the local database (and from my experience with the implementation for OSM, parsing the JSON and inserting chargers into the DB also takes much more time than the download itself).
In general, EVMap will continue to support both. So we could offer both options for Nobil. On the other hand, with such a small download size, there's not much of a downside to storing it all locally - might also reduce the load on Nobil's servers in the long term.
Yeah. I think the data processing (JSON parsing into Kotlin objects, then filtering) is usually the main bottleneck when loading a large map region with thousands or even ten-thousands of chargers - server-side filtering and even server-side clustering makes it a lot faster for GoingElectric where that is supported. Also, depending on what filters we implement, there might be some filters where we can only determine the available options by iterating through the whole dataset once (e.g., the network filter mentioned in the comments) and then storing that as
not at the moment, because local filtering happens inside the API implementation |
This adds a NOBIL data source. Data is available in Denmark, Finland, Iceland, Norway and Sweden and the data is collected both from operators via OCPI and contributed by the public.
This is a draft implementation where some functionality is missing. I'm interested in hearing if this is of interest, and if so, what should I fix/work on going forward?
Known problems / missing features: