Add EU wide Traffic Data via DataEX2

[sauerbread]

Hi everyone,
so in a quest for providing traffic info for organic maps Ive mapped out pretty much all 27 memberstates + Norway, Iceland, UK and New Zealand (yes unrelated but, idk.). These are either state or state-owned Sites/Apps/APIs which could be borrowed NewPipe Extractor style, but after adding Estonia Ive discovered Datex2. Datex2 is pretty much a glorified XML Data Exchange Format that is provided through National Access Points (NAP) and soon to be released NAPCore. These NAPs provide a variety of Data Sources from state owned or third party sources, including Traffic Data from goverment agencies (TomTom & Here try really hard, but there are free sources right beneath it). So as an app developer we need to "subscribe" to these data sources for an API key. In general the data is free in some cases, e.g. Swiss, there might be quota limits. In the docs, Datex2 convienently suggests to bind to the data via JAXB. The EU generously provides a full list of participating Countries with others to follow. So either we could "gently" scrape the data from the mapped websites or we use Datex2. What are your thoughts?

TL;DR: Free EU Traffic Data from Official Sources for us to use.

[sauerbread]

Also skimming through the docs there is even more data available than just plain traffic data. There is also parking, service information, electromobility infrastructure, traffic regulations and more available.

[biodranik]

That looks promising, thanks for your investigation. It is obvious that to properly implement traffic info for our users we need a backend server that will proxy and cache responses to users' requests. In this case, we can stay within allowed limits and reduce expenses.

The implementation details will be hidden on the server, so probably it should be possible to mix different approaches to collect data from different sources.

What do you think? Any ideas or details on the implementation/architecture?
Also, am I right that it is not required to contribute back users' tracks in realtime?

[TobiPeterG]

@biodranik Thank you for your response :)
The only easy accessible source I found is from TomTom and their API is paid and proprietary.
I also registered at mobilithek.info since I'm German and I would like to test it, but the available sources are all only for specific regions and it is not easy to tell how often they get updated.
Maybe we should create our own service, where organisations can upload their traffic data to and create a hub on our own. We could then feed it with other sources as well.
France seems to have a centralized data source for example.
We could then use it in organic maps and also allow users to report traffic jams (or opt-in for automatic reporting?). We could also try to get other projects liks OSMand to use it and we could build an open service for traffic data :)

I guess a first step would be to create a prototype server that reports traffic jams everywhere or at one specific place to check if it works.

What do you think?

EDIT: Maybe we could build on top of this?
https://github.com/opentraffic/otv2-platform

[biodranik]

...organisations discover OM and contribute there traffic info regularly.

Sounds like a dream. It makes sense to focus on listing and retrieving data from existing sources, and converting it into client-friendly format. Adding there data from clients can be done as the next step.

So yes, building a server side part. Or finding and reusing something that works and fits well.

[TobiPeterG]

I've created an initial README summarizing the previous discussions
https://github.com/TobiPeterG/OSMTraffic

Please have a look and check if the points are valid and if I should change or add something :)

[TobiPeterG]

I created an initial server in go that can be built with docker-compose:
https://github.com/TobiPeterG/OSMTraffic-server

It uses the Autobahn GmbH APi to get traffic information in Germany and their response is reformated to GEOJson.
Please tell me if this is the desired format that OM could use and how OM would request the traffic info (e.g. coordinates?)

[dryusername]

Hi, the Swedish public transport agencies have a project that is related: https://www.trafiklab.se/

They also collaborate with other EU countries to make a universal and open source traffic system with live positioning of all EU public transit vehicles in real time.

Would be great if it could be implemented into the app 😊

[TobiPeterG]

I started working on the server side, however, this will a) still take quite some time and b) will not cover a lot, especially in the beginning.
A thought I had today was that maybe it would be useful to implement the TomTom traffic API for now and offer it in a "subscription" version of the app, with the only difference being access to live traffic data. This way, we can already make the app ready for receiving and reporting traffic data, without the open traffic Server being ready and maybe have them in parallel later on, with the open traffic being available to everyone. The subscription could be e.g. 2€/month to cover the cost and support the development of the traffic API. This way, the app would get way more useful for car navigation by having live traffic data and the development of an open alternative would still be supported. :)

Another unrelated thought would be to allow users to have a "bounty" program to fix bugs/implement features, e.g. offering 20€ for fixing bug x, implementing feature y. This would, of course, be optional, but might attract new contributors and motivate overall.
What are your thoughts on this?