Question: Stability of systems.csv?

[barbeau]

It seems like systems.csv would be a nice source for discovery of GBFS-compatible systems.

Questions:

• How stable is the structure of this file? Safe to query programmatically via https://raw.githubusercontent.com/NABSA/gbfs/master/systems.csv without fear of field names and/or column order changing?
• Is System ID guaranteed to be unique?
• Is anyone else currently known to be querying this programmatically?

[jcn]

Early in the discussions, it was decided that the maintainers of the spec did not want to take on the burden of ensuring the uptime of a global GBFS registry. This has a couple of implications:

1. As per the spec the system_id should be globally unique, but it is up to the provider to ensure uniqueness
2. The systems.csv file is provided as a convenience, but as it is not technically part of the spec, its stability is not guaranteed
3. systems.csv is less than 2 months old, so we don't actually know if anyone else is querying it programmatically - I'd love to know this as well

With those caveats in mind, the columns names will probably not change now that it's actually starting to get some use, and we will make our best effort to not change the order of the columns either.

[barbeau]

@drewda Any thoughts on maintaining a global GBFS directory? Would the systems.csv file contained in this GBFS repo work for your needs as an API?

[rolinger]

I am planning on accessing this programmatically. However, I prefer to use SoBi's API for their network of bike shares. Thus in order to do that this service.csv file needs to have a column parameter that defines who the bikeshare network is - otherwise I have to statically map Sobi networks shares and parse that against this GBFS csv file to weed out the duplicates leaving only a list of none Sobi networks. But...since a network ID does not exist...i must manually and statically compare the Sobi list against the none SoBi GBFS locations.

[jcn]

@rolinger It's a bummer that SoBi doesn't include the GBFS system name, or link directly to the GBFS auto-discovery url in their /networks.json endpoint, which would solve at least your particular problem. Maybe @cubbi could chime in on this particular case.

[cubbi]

@jcn - I am not sure I follow. What is "/networks.json" file? We have unique system_id in every system_information.json, and every SoBi landingpage have proper header in the section.

[jcn]

@cubbi Oh, my bad. I was reading your API and thought that /networks.json was a global network endpoint, not just for the authenticated user.

Either way though, maybe @rolinger can explain the particular use case where a mapping between a SoBi network and the list of GBFS networks would be useful.

[rolinger]

Let me explain, the GBFS specifically does not list "Social Bicycles" or "Sobi" anywhere. "Bcycle" includes their name in their URL but not in the systems.csv or in the top level json per bikeshare (ie: system_information.json. Half of the GBFS listed sites are Sobi sites, a handful are Bcycle sites - but there is virtually no way of telling one from the other. In some Sobi json files you will see "socialbicycles" in the domain/url - but in other cases like "Coast Bikes" there is no mention of Sobi/socialbicycles" anywhere thus there is no way for me to decipher that they are in fact a part of the SoBi network.

Now, if I go access the specific SoBi API...then yes, I can extract that info. But not everyone has their own API and only rely on the GBFS. And for those that do, there is no way for me to tell if they are part of a larger network or if they are independent. Being able to group by Network provides useful for a variety of coding AND informational uses for users.

[jcn]

I prefer to use SoBi's API for their network of bike shares. Thus in order to do that this service.csv file needs to have a column parameter that defines who the bikeshare network is - otherwise I have to statically map Sobi networks shares and parse that against this GBFS csv file to weed out the duplicates

@rolinger Are you specifically saying that you're intending to use the systems.csv file as your canonical source for bike share systems, but when you encounter a SoBi system then you're planning on just jumping over and using the SoBi API directly and are planning on ignoring their GBFS feeds?

If that's the case, that's probably not the best use of the systems.csv file, which is specifically intended as a convenience for linking to the GBFS feeds of different systems, not as a de facto listing of all bikeshare systems around. In that case, you're probably better off maintaining your own list, enhanced with the additional app-specific data that you're looking for.

[rolinger]

@jcn - I will use systems.csv as one part of trying to find/define all, or as many, bikeshares as possible. Sobi, GBFS, api/citybik.es - a composite list from multiple sources so I can represent as many as possible in my app. But with GBFS no defining the shared network (ie: socialbicycles, or bcycles) it become programmatically impossible to group the various bikeshares by network and/or decipher overlaps (or independents) from other sources.

Its like just naming all cars by Ford...but never actually defining the manufacturer name "Ford" anywhere. Some just so happen to mention Ford, while others do not. Without specifically naming the network service ("Ford"), then its impossible to determine who some of the city bikesahres (the "cars") belong to.

[jcn]

As a standard, I don't see GBFS ever dictating a naming scheme for the vendors / operators.

That said, there are some proposals in #34 that might work (specifically including vendor in the system information file). But I'd imagine that would always be optional.

[rolinger]

Any place is better than no place. and I agree, having it in the standard top level json makes more sense.

[aospan]

Found this thread while learning how to aggregate all available GBFS data.
And found this R package using systems.csv:
https://github.com/ds-civic-data/gbfs/blob/master/R/get_gbfs.R

just FYI.

[heidiguenin]

We're considering the benefits of moving systems.csv to its own repo - to be able to have a different set of maintainers, to make the change logs easier to parse, etc.

Before making that decision, it would be helpful to know how that would affect consumers of the file and get your feedback on the best path forward for communicating that change.

[heidiguenin]

Still looking for feedback on my last comment. As of today 83 of the 127 PRs (open and closed) related to systems.csv rather than to the GBFS itself.

[gcamp]

No consequences on this change for us, we'd be in favour.

[hunterowens]

Pro this change, this is also a huge headache for us in maintain the similar listing inside mobility-data-specification, can see the needs to have it be governed / controlled / monitored differently.

…

On Mon, Dec 30, 2019 at 7:51 AM Guillaume Campagna ***@***.***> wrote: No consequences on this change for us, we'd be in favour. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#28>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANHXYW3GQQR2YLPWPNPZI3Q3IKILANCNFSM4CDSFEYQ> .

[mplsmitch]

Closing this issue - this topic has been taken up in #249