GBFS: Performance/Memory Improvements + Vehicle Types #670
Conversation
# Conflicts: # .pkg # .pkg.lock # ui/src/lib/getModeName.ts
include/motis/gbfs/lru_cache.h
Outdated
| auto write_lock = std::unique_lock{mutex_}; | ||
| if (auto it = cache_map_.find(key); it != cache_map_.end()) { | ||
| if (auto const lru_it = | ||
| std::find(lru_order_.begin(), lru_order_.end(), key); |
There was a problem hiding this comment.
| std::find(lru_order_.begin(), lru_order_.end(), key); | |
| std::find(lru_order_.begin(), lru_order_.end(), key); |
utl::find
But in general looks expensive in case the cache becomes large?
There was a problem hiding this comment.
lru_order_ only stores numbers and I expected the cache size to be <= 1000. Finding a number and moving memory around (in move_to_front) in <= 8 KB should be fast enough I hope. We could probably limit gbfs_provider_idx_t (the entries in this vector) to uint16_t instead of size_t to make it more compact.
What alternative did you have in mind? For example, I don't think an additional map key -> lru index is very helpful because the indices of all the entries change every time move_to_front is called (unless only the first entry is accessed), so keeping that map updated would be expensive.
| namespace motis::gbfs { | ||
|
|
||
| template <typename T> | ||
| struct partition { |
There was a problem hiding this comment.
how is this related to boost::interval_set or boost::interval_map?
There was a problem hiding this comment.
Unless I'm missing something, I don't think they are related. The GBFS code doesn't use intervals. The vehicle types are just mapped to numbers 0..n (index into the list of vehicle types) for the partitioning.
partition.h implements Partition refinement, which I don't think Boost.Icl does (and I don't see how it would relate to intervals). The idea is that initially, all vehicle types are equal (= a single set containing all vehicle types), then refine is called for each combination of vehicle types that share some property but are different from all other vehicle types (this includes the form factor (bike, scooter etc.), vehicle types that can be returned at a station (vs. all the other ones that can't be returned there), and geofencing zone rules that are vehicle type specific). If not all vehicle types in a set share that property, the set is split into two sets.
# Conflicts: # .pkg.lock # src/endpoints/routing.cc # src/journey_to_response.cc # src/street_routing.cc
# Conflicts: # .pkg.lock
# Conflicts: # ui/src/lib/i18n/en.ts
# Conflicts: # .pkg.lock # include/motis/endpoints/routing.h # src/endpoints/routing.cc # src/journey_to_response.cc # test/routing_test.cc
# Conflicts: # .pkg.lock
RENTALmode with additional filters for form factors (bike, cargo bike, scooter etc.) and propulsion type (human, electric etc.).ttlfield is now processed to only download files when necessary.ttlis not expired or the contents are unchanged), the existing data is reused.gbfs.cache_size) are kept in memory. These bitfields are also compressed.Most GBFS providers don't actually use multiple vehicle form factors. Here is an example config for Switzerland, where some providers in Zurich provide bikes and scooters: