Faster alternative routes for longer routes #1524

baumboi · 2019-01-17T00:07:16Z

This pull request changes the algorithm for alternative routes. It now supports CH and has the possibility to be precomputed. Since precomputing needs the graph to be split into different parts, I added a GraphPartition class which may also be useful for other purposes as well.

The whole algo is based on this paper:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.665.4573&rep=rep1&type=pdf
This poster summarizes everything quite well and has some nice visualisations
https://algo2.iti.kit.edu/download/ls-csarr-13-poster.pdf

I did a lot of testing on this algorithm and found out, that the new algo with CH, but without preparation should take about 10x the time the classic CH needs to find a route (~15 ms vs ~1.5 ms). In 40% to 50% of all cases at least one alternative is found in Saarland (about 100K nodes, the biggest graph my computer can do), but this number should increase with the size of the graph.
With precomputing, the success rate is slightly higher and the algorithm is 2x to 3x faster than without (~6 ms). For bigger graphs the prepared algo should get even faster, relatively to the unprepared one, as the number of precomputed viaNodes should stay the same, while the number of during the request computed contact nodes increases with the size of the graph.
All known factors of the old algorithm will stay the same, except for minPlateauFactor which isn't needed anymore. I lowered maxWeightFactor from 1.4 to 1.25 and increased maxShareFactor from 0.6 to 0.75 in order to get better results. With the old values some alternatives did not seem very reasonable. Using theese new values, bad alternatives became very rare (maybe about 1 out of 25). I also increased maxPaths from 2 to 3. If you want better query times you can lower it again together with additionalPaths.

Precomputation itself takes about 5x to 10x the time preparing CH needs (~25 s vs ~180 s), but it heavily depends on the precision and the amount of areas the graph is split into. Precision should be fine at a value of 5 (default) or 4, meaning 5 (or 4) nodes are chosen on average per area to run alternative route requests from and to during precomputation. Since this has to be done for nearly each pair of areas, about
(precision ^ 2) * (areas ^ 2)
requests need to be done to prepare alternative routes. I can't really tell how precomputing times will scale with the size of the graph. Lowering the amount of areas will drop preparing times a lot, but will also result in bigger area sizes. And because the prepared algorithm can only be used for nodes in two different, not directly connected areas, this may result in higher query times for short routes. To get the best out of it, we should test up to which graph size the unprepared algorithm has good query times and match the amount of areas accordingly. (the unprepared algo takes 15 ms on average in Saarland, so good query times without preparing should be able to up to maybe 2 million nodes?)

I'm really excited to know how the algorithm performs on a much larger graph like Germany.

I wasn't quite sure on how to implement the alternative route preparation with CH, so I added the method createPrepareAlternativeRoute(Graph graph, AlgorithmOptions opts) to PrepareContractionHierachies. Is there a better way to do this?
Currently, to let my algorithms know if they run CH, I'm looking whether the Graph is a CHGraph and the Weighting is a PreparationWeighting like in line 64 in PrepareAlternativeRoute. Is there another way to do this?
Moreover, in AlternativeRouteAlgorithm in line 193 to 205, I don't know how to get a "normal" Weighting out of a PreparationWeighting. I'm sure there is a better solution than this^^

Alternative routes can now be computed faster and support CH

karussell · 2019-01-17T20:46:09Z

@baumboi really nice - thanks a lot!

I'm curious to try this out. Can you try to make it compile? See the error messages at the moment here. Highly likely you started your work from an older master version and this needs an update. The following should make it work:

update the master branch of your fork
go to the alternative branch: git checkout alternative
merge the changes of master into the alternative branch: git merge --no-ff master
now you need to resolve conflicts and run tests
commit and push

baumboi · 2019-01-18T14:44:58Z

Now it should work. It was indeed the old master version, but I also forgot to change two tests in GraphhopperIT.

…adding a new AlgoFactoryDecorator. On my laptop, the preparation takes about 3 minutes to compute for Germany per weighting and vehicle, compared to about 5 minutes for CH. As the prepared alternative route algorithm will only be used for long routes (maybe 200 km or longer, depending on amount of areas in the partition), preparation won't really be needed for FlagEncoders like "foot" or "bike". To save computation time we could even go as far as preparing alternative routes only for "car" and using the data for other related vehicles like "motorcycle". Although I didn't try this out, it should also give good results. I rewrote GraphPartition to improve its efficiency. It's now able to partition Germany in less than 15 seconds. Using a lower precision value, this time decreases to below 5 seconds. Furthermore, I improved the algorithm for CH and added alternative routes for the Landmark-algorithm. CH gives very good results - with and without preparation. LM on the other hand has some problems with finding alternative routes. It's search space mostly expands into one direction compared to CH which expands evenly into every direction, making it much easier to find contact points between search spaces. Moreover the normal LM / AStar algorithm won't have to expand further after finding the best path, which means we have to continue searching to find alternatives, increasing queue times by a lot. Finally, CH will only find a few, good contact points, mostly located on highways, compared to LM / AStar which will sometimes find more than 100 points with most of them being located on small provincial roads. This means we have to check way more alternatives when using LM, also increasing queue times. To prevent queue times over 1 second, I added two barriers to the LM algorithm, stopping the search spaces to grow too large and finding too many contact points. This results in very low success rate for the unprepared LM algorithm of below 10%, meaning less than 1 route out of 10 has at least one alternative. With the current parameters at least one alternative can be found for about 2 out of 3 routes. After running a few benchmarks the average queue times and success rates with this version for Germany on my laptop are: - default CH: ~3 ms - CH with alternatives: ~30 ms ~65 % - CH with prepared alternatives: ~20 ms ~60 % - default LM: ~80 ms - LM with alternatives: ~300 ms ~8 % - LM with prepared alternatives: ~400 ms ~35 %

karussell

Thanks a lot for this work!

This is really nice and worked out of the box! Especially that the preparation is now that fast. The same for the graph partitioning.

It is really nice that we even get more than 2 alternatives - is this even part of the paper :) ?

There are a couple of changes that I commented and also:

alternative routes should not be returned by default for backward compatibility
alternative routes preparation should not be enabled by default (this should also reduce the changes in the tests)
disabling AR does not work -> no is converted to false -> this is also a bug for LM -> explicit lm weightings disabling not working as intended #1076
alternative routes should be sorted by the weight (?) http://localhost:8989/maps/?point=51.305721%2C14.558258&point=52.190772%2C12.810059 -> this is probably outside the scope of this PR

The most important thing that needs attention is that there is something wrong for non-CH AR (maybe all the rest can be fixed even after merge):

there is a bug for LM & AR (probably at meeting point, see image below): http://localhost:8989/maps/?point=52.556316%2C13.393021&point=52.504311%2C13.342552&locale=en&vehicle=car&weighting=fastest&ch.disable=true
this also happens for other routes for just Dijkstra & AR: http://localhost:8989/maps/?point=52.556316%2C13.393021&point=52.504311%2C13.342552&locale=en&vehicle=car&weighting=fastest&lm.disable=true&ch.disable=true

It would be also nice if you can "somehow" increase the alternative route success for "LM with alternatives, not prepared". Even if the average query time increases to 1.5sec. Because current users of alternative routes would otherwise see this a regression :)

karussell · 2019-07-08T17:59:29Z

core/src/main/java/com/graphhopper/GraphHopper.java

@@ -121,10 +125,13 @@
    public GraphHopper() {
        chFactoryDecorator.setEnabled(true);
        lmFactoryDecorator.setEnabled(false);
+        arFactoryDecorator.setEnabled(true);


I would not enable this by default. Then there is also no need to change the many tests.

karussell · 2019-07-08T18:00:25Z

api/src/main/java/com/graphhopper/util/Parameters.java

+        public static final String ADDITIONAL_PATHS = PREPARE + "additional_paths";
+
+        public static final String AREAS = PREPARE + "areas";
+    }


I would move the AltRoute section into this too

karussell · 2019-07-08T18:01:41Z

core/src/main/java/com/graphhopper/routing/AStarBidirection.java

@@ -57,7 +57,7 @@
 * @author jansoe
 */
 public class AStarBidirection extends AbstractBidirAlgo implements RecalculationHook {
-    private ConsistentWeightApproximator weightApprox;
+    protected ConsistentWeightApproximator weightApprox;


Is this required only for toString?

karussell · 2019-07-08T18:06:50Z

core/src/main/java/com/graphhopper/routing/util/AbstractAlgoPreparation.java

+
+    protected void setPrepared() {
+        prepared = true;
+    }


This should not be necessary if doWork is called.

karussell · 2019-07-08T18:09:08Z

tools/src/main/java/com/graphhopper/tools/Measurement.java

+                    if (ch)
+                        req.setAlgorithm(DIJKSTRA_BI);
+                    else
+                        req.setAlgorithm(ASTAR_BI);


This shouldn't be necessary. Everything that is not related to alternative routes should stay like before.

karussell · 2019-07-08T18:14:22Z

core/src/main/java/com/graphhopper/routing/AlternativeRoute.java

-    private ViaNodeSet viaNodes;
-    private ArrayList<ViaPoint> viaPoints;
-    private boolean longAlgo;
+public class AlternativeRoute extends AStarBidirection {


This looks very similar to AlternativeRouteCH, can we reuse code here or why did you choose to copy it?

…isabling AR preparation by default. I also fixed the bug and changed some parameters to improve the success rates for the unprepared LM algorithm. The new average queue times and success rates for Germany are: - LM with alternatives: ~2500 ms ~25 % - LM with prepared alternatives: ~700 ms ~40 % Since the prepared algorithm needs the unprepared one for short routes within neighboring partitions its average queue time also get a bit worse to about 700 ms in my tests. However, its performance for longer routes stays the same. The quite low success rate of 25 % for the unprepared algorithm is due to its bad performance for long routes. When testing it in smaller graphs like Berlin or Baden-Württemberg, it shows much better results: - unprepared AR-LM in Berlin: ~150 ms ~90 % - unperpared AR-LM in Baden-Württemberg: ~1100 ms ~50 %

karussell · 2019-08-20T10:37:35Z

Thanks, this is yet another nice improvement. I was not able to find the bug. Additionally the non-CH alternative success rate was indeed improved :)

Still there seems to be "obviously suboptimal" alternatives. E.g. this route: http://localhost:8989/maps/?point=52.491874%2C13.36092&point=52.548697%2C13.367443&vehicle=car&algorithm=alternative_route has two strange alternatives:

The "left alternative" is really suboptimal as you can see here: http://localhost:8989/maps/?point=52.516012%2C13.347015&point=52.530948%2C13.347015

The "middle alternative" is also suboptimal: http://localhost:8989/maps/?point=52.519805%2C13.355802&point=52.521861%2C13.360662

Could it be that there is still something wrong with picking the candidate node set? E.g. in the "middle suboptimality" case I would have expected that a candidate on Alice-Berend-Str is always overruled by a candidate on the parallel Lüneburge Str.
Another problem can be if a contact node is directly chosen without looking at its weights. I think (not 100% sure) that the contact node can only be used if the sum of the from+to weights in the queue are bigger then the from+to weights of that contact node.

I'm using

  prepare.ch.weightings: fastest
  prepare.ch.edge_based: off
  prepare.lm.weightings: fastest
  prepare.ar.weightings: no

…algorithm, finding wrong or bad contact nodes. Now the algorithm finds more contact nodes and does this much more efficiently, which results in better success rates and better queue times. The new testing results for Germany are: - CH without alternatives: ~4 ms - CH with unprepared alternatives: ~35 ms ~80 % - CH with prepared alternatives: ~20 ms ~70 % - LM8 without alternatives: ~70 ms - LM8 with unprepared alternatives: ~650 ms ~30 % - LM8 with prepared alternatives: ~450 ms ~50 %

karussell · 2020-01-12T21:37:39Z

Although this PR was faster it was replaced by #1722. We'll revisit this PR to e.g. improve perf and for fixing the "motorway loop"-bug.

baumboi added 13 commits January 16, 2019 23:06

Add AlternativeInfo.java

4476a7c

Add AlternativeRouteAlgorithm.java

da864d0

Add ContactPoint.java

f61819c

Add GraphPartition.java

0d47b2b

Add PrepareAlternativeRoute.java

797f5f6

Add ViaNodeSet.java

f3447bb

Add ViaPoint.java

07fc511

Change the alternative route algorithm

094e1a5

Alternative routes can now be computed faster and support CH

Remove the plateau factor

5268d2b

Add the possibility to create alternative routes

ca77661

Add new import statement

6278c55

Remove the approximation for alternative routes

917f267

change and add some tests to match new algorithm

2199b12

TimMcCauley mentioned this pull request Jan 17, 2019

Enable GraphHopper's alternative wayfinding methods GIScience/openrouteservice#377

Closed

baumboi added 5 commits January 18, 2019 13:29

Update PrepareContractionHierarchies.java

c983580

Update PrepareAlternativeRoute.java

c92b980

Update GraphPartition.java

f0dbb5a

Update AlternativeRoute.java

d788a5c

Update GraphHopperIT.java

223e346

baumboi added 2 commits January 20, 2019 21:01

Change determination if algo uses CH

5e4511f

Change determination if algo uses CH

6d393b8

karussell changed the title ~~Better alternative routes~~ Faster alternative routes for longer routes Jan 21, 2019

baumboi added 6 commits January 23, 2019 17:11

PrepareAlternativeRoute is no longer a RoutingAlgorithmFactory

419b6e1

Add option to prepare alternative routes

490a67f

Add PREPARE_ALTERNATIVE_ROUTE parameter

2daf8de

Update some tests to match changes

279e063

Update Params

2b591ee

Update PrepareContractionHierarchies.java

59113db

Change default to false to prevent tests from crashing

338c8f5

takb mentioned this pull request Jan 28, 2019

Feature #377 alternative routes GIScience/openrouteservice#403

Closed

12 tasks

karussell requested changes Jul 9, 2019

View reviewed changes

easbar mentioned this pull request Aug 29, 2019

Do not consider turn cost support of other encoders when selecting traversal mode #1705

Merged

easbar mentioned this pull request Sep 11, 2019

Alternative routes with node-based CH #1722

Merged

karussell closed this Jan 12, 2020

karussell mentioned this pull request Jan 27, 2020

Alternative route improvements (for CH) #1872

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Faster alternative routes for longer routes #1524

Faster alternative routes for longer routes #1524

baumboi commented Jan 17, 2019

karussell commented Jan 17, 2019

baumboi commented Jan 18, 2019

karussell left a comment •

edited

Loading

karussell Jul 8, 2019

karussell Jul 8, 2019

karussell Jul 8, 2019

karussell Jul 8, 2019

karussell Jul 8, 2019

karussell Jul 8, 2019

karussell commented Aug 20, 2019 •

edited

Loading

karussell commented Jan 12, 2020

Faster alternative routes for longer routes #1524

Faster alternative routes for longer routes #1524

Conversation

baumboi commented Jan 17, 2019

karussell commented Jan 17, 2019

baumboi commented Jan 18, 2019

karussell left a comment • edited Loading

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell Jul 8, 2019

Choose a reason for hiding this comment

karussell commented Aug 20, 2019 • edited Loading

karussell commented Jan 12, 2020

karussell left a comment •

edited

Loading

karussell commented Aug 20, 2019 •

edited

Loading