Separated the logic of branch and bound search #97

Kolaru · 2018-09-29T18:54:14Z

I finally found the courage to update to 0.7/1.0 and to implement the idea proposed by @dpsanders in #72 to separate the logic of the search from the rest of the package. Therefore this PR superseed #72 (I do not update it though, as the scope is quite different).

Changes in this PR:

Introduction of a generic customizable branch and bound search interface.
Introduction of a tree type as the fundamental data structure for the search. Some advantage come from that as for example pretty printing of the tree. The roots function still return a simple list of intervals.
Separation of the branch and bound logic in the rootsearch_iterator.jl file, so the transition to the code in another package can be done easily.
Rework of the interface to avoid storing functions in an object (and the kind of akwardly high number of parameters that resulted).
Use of Root object during the search rather than interval, which is forced by the genericity of the branch and bound algorithm (status of roots does not exactly match with the status of the elements in a search).
Fix of Repeating Newton gives :unknown #29 along the way.

codecov-io · 2018-09-29T19:38:23Z

Codecov Report

Merging #97 into master will decrease coverage by 3.4%.
The diff coverage is 58.74%.

@@            Coverage Diff             @@
##           master      #97      +/-   ##
==========================================
- Coverage   59.58%   56.17%   -3.41%     
==========================================
  Files          11       12       +1     
  Lines         532      623      +91     
==========================================
+ Hits          317      350      +33     
- Misses        215      273      +58

Impacted Files	Coverage Δ
src/IntervalRootFinding.jl	`5.55% <ø> (ø)`	⬆️
src/branch_and_bound.jl	`38.82% <38.82%> (ø)`
src/roots.jl	`72.88% <85%> (-8.16%)`	⬇️
src/contractors.jl	`95.55% <94.44%> (-1.95%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update becade4...864a337. Read the comment docs.

dpsanders · 2018-09-29T21:08:14Z

This looks great, thanks a lot!

I am a bit worried about the performance implications of using a tree like that. Have you done any benchmarks?

In general, I think we should try to leave as general as possible the data structures used, so you could specify that you wanted to store the nodes in a tree, or a vector as before, etc. How possible would that be?

Kolaru · 2018-09-29T22:52:41Z

So I quickly added some benchmark using the Smiley and Chun examples introduced by @gwater and the PkgBenchmark.jl package (this PR is necessary to run its basics tasks though). Here are the results:

Master:

ID	time	GC time	memory	allocations
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 2.2"]`	12.457 ms (5%)		8.41 MiB (1%)	250556
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 5.2"]`	215.253 ms (5%)	24.078 ms	115.33 MiB (1%)	3433780
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 5.4"]`	9.902 ms (5%)		7.51 MiB (1%)	219406
`["Smiley", "Newton", "Smiley and Chun (2001), Example 2.2"]`	12.983 ms (5%)		8.76 MiB (1%)	259155
`["Smiley", "Newton", "Smiley and Chun (2001), Example 5.2"]`	255.685 ms (5%)	28.522 ms	136.83 MiB (1%)	4044788
`["Smiley", "Newton", "Smiley and Chun (2001), Example 5.4"]`	12.438 ms (5%)		9.32 MiB (1%)	271491

This PR:

ID	time	GC time	memory	allocations
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 2.2"]`	12.540 ms (5%)		8.46 MiB (1%)	252695
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 5.2"]`	226.340 ms (5%)	27.452 ms	116.05 MiB (1%)	3450366
`["Smiley", "Krawczyk", "Smiley and Chun (2001), Example 5.4"]`	9.968 ms (5%)		7.52 MiB (1%)	220525
`["Smiley", "Newton", "Smiley and Chun (2001), Example 2.2"]`	13.686 ms (5%)		8.81 MiB (1%)	261324
`["Smiley", "Newton", "Smiley and Chun (2001), Example 5.2"]`	285.322 ms (5%)	37.126 ms	137.49 MiB (1%)	4064908
`["Smiley", "Newton", "Smiley and Chun (2001), Example 5.4"]`	12.503 ms (5%)		9.35 MiB (1%)	272998

It looks like this PR implies about 5% to 10% penalty (PkgBenchmark comparison is not yet functional for Julia 0.7/1.0, so it's the best estimate I can easily do for now). It may be possible to remove this penalty completely, since the tree structure closely correspond to a branch and bound algorithm.

On the other hand I was expecting way more allocations with this PR. Good surprise I guess.

As for allowing to store the data in arbitrary structure, while designin this PR I considerd that the ability to easily convert the result toward any data structure would be sufficient. It is also the simplest solution from the developper side.

However if the current penalty we pay to keep maximal information about the run of the algorithm is too high (seems reasonnable to me, but I essentially use the package to solve rather simple equations so I may not be representative), it is possible to make it more generic (may become tricky thought).

dpsanders · 2018-10-01T00:17:42Z

Wow, that's great! Let me look over this in detail then.

dpsanders · 2018-10-02T02:21:32Z

Do you have a simple example to show how to use the new functionality / see the tree etc?

I have been thinking that roots should create a RootsProblem (or whatever) object that stores all the parameters used and all the intermediate results etc.

Kolaru · 2018-10-02T18:32:19Z

I updated the example in the examples dir and put it in a new file (root_search_iterator.jl). It is abundantly commented, so I hope everything is clear.

By default, the intermediate results are not stored in the tree to spare memory. However, one of the example show how to do this by changing the process function.

Also the search object already store all final parameters (final in the sense that the contractor is stored, but not the input parameter such as the original function or if it uses automatic differentiation for example).

dpsanders · 2019-02-02T18:37:58Z

Sorry, this now has conflicts :/

dpsanders · 2019-02-06T02:43:46Z

src/branch_and_bound.jl

+tuples `(node_id, lvl)` where `lvl` is the depth of the node in the tree.
+"""
+struct BBTree{DATA}
+ nodes::Dict{Int, Union{BBNode, BBLeaf{DATA}}}


Is there a way of avoiding this Union?

It is possible to have two dict: one for the branchings (BBNode) and one for the leaves (BBLeaf{DATA}).

dpsanders · 2019-02-07T01:52:48Z

I can attempt to rebase this if you would like.

Kolaru · 2019-02-08T02:13:06Z

@dpsanders No need to, thanks. I'll do it soon.

Allow to have type predictability for the full iteration cycle.

- Introduce default strategies - Rework interface to allow using them - Changes break tests for complex Krawczyk method which expectation has been lowered

Kolaru · 2019-02-24T01:48:04Z

Below is the benchmark of the current state of this PR against master. For now we see a slowdowns ranging from 5% to 20% in worst cases. I'll try to identify if there is a bottleneck and if I can do something to speed things up.

The decrease in memory usage is a welcomed suprise.

Benchmark Report for IntervalRootFinding

Job Properties

Time of benchmarks:
- Target: 24 Feb 2019 - 01:40
- Baseline: 24 Feb 2019 - 01:42
Package commits:
- Target: 46dea2
- Baseline: becade
Julia commits:
- Target: 80516c
- Baseline: 80516c
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["Dietmar-Ratz", "Dietmar-Ratz 1", "Krawczyk"]`	1.13 (5%) ❌	0.21 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 1", "Newton"]`	1.11 (5%) ❌	0.21 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 2", "Newton"]`	1.05 (5%) ❌	1.01 (1%)
`["Dietmar-Ratz", "Dietmar-Ratz 3", "Krawczyk"]`	1.05 (5%) ❌	1.03 (1%) ❌
`["Dietmar-Ratz", "Dietmar-Ratz 3", "Newton"]`	1.08 (5%) ❌	1.04 (1%) ❌
`["Dietmar-Ratz", "Dietmar-Ratz 3", "Slope expansion"]`	1.06 (5%) ❌	1.00 (1%)
`["Dietmar-Ratz", "Dietmar-Ratz 4", "Automatic differentiation"]`	1.05 (5%) ❌	1.00 (1%)
`["Dietmar-Ratz", "Dietmar-Ratz 4", "Krawczyk"]`	1.05 (5%) ❌	0.98 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 4", "Newton"]`	1.04 (5%)	0.96 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 5", "Krawczyk"]`	1.18 (5%) ❌	0.19 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 5", "Newton"]`	1.14 (5%) ❌	0.19 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 6", "Krawczyk"]`	1.04 (5%)	0.90 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 6", "Newton"]`	1.04 (5%)	0.94 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 7", "Automatic differentiation"]`	1.06 (5%) ❌	1.00 (1%)
`["Dietmar-Ratz", "Dietmar-Ratz 7", "Krawczyk"]`	1.12 (5%) ❌	0.33 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 7", "Newton"]`	1.09 (5%) ❌	0.33 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 9", "Krawczyk"]`	1.09 (5%) ❌	0.18 (1%) ✅
`["Dietmar-Ratz", "Dietmar-Ratz 9", "Newton"]`	1.10 (5%) ❌	0.18 (1%) ✅
`["Rastigrin stationary points", "Krawczyk"]`	1.06 (5%) ❌	1.00 (1%)
`["Rastigrin stationary points", "Newton"]`	1.09 (5%) ❌	1.01 (1%)
`["Smiley", "Smiley and Chun (2001), Example 5.2", "Krawczyk"]`	1.07 (5%) ❌	1.01 (1%)
`["Smiley", "Smiley and Chun (2001), Example 5.2", "Newton"]`	1.09 (5%) ❌	1.01 (1%)
`["Smiley", "Smiley and Chun (2001), Example 5.4", "Newton"]`	1.06 (5%) ❌	1.01 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["Dietmar-Ratz", "Dietmar-Ratz 1"]
["Dietmar-Ratz", "Dietmar-Ratz 2"]
["Dietmar-Ratz", "Dietmar-Ratz 3"]
["Dietmar-Ratz", "Dietmar-Ratz 4"]
["Dietmar-Ratz", "Dietmar-Ratz 5"]
["Dietmar-Ratz", "Dietmar-Ratz 6"]
["Dietmar-Ratz", "Dietmar-Ratz 7"]
["Dietmar-Ratz", "Dietmar-Ratz 8"]
["Dietmar-Ratz", "Dietmar-Ratz 9"]
["Linear equations", "n = 10"]
["Linear equations", "n = 2"]
["Linear equations", "n = 5"]
["Rastigrin stationary points"]
["Smiley", "Smiley and Chun (2001), Example 2.2"]
["Smiley", "Smiley and Chun (2001), Example 5.2"]
["Smiley", "Smiley and Chun (2001), Example 5.4"]

Julia versioninfo

Target

Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
      Microsoft Windows [version 10.0.17763.316]
  CPU: Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3504 MHz   41728312            0     24420343    250501062      4959093  ticks
       #2  3504 MHz   42282593            0     17158515    257208390       409375  ticks
       #3  3504 MHz   61348546            0     15767125    239533843       290093  ticks
       #4  3504 MHz   47695703            0     16460234    252493578       311796  ticks
       
  Memory: 15.95343017578125 GB (8507.94140625 MB free)
  Uptime: 622735.0 sec
  Load Avg:  0.0  0.0  0.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Baseline

Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
      Microsoft Windows [version 10.0.17763.316]
  CPU: Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz: 
              speed         user         nice          sys         idle          irq
       #1  3504 MHz   41786609            0     24435312    250581875      4961656  ticks
       #2  3504 MHz   42332593            0     17169453    257301531       409421  ticks
       #3  3504 MHz   61406390            0     15776750    239620453       290281  ticks
       #4  3504 MHz   47762937            0     16471140    252569515       311968  ticks
       
  Memory: 15.95343017578125 GB (8257.20703125 MB free)
  Uptime: 622889.0 sec
  Load Avg:  0.0  0.0  0.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Kolaru · 2019-03-03T23:58:43Z

It seems like most of the overhead comes from the creation of the tree and the associated dictionnaries. However due to the profiler sometime crashing on Windows (JuliaLang/julia#30902) I have not yet been able to do extensive profiling.

Also note that the build failures only concern Julia 0.7.

dpsanders · 2019-03-04T00:33:15Z

I would say that we should just remove support for Julia 0.7 and merge this.

Kolaru · 2019-03-04T00:44:27Z

Since the implementation is accepted, I can now do the tedious but necessary secondary work (add tests, check the examples are still valid and write some beautiful documentation). Hopefully I will be able to do it soon and then it will be ready for merging.

Kolaru · 2019-03-06T00:22:34Z

@dpsanders This is now ready.

dpsanders · 2019-03-07T18:46:42Z

Could you please update REQUIRE to put Julia 1.0 as the lower bound.
Or change the settings on your fork to allow me to modify your PR.

Kolaru · 2019-03-07T22:43:10Z

@dpsanders Done. I've also remove 0.7 build from travis and appveyor.

dpsanders · 2019-03-07T22:56:36Z

This looks really great, thanks!

I think the best thing to do is merge once green and then iterate later if necessary.

dpsanders · 2019-03-07T23:10:50Z

I think some of the printing etc. could probably be simplified using https://github.com/Keno/AbstractTrees.jl

I'm not sure if there are any ready-made packages for trees that could simplify some of the tree logic itself.

dpsanders · 2019-03-07T23:40:48Z

OK I'm going to merge for now.

Thanks a lot!

Kolaru · 2019-03-07T23:57:19Z

@dpsanders When I started working on this PR I checked for such packages. The closer which seems available are networks from packages such as LightGraphs.jl. However I preferred to implement my own tree type, as networks are more general and I was a bit worried about performance because of that.

I'll look into simplifying some part of this with AbstractTree.jl once I've dealt with my other pending PR.

dpsanders · 2019-03-08T00:02:21Z

Yes, true, LightGraphs.jl has a lot of relevant functionality. There's no rush in any case!

Kolaru mentioned this pull request Dec 12, 2018

Add benchmarks #105

Merged

dpsanders reviewed Feb 6, 2019

View reviewed changes

Kolaru added 19 commits February 23, 2019 17:51

Implement SearchStrategy for root finding

25d8c42

Fix bug for structures without sizehint!

1971585

Decouple the iterator and its state

ff6c672

Parametrize RootSearchState on container type

cc37cf2

Allow to have type predictability for the full iteration cycle.

Redefine RootSearch with parametric SearchStrategy

9837a6e

Create separate file for the RootSearch iterator

6750f2c

Rewrite documentation for RootSearch related objects

8e32f1a

Rework roots interface to allow choosing a strategy

828aefd

- Introduce default strategies - Rework interface to allow using them - Changes break tests for complex Krawczyk method which expectation has been lowered

Add simple test for SearchStrategy

f9fa542

Fix type instability and some other minor issues

b710b53

Introduce BBSearch iterator

61f249a

Adapt remaining of IntervalRootFinding to BBSearch

1c160c9

Solve bugs for 1D roots

d06d711

Update to 0.7

ab14c2a

Switch to interface based BB search

8cbb7f7

Add pretty printing of WorkingTree

c1dc40e

Fix Krawczyk 2D bug

5f5fcf5

Fix RootSearch tests

65b5754

Add doc and use better name

5f40f0d

Kolaru added 5 commits February 23, 2019 17:54

Update example and fix related bugs

0f71673

Fix NaN interval integration with tolerance

5790f7c

Fix bug when the starting interval can be excluded at first iteration

429cedb

Solve bug in pretty printing of tree

16afcea

Fix rebase issues

44f8a0c

Kolaru force-pushed the separated_branch_and_bound branch from ec42814 to 44f8a0c Compare February 23, 2019 17:57

Split leaves and nodes

46dea20

Kolaru added 4 commits March 5, 2019 02:50

Complete BB docstrings

102aa0b

Finalize simple example

e45b06b

Add tests

cc2f006

Correct struct def in tests

063be9e

Kolaru mentioned this pull request Mar 6, 2019

Allow keywords arguments for roots #116

Open

Drop 0.7 support.

864a337

dpsanders merged commit a58edad into JuliaIntervals:master Mar 7, 2019

dpsanders mentioned this pull request Mar 7, 2019

Repeating Newton gives :unknown #29

Closed

Kolaru deleted the separated_branch_and_bound branch March 7, 2019 23:57

Kolaru mentioned this pull request Mar 8, 2019

Allow control over the way the intervals are selected to be processed #72

Closed

Kolaru mentioned this pull request Apr 15, 2019

Correct multidim Krawczyk #114

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Separated the logic of branch and bound search #97

Separated the logic of branch and bound search #97

Kolaru commented Sep 29, 2018

codecov-io commented Sep 29, 2018 •

edited

Loading

dpsanders commented Sep 29, 2018

Kolaru commented Sep 29, 2018

dpsanders commented Oct 1, 2018

dpsanders commented Oct 2, 2018

Kolaru commented Oct 2, 2018

dpsanders commented Feb 2, 2019

dpsanders Feb 6, 2019

Kolaru Feb 6, 2019

dpsanders commented Feb 7, 2019

Kolaru commented Feb 8, 2019

Kolaru commented Feb 24, 2019

Kolaru commented Mar 3, 2019 •

edited

Loading

dpsanders commented Mar 4, 2019

Kolaru commented Mar 4, 2019 •

edited

Loading

Kolaru commented Mar 6, 2019

dpsanders commented Mar 7, 2019

Kolaru commented Mar 7, 2019

dpsanders commented Mar 7, 2019

dpsanders commented Mar 7, 2019

dpsanders commented Mar 7, 2019

Kolaru commented Mar 7, 2019

dpsanders commented Mar 8, 2019

Separated the logic of branch and bound search #97

Separated the logic of branch and bound search #97

Conversation

Kolaru commented Sep 29, 2018

codecov-io commented Sep 29, 2018 • edited Loading

Codecov Report

dpsanders commented Sep 29, 2018

Kolaru commented Sep 29, 2018

dpsanders commented Oct 1, 2018

dpsanders commented Oct 2, 2018

Kolaru commented Oct 2, 2018

dpsanders commented Feb 2, 2019

dpsanders Feb 6, 2019

Choose a reason for hiding this comment

Kolaru Feb 6, 2019

Choose a reason for hiding this comment

dpsanders commented Feb 7, 2019

Kolaru commented Feb 8, 2019

Kolaru commented Feb 24, 2019

Benchmark Report for IntervalRootFinding

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Kolaru commented Mar 3, 2019 • edited Loading

dpsanders commented Mar 4, 2019

Kolaru commented Mar 4, 2019 • edited Loading

Kolaru commented Mar 6, 2019

dpsanders commented Mar 7, 2019

Kolaru commented Mar 7, 2019

dpsanders commented Mar 7, 2019

dpsanders commented Mar 7, 2019

dpsanders commented Mar 7, 2019

Kolaru commented Mar 7, 2019

dpsanders commented Mar 8, 2019

codecov-io commented Sep 29, 2018 •

edited

Loading

Kolaru commented Mar 3, 2019 •

edited

Loading

Kolaru commented Mar 4, 2019 •

edited

Loading