Segfault on Nightly in BTreeSet and BTreeMap #33197
Comments
apasel422
added
A-collections
sfackler
added
I-nominated
T-libs-api
|
I was just looking at this briefly and I'm reminded the collection can not trust the user-supplied comparison functions (Ord) and thus the fix must be something more involved than comparing the input keys. |
|
So it seems we must first compare start to end (if start>end, stop). Then go forward but check for the end of the BTree. |
|
I see a few possible options here:
Ironically, the |
alexcrichton
added
P-medium
Note that safety should never depend on well-behavedness of |
|
As a variant of @gereeter's second bullet point, the structural check can be done as a post-processing step in which we walk up the tree from both handles, keeping track of the path we take as we go (i.e. the index of the edges). Once we finish ascending, we compare the paths lexicographically, and return an empty This has the benefit of avoiding additional complexity in the descent code and iteration code, and is amenable to removal via specialization if we ever get an |
|
Reproduced on recent Just wanted to know, is there any progress regarding the issue? Looks like it's quite annoying bug because of it's cryptic nature. I spent quite a time trying to find why the program exits silently even without a panic message. Only debugger helped me to find the actual reason. Recently there were some changes in the range API. Are they affecting the issue in some way? |
|
Could you share the Ord implementation testcase that caused this? Last time I checked the old testcases for this didn't work. |
|
Unfortunately I don't have minimal reproducing impl, but I'm working on it. Last observations were that if I replace in my code I have rather complex keys ( Currently I'm not sure, whether it's due to |
|
Indeed, in my case crash occurs when broken boundary logic yields the range where lower bound is actually greater that the upper one (in terms of Please note that it's highly dependant on the actual in-memory structrure of the tree. In my case for this to happen btree should be filled to a certain point. |
Dont segfault if btree range is not in order This is a first attempt to fix issue rust-lang#33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
Dont segfault if btree range is not in order This is a first attempt to fix issue rust-lang#33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
Dont segfault if btree range is not in order This is a first attempt to fix issue #33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
Dont segfault if btree range is not in order This is a first attempt to fix issue rust-lang#33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
Dont segfault if btree range is not in order This is a first attempt to fix issue #33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
Dont segfault if btree range is not in order This is a first attempt to fix issue #33197. The issue is that the BTree iterator uses next_unchecked for fast iteration, but it can be tricked into running off the end of the tree and segfaulting if range is called with a maximum that is less than the minimum. Since a user defined Ord should not determine the safety of BTreeMap, and we still want fast iteration, I've implemented the idea of @gereeter and walk the tree simultaneously searching for both keys to make sure that if our keys diverge, the min key is to the left of our max key. I currently panic if that is not the case. Open questions: 1. Do we want to panic in this error case or do we want to return an empty iterator? The drain API panics if the range is bad, but drain is given a range of index values, while this is a generic key type. Panicking is brittle and returning an empty iterator is probably the most flexible and matches what people would want it to do... but artificially returning a BTreeMap::Range with start==end seems like a pretty weird and unnatural thing to do, although it's doable since those fields are not accessible. The same question for other weird cases: 2. (Included(101), Excluded(100)) on a map that contains [1,2,3]. Both BTree edges end up on the same part of the map, but comparing the keys shows the range is backwards. 3. (Excluded(5), Excluded(5)). The keys are equal but BTree edges end up backwards if the map contains 5. 4. (Included(5), Excluded(5)). Should naturally produce an empty iterator, right?
|
This is fixed |
|
Thanks! |
An iterator created by
BTreeMap::range,BTreeMap::range_mutandBTreeSet::rangecan wander off of the underlying data and segfault. It is caused by supplying an upper bound that is lower than the lower bound and there is an element between them.Code to reproduce: http://is.gd/HoNWLH
tested also locally on
rustc 1.10.0-nightly (b5ba5923f 2016-04-21).The iterator ends when the iterated element equals to the upper bound: https://doc.rust-lang.org/src/collections/up/src/libcollections/btree/map.rs.html#898-908. If the upper bound is lower than the lower bound, this condition is never met. This is not checked for while creating the iterator in https://doc.rust-lang.org/src/collections/up/src/libcollections/btree/map.rs.html#464-588.
The text was updated successfully, but these errors were encountered: