sort: add Find

[robpike]

I have never used sort.Search to build a sorted list (slice). It's too expensive and there are better ways. However, I use it often to do what its name suggests: search.

The API is designed for the growth case, which is fine and general, but the general (growth) case is rare, perhaps very rare, and handling the return value is clumsy†.

I suggest adding another function, say sort.Contains, that just says yes or no to the question of presence of the element. Here is sort.Search's use for the pure search case, right from the documentation:

i := sort.Search(len(data), func(i int) bool { return data[i] >= x })
        if i < len(data) && data[i] == x {
        	// x is present at data[i]
        } else {
        	// x is not present in data,
        	// but i is the index where it would be inserted.
        }

Besides the messy handling, that extra comparison has always bothered me.

Here is what that would look like with the proposed function:

 if sort.Contains(len(data), func(i int) bool { return data[i] >= x }) {
      	// x is present at data[i]
}

This seems so much more intuitive and semantically lightweight. If you needed the index - but why would you if you weren't going to insert there? - you could go back to the existing Search function.

It should be very simple to implement by having both Search and Contains call a helper that reports presence, and returning the appropriate result, or by just writing Contains itself - like Search, it's very short.

I already filled in the proposal questionnaire here: #45624

† And poorly documented, but that is fixable.

[robpike]

By the way, even in the event type parameters cause a rethink of this package, a lighter-weight Contains function would be nice to have.

[DeedleFake]

How would this work? I was under the impression that the reason for the secondary data[i] == x check is because the f function doesn't necessarily provide enough information to be sure if the value at i is actually the one being searched for or not. Since sort.Contains() doesn't actually return i, how would you be sure if the value was correct or not?

I'm also curious what the better way to build a sorted slice is that you mentioned. I have used sort.Search() to do so, many times. The only way I can think of that's even equivalent would be to build a heap and then pop the whole thing once you're done adding stuff to it, but just off the top of my head both seem to be O(n * log(n)) to me.

Edit: Actually, I think that the heap method would be better as it doesn't require as many copies, and potentially allocations, when adding elements thanks to not having to insert into the middle of a slice. It does use interfaces if using container/heap, though, which is a whole different type of overhead.

[robpike]

Whatever else it might do, it can do another test. It only has <= but two calls of that will tell it == (NaNs aside, but we're already on thin ice with NaNs. They're not even numbers.)

In case it's not clear from what I wrote above, that final test (or two) may be necessary, but I want the function to do it, not the caller.

[DeedleFake]

As proposed, there's no way for the function to determine actual equality. The function only takes the index to check against and, therefore, knows nothing about the value actually being checked. For example, given f is func(i int) { return data[i] >= 3 }, consider the the slices []int{1, 2, 3, 4} and []int{1, 2, 4, 5}. f(i) returns the exact same values for all indices in both cases, but the value is actually only present in one of the slices. Unless I'm somehow just completely missing something here, the API proposed does not seem capable of determining for sure whether or not the value being searched for is actually in the slice.

It only has <= but two calls of that will tell it ==

It doesn't have <=, though. It only has x <=. The x is built in to the function passed in. Without the ability change the order of the two things being compared, it can't be used to determine equality.

The only mention that the proposal seems to make for separately determining equality is

It should be very simple to implement by having both Search and Contains call a helper that reports presence, and returning the appropriate result, or by just writing Contains itself - like Search, it's very short.

but it's not clear what exactly that would look like. It sounds like you're saying that sort.Contains() needs a second function passed to it, maybe func(i int) bool, that it will call to determine the final equality. If so, that would work, but I don't see a whole lot of advantage over the existing sort.Search(), and having to pass two closures to sort.Contains() seems more cumbersome to me, not less.

This

present := sort.Contains(
  len(data),
  func(i int) bool { return data[i] >= x },
  func(i int) bool { return data[i] == x },
)
if present {
  // x is present in data
}

doesn't seem much better to me than this

i := sort.Search(len(data), func(i int) bool { return data[i] >= x })
if (i < len(data)) && (data[i] == x) {
  // x is present at data[i]
}

Edit: It occurs to me that this is fairly straightforwards with generics:

Playground Link

func Contains[T any, S ~[]T](s S, x T, less func(v1, v2 T) bool) bool {
	i := sort.Search(len(s), func(i int) bool { return less(x, s[i]) }) - 1
	return (i >= 0) && !less(s[i], x)
}

This is limited to slices, but that probably covers 99% of cases. A more complicated one that covers other cases is also possible, but probably not worth the trouble.

Edit 2: Fix an instance of return condition ? true : false. Sigh... That's what I get for writing code at 2:00 A.M.

[robpike]

I had forgotten the challenge of moving the equality check away from the caller. I hadn't thought it through - even though I remember talking to @griesemer about the challenge when he was designing it. Mea ignoramica.

But that affects only the mechanism, not my desire, which is to simplify the API. I like your solution, at least as a discussion point. I can't help feel there's a clean API trying to be found. C's bsearch isn't all that nice to use either, though.

[robpike]

There's even this nice (untested) possibility for the simplest and possibly commonest case, if we can choose a different name for it.

func Lookup[T constraints.Comparable, S ~[]T](s S, x T) bool {
	i := sort.Search(len(s), func(i int) bool { return x <= s[i] })
	return i < len(s) && x==s[i]
}

[mlevieux]

I'm probably about to say something completely wrong, but isn't the problem solved by asking for a function able to report for multiple states? Like here we're focusing on a closure like func (i int) bool, but we could totally go for func (i int) int that'd be a comparator function so -1 on <, 0 on == and 1 on >. So sort.Contains would take this function and return true if it could compare some value for equality, false otherwise. This makes writing the closure longer and probably a little-less readable. But it also has the advantage of letting the caller define the equality relations according the use case, which is not possible through the use of constraints.Comparable.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

It sounds like we agree that sort.Search is hard to use, but there's not an obvious replacement. In particular, anything that can report an exact position has to take a different function (a 3-way result like strings.Compare, or multiple functions).

For sake of argument, we could do:

func Find(n int, cmp func (i int) int) int

where the cmp function returns <0, 0, or >0 to say whether the item being sought is less than, equal to, or greater than element at position i.

(In comparison, sort.Search's f reports whether the item being sought is less than the element at position i; that is, it's Less(item, elem[i]), whereas Find takes Cmp(item, elem[i]).)

Is that something we should consider adding?

[rsc]

When we do resolve this discussion, perhaps we should revisit #47619 (comment).

[robpike]

@rsc I am in favor of adding a function with the signature you suggest, and happy with the name.

[rsc]

Sounds like people are generally in favor of adding sort.Find.
It looks like we'd need to add slices.BinaryFind and slices.BinaryFindFunc as well.
The difference between Search and Find in all cases would be that Find returns -1 when the element is not present,
whereas Search returns the index in the slice where it would be inserted.

Do I have that right? Does anyone object to adding Find (and slices.BinaryFind and slices.BinaryFindFunc)?