Adding some utility methods to Maybe/Result

[ariebovenberg]

Have you considered adding some handy utility functions to Maybe and Result?
In using returns I'm missing many helpful methods you can find in Rust, for example.
Adding a few of these could increase the usability of the containers of returns.

Related: #1091

Here they are, in my subjective order of usefulness:

Setdefault

Set a value if one is not yet present.
The name setdefault would be consistent with the python dict method.
(Although it might be good to choose another name to make clear that
the method doesn't mutate.)

>>> Some(6).setdefault(7)
Some(6)
>>> Nothing.setdefault(3)
Some(3)

Zip

Combine the values from two containers into a tuple, if both are successful.

>>> Some(8).zip(Some(2))
Some((8, 2))
>>> Some(7).zip(Nothing)
Nothing
>>> Nothing.zip(Some(3))
Nothing
>>> Nothing.zip(Nothing)
Nothing

Filter

Keep a value based on a predicate

>>> Some("9").filter(str.isdigit)
Some("9")
>>> Some("foo3").filter(str.isdigit)
Nothing
>>> Nothing.filter(str.isdigit)
Nothing

And

AND logic operation

>>> Some(5).and_(Some(9))
Some(9)
>>> Some(9).and_(Nothing)
Nothing
>>> Nothing.and_(Some(8))
Nothing

Or

OR logic operation

>>> Some(5).or_(Some(9))
Some(5)
>>> Some(9).or_(Nothing)
Some(9)
>>> Nothing.or_(Some(8))
Some(8)
>>> Nothing.or_(Nothing)
Nothing

[CucumisSativus]

I think Setdefault is already in the library, in the form of value_or().
With zip (or map_n) and filter I really think those would be valuable additions to the library.
Could you show how would you use and and or, I think I am missing the idea how could those be used in the real source code

[ariebovenberg]

I'll illustrate with this example:

from typing import NamedTuple


class User(NamedTuple):
    id: int
    nickname: Maybe[str]
    first_name: Maybe[str]
    last_name: str
    joined: datetime
    job: Maybe[str]
    location: Maybe[int]

Setdefault

There difference with value_or() is that setdefault() doesn't unwrap the value. This can be useful when you want to keep the container. For example when updating a Maybe attribute.

user = user._replace(
    status=user.status.setdefault('Hello World')
)
# equivalent to
user = user._replace(
    status=user.status.success_type(user.status.value_or('Hello World'))
)

Or

Like Python's or, it is useful for fallback values:

# same behavior as python's `or` operator: returns the first success value, otherwise the second
salutation: Maybe[str] = user.nickname.or_(user.first_name)

(The difference with value_or() is that you combine two wrapped values)

And

I can't think of a super convincing case here, but it'd make sense to have it as a complement to or_(). I'm sure this basic logical operation would be useful to some people. Also, it's probably more useful in Result, where the failure type contains useful information.

# same behavior as python's `and` operator: returns the first failure value, otherwise the second
work_location: Maybe[str] = user.job.and_(user.location)

[CucumisSativus]

Thanks a lot for the clarifications, I should take a look at the issue slightly later in the morning :)

I am happy to implement zip and filter, for the other methods I'm not entirely convinced. But if @sobolevn says those are ok to be implemented, i am happy to do it as well :)

[ariebovenberg]

@CucumisSativus Thanks for the feedback!

😉 If you don't mind though, I was actually really looking forward to implementing these, after being given the green light. I'm really interested in diving into returns internals 🤩

edit: reading back I see I didn't make this clear before. How about we each take half of whatever methods get approved to add?

[CucumisSativus]

@ariebovenberg Sorry I rushed a bit and started with filter, but there are plenty of other methods and I'm happy to share them with you :). My time is a bit limited so I don't think I can realistically do more than filter this week, so please choose the methods you want to implement. Let's use this issue for coordination when it comes to implementing them.

[sobolevn]

🎉

[ariebovenberg]

@sobolevn agree to implement all methods?

• filter
• zip
• setdefault -- also agree on the name?
• and_ and or_

Then we can start the work.

[sobolevn]

@ariebovenberg you would love to have as much utils as possible, but. There's one very important step here:

We really prefer to have functions over methods. For example. We can create def and_(self, other) method in Maybe / Result / IOResult / etc. Or we can create a single function which works with any (or only some) given containers.

The difference is not just semantical. If we add these as methods, we would have to add HKT interfaces on top of that. And make our classes more complex by adding more utility methods.

Functions on the other hand only require existing interfaces and do not polute classes with extra methods.

[sobolevn]

We store utils here: https://github.com/dry-python/returns/tree/master/returns/methods

[ariebovenberg]

I'm definitely willing to defer to your preference on this matter. But let's see if I can make the case for methods...

In the absence of first-class support for currying and composition in Python, I feel methods have an advantage on readability.

x.filter(str.isdigit).or_(y).setdefault('foo')

setdefault(or_(filter(x, str.isdigit), y), 'foo')  # less readable IMHO

For these particular methods:

• zip, and_ and or_ would work OK as standalone functions (i.e. they work on all success/failure types).
• setdefault is mainly a convenience helper. Having to import it separately kind of defeats the purpose IMHO. I'd go for a method.
• filter looks like something only possible for Maybe (since it needs to be able to 'empty' a container). Seems like a method would be more appropriate?

[sobolevn]

zip, and_ and or_ would work OK as standalone functions

Awesome! 👍

Having to import it separately kind of defeats the purpose IMHO. I'd go for a method.

Ok, let's make it the very last item to implement then! We can think some more about different choices.

filter looks like something only possible for Maybe (since it needs to be able to 'empty' a container). Seems like a method would be more appropriate?

It is still fine to use it as a function, because we have MaybeLikeN (or MaybeBasedN) and people can theoretically make their own Maybe types. So, function can help to remove one extra implementaion step to them.

[ariebovenberg]

@CucumisSativus probably best if you pick up zip as well. Just like filter it only works on maybes. I'll pick up setdefault, and_, or_

[CucumisSativus]

I am happy to pick zip, but I would like to challenge that it should be only implemented for maybe.

For result I would see it like this

>>> Success(1).zip(Success("str"))
Success(1, "str")
>>>Failure(Exception("a")).zip(Success(1))
Failure(Exception("a")
>>>Failure(Exception("a")).zip(Failure(Exception("b"))
Failure(Exception("a"))
>>>Success(1).zip(Failure(Exception("a"))
Failure(Exception("a"))

For IO

>>>IO(1).zip(IO(2))
IO((1,2))

With IOResult I think it could work if both results have the same error type.

@sobolevn @ariebovenberg what do you think about it?

[ariebovenberg]

@CucumisSativus Interesting! I was thinking about the signature Result[T, U], Result[V, W] -> Result[(T, V), (U, W)], which would not be possible for arbitrary errors.

But Result[T, U], Result[V, W] -> Result[(T, V), U | W] is indeed possible! The signature still leaves unclear whether to take the first or second failure value. I'd agree with you that taking the first is 'more correct', as (1) zip is semantically similar to and (which also returns the first failure), and (2) this makes zip consistent with apply. So seems like a good idea to me!

Makes sense to implement for IO indeed -- in fact, for all applicatives 🤔

[ariebovenberg]

Also a decision to make: if we make zip a standalone function, it'd conflict with Python's builtin zip. We need to choose one of:

1. it's up to the library users to import zip in a qualified manner
2. we rename zip to zip_ or something else
3. we make zip a method

[djbrown]

any news here? I would love to use those utilities 🤩

[ariebovenberg]

@djbrown I suppose not 😅 . Feel free to pick this up if you like, I probably won't have time for it the coming months.

[ryangrose]

I feel like a lot of these can easily be created with do notation can't they?

# Zip
Maybe.do(
  (first, second)
  for first in Success(1)
  for second in Success("Str")
)
# And
Maybe.do(
  first and second
  for first in Success(5)
  for second in Success(9)
)
# Or
Maybe.do(
  first or second
  for first in Success(5)
  for second in Success(9)
)

It seems like there's a common pattern here, so maybe a function to lift a function to operate on two monads could be more useful.

def map_n(f):
  def wrapped(*args):
    return Maybe.do(
      f(next(arg) for arg in args)
   )
  return wrapped

map_n(zip)(Some(1), Some(2))

I'm slightly hesitant about adding primitives if they're easy to lift though? What do you all think? (Relatedly, I had a hard time doing any of this in pointfree syntax.. Let me know if anyone can think think of a cleaner approach)

And to me filter seems inconsistent: why would it return Nothing instead of Some("")? It seems to unnecessarily flatten the data just because "" is false-y. Consider:

def get_prices_from_db() -> Maybe[List[int]]:
    ...

prices = get_prices_from_db() 
prices.filter(lambda price: price > 5).map(len)  # This should probably return Some(0) instead of Nothing if the data existed

In other words, filter unnecessarily changes the type: an empty list is still a valid list but Nothing represents invalid state. The goal of a container like this is to abstract out the actual container and compose pure functions. This would make that difficult.

Setdefault seems very promising but I agree it needs a new name. Rust has a similar function that they call or_else. Perhaps that could be be a good name here.

[ariebovenberg]

@ryangrose

1. zip, or_ and and_ can indeed be expressed otherwise, but the whole appeal is to have one call like a.zip(b) instead of a multi line do-expression.
2. In my original proposal (which I just copied from Rust, and I believe Haskell implements as well), I imagined or_, and and_ to explicitly not act on the contained value.

>>> Some(1).and_(Some(9))
Some(9)
>>> Some(0).and_(Some(9))
Some(9)  # it shouldn't matter that 0 is falsey, the `and_` method only cares about Some/Nothing

3. Similarly, filter shouldn't have special behavior in case of iterable content. In that case the contained type never changes and the issue you mention does not exist.

>>> Some(9.4).filter(float.is_integer)
Nothing
>>> Some(3.0).filter(float.is_integer)
Some(3.0)
>>> Nothing.filter(float.is_integer)
Nothing

iterable values aren't special:

>>> def has_at_least_3_items(a: list) -> bool:
...        return len(a) >= 3
...
>>> Some([5, 3]).filter(has_at_least_3_items)
Nothing
>>> Some([1, 4, 8, 2]).filter(has_at_least_3_items)
Some([1, 4, 8, 2])
>>> Nothing.filter(has_at_least_3_items)
Nothing

4. or_else seems like a fine name instead of setdefault, although or_else in Rust's variant is lazy (it takes a FnOnce)

[ryangrose]

Ah that makes much more sense. Thanks @ariebovenberg!

Definitely agree on the appeal. I was worried about adding methods that act on the value of containers to a container since that seemed superfluous. Now that I know they aren't acting on the value of the containers I see how they're worth it :)

Yeah the lazy evaluation was going to be my next question. I imagine that was for performance implications: don't need to calculate or allocate for the condition unless it's met. What do you think about that? I'm leaning towards a value/eager evaluation like your proposal since it's simpler to use.

And while we're at it, looks like rust has some similar functions that look nice: like zip_with, unzip, map_or, contains, etc. Should we include these as well? Seems like they'd be easy to implement while we're making the others