WISH: Pass an expression to '.expr' as-is to mirai()

[HenrikBengtsson]

Consider the following example:

a <- 3
b <- 4
m <- mirai(2 * a + b, .args = list(a, b))

Issue

Now, assume that I have that R expression and a list of variables as standalone objects, e.g.

.expr <- quote(2 * a + b)
a <- 3
b <- 4
globals <- list(a = a, b = b)

To use this setup with mirai(), I need to do something like:

.expr2 <- bquote({
  local({
    envir <- parent.env(environment())
    for (name in names(globals)) {
      assign(name, value = globals[[name]], envir = envir)
    }
  })
  .(.expr)
})

call_expr <- bquote(mirai(.(.expr2), .args = list(globals)))
m <- eval(call_expr)

Wish

If there would be an option to not substitute() the .expr argument, then I could instead do:

m <- do.call(mirai, args = c(list(.expr), globals, .substitute = FALSE))

[wlandau]

Related: @HenrikBengtsson, I like the substitute argument in future::future() with default TRUE. Development crew recently adopted this: wlandau/crew#63.

[wlandau]

I will also add that I found .args to be a bit different from my initial expectation. I would have expected the following to return 7:

library(mirai)
m <- mirai(2 * a + b, .args = list(a = 2, b = 3))
m$data
#> 'miraiError' chr Error in eval(expr = envir[[".expr"]], envir = envir, enclos = NULL): object 'a' not found

If that had worked, then the following would have worked as well.

args <- list(.expr = quote(2 * a + b), .args = list(a = 2, b = 3))
m <- do.call(what = mirai, args = args)

[wlandau]

The current solution for crew is to create an argument list that includes .expr next to arguments like a and b and then use do.call(what = mirai, args = "...") on the whole thing.

[shikokuchuo]

I can see the logic of having a substitute argument, and yes it is useful to pass in pre-constructed language objects.

However the evaluation you attempted can be achieved simply by:

m <- mirai(.expr, .args = list(.expr, a, b))

m$data
[1] 10

[HenrikBengtsson]

I will also add that I found .args to be a bit different from my initial expectation.

I actually had a long second "exposition" on this part, but decided to save it for later. I found it surprising that it uses substitute() to infer variable names and to pull in their values into the internal arglist list, instead of just appending the content as-is. Other than having to type a = a, instead of just a, I don't see how:

m <- mirai(2 * a + b, .args = list(a, b))

adds much benefit over using:

m <- mirai(2 * a + b, a = a, b = b)

While look the code for this, I realize that:

m <- mirai(2 * a + b, .args = c(a, b))

works too, which to me is one step closer to what's happening under the hood. FWIW, if of any help to get ideas, in future(), the latter is achieved as:

f <- future(2 * a + b, globals = c("a", "b"))

The disadvantage of specifying variable as character strings, is that you can't catch mistakes by code inspection; a typo in a name will only be detected at run time. Continuing, future() also supports:

f <- future(2 * a + b, globals = list(a = 3, b = 4))

and a few other use cases.

[HenrikBengtsson]

However the evaluation you attempted can be achieved simply by: ...

m <- mirai(.expr, .args = list(.expr, a, b))

Interesting, and honestly ... a bit "scary" :) My gut feeling is that this is takes non-standard evaluation (NSE) to another level, and it feels like you would have to understand what's going on under the need to be comfortable in using that. Also, it seems to only work if you call it exactly .expr. For example, the following doesn't work for me:

E <- quote(2 * a + b)
a <- 3
b <- 4
m <- mirai(E, .args = list(E, a, b))
m$data
## 'unresolved' logi NA

[shikokuchuo]

Interesting, and honestly ... a bit "scary" :) My gut feeling is that this is takes non-standard evaluation (NSE) to another level, and it feels like you would have to understand what's going on under the need to be comfortable in using that.

Would have been nice if it were really 'scary' :) Unfortunately as your example show, this doesn't work. The '.expr' provided in '.args' simply overwrites the expression and is evaluated as is i.e. non-substituted, hence works.

I guess if there is no better method then mirai should have a '.substitute' argument as suggested.

[shikokuchuo]

Having said that, what you actually want to do is to evaluate the language object, in which case:

m <- mirai(eval(E), .args = list(E, a, b))

m$data
[1] 10

[shikokuchuo]

E <- quote(2 * a + b)
a <- 3
b <- 4
m <- mirai(E, .args = list(E, a, b))
m$data
## 'unresolved' logi NA

@HenrikBengtsson Thanks for helping me find a bug - the above should have worked, and now does - returning the language object below:

E <- quote(2 * a + b)
a <- 3
b <- 4
m <- mirai(E, .args = list(E, a, b))
m$data
2 * a + b

It was a general failure to send serialized language objects (as they were being evaluated in the call to serialise at the C level). More importantly it allows evaluation of, for example:

a <- "a + 1"
m <- mirai(str2lang(a), a = a)
> m$data
a + 1

Then it is a question of whether a separate interface for passing in calls makes sense, or as I mention above - just using eval().

[shikokuchuo]

Fixing the above has convinced me that language objects are indeed a special case, and deserve to be handled as such.

I have implemented a simple method for detecting language objects passed as '.expr' and sends those unsubstituted, so the below now works in mirai 0.8.2.9009.

lang <- quote(a + b + 2)
a <- 2
b <- 3
m <- mirai(lang, .args = list(a, b))
call_mirai(m)$data
[1] 7

I would like to know what you both think.

I am trying to avoid a .substitute argument as [1] a core design principle for mirai is simplicity and [2] it is not going to be well understood by 99% of users.

-> In cc9b375 v0.8.2.9016 the method has been updated to a much more efficient one-liner. Hardly any overhead.

[HenrikBengtsson]

I'm confirming that the following works with mirai 0.8.2.9036:

.expr <- quote(2 * a + b)
globals <- list(a = 3, b = 4)
m <- do.call(mirai::mirai, args = c(list(.expr), globals))
m$data
## [1] 10

I also verified that .expr is not evaluated until on the worker;

.expr <- quote(stop("boom"))
m <- do.call(mirai::mirai, args = list(.expr))
m$data
## 'miraiError' chr Error in eval(expr = envir[[".expr"]], envir = envir, enclos = NULL): boom