DO evaluation of path! and lit-path! not consistent with word! and lit-word!

[rebolbot]

Submitted by: cyphre2

Evaluation of path!s and word!s using DO in R3 differs from R2 and looks buggy and inconsistent.
Noone on Altme was able to tell if the change was intentional or it is just some bug. But it looks more like a bug.

In R2 (semi-consistent but wrong, not fixable):
>> a: [[1 + 2]]
== [[1 + 2]]
>> b: [a a/1]
== [a a/1]
>> do first b
== [1 + 2]  ; evaluated twice, should be [[1 + 2]]
>> do second b
== [1 + 2]
>> b: ['a 'a/1]
== ['a 'a/1]
>> do first b
== 'a  ; should be word! not lit-word!
>> do second b
== 'a/1  ; should be path! not lit-path!

In R3 (correct with any-words, not with any-paths):
>> a: [[1 + 2]]
== [[1 + 2]]
>> b: [a a/1]
== [a a/1]
>> do first b
== [[1 + 2]]  ; correctly just dereferences a
>> do second b
== a/1  ; should evaluate the path to [1 + 2], not return a/1
>> b: ['a 'a/1]
== ['a 'a/1]
>> do first b
== a  ; correctly converts lit-word! to word!
>> do second b
== 'a/1  ; should convert lit-path! to path!

^{CC - Data [ Version: alpha 96 Type: Bug Platform: Windows Category: Native Reproduce: Always Fixed-in:none ]}

[rebolbot]

Submitted by: BrianH

Noted the R2 bugs in the code, and annotated the R3 code with the intentional changes. R3 DO's treatment of any-words corrects R2's bugs, but its treatment of any-paths needs work.

[rebolbot]

Submitted by: mikerev

where is the logic?
a: 11 t: [a :a 'a /a]
r: [] foreach x t [append r get/any x] r
== [11 11 11 11]
r: [] foreach x t [append r do x] r
== [11 11 11 /a]
r: [] foreach x t [append r get/any :x] r
== [11 11 11 11]
r: [] foreach x t [append r do :x] r
== [11 11 a /a]
reduce t
== [11 11 a /a]

[rebolbot]

Submitted by: BrianH

A lit-word is an active value, so when your second example assigns 'a to x, evaluating x returns the word! a, which is then GET/any'd by DO. To give you a reason why DO word! is treated as GET/any, look at this example:

>> a: does [11]
>> t: [a :a 'a /a]
== [a :a 'a /a]
>> map-each x t [get/any x]
== [make function! [[][11]] make function! [[][11]] make function! [[][11]] make function! [[][11]]]
>> map-each x t [do x]
== [make function! [[][11]] make function! [[][11]] make function! [[][11]] /a]
>> map-each x t [get/any :x]
== [make function! [[][11]] make function! [[][11]] make function! [[][11]] make function! [[][11]]]
>> map-each x t [do :x]
== [make function! [[][11]] make function! [[][11]] a /a]
>> reduce t
== [11 make function! [[][11]] a /a]

By having DO word! be the same as GET/any word! we can avoid having to check word! values before DO'ing them, which is more secure.

[rebolbot]

Submitted by: BrianH

When you DO active values, it would be less confusing if that was consistent with how those active values are treated when DO encounters them inline in a code block. Afaik, in R3 all active value types except lit-path are treated consistently in this way. Do you have any examples of other types for which that isn't true? We want to set the right balance here.

The word! and path! types are not really active values in R3, they are just special cases in the evaluation rules of inline code. We don't want them to be active values because their evaluation can cause side effects beyond the behavior of the type itself. It makes sense to have function types stay active since side effects are already part of the type itself, so they're expected.

The behavior of the lit-word! type was changed on purpose, but I can't remember what the practical (instead of principled) reason was, and the code that needed this change may no longer exist - at the time, all R3 code was in Carl's GUI or the mezzanines, and the mezzanines don't use lit-words explicitly. If you have a compelling reason to define the lit-word/lit-path evaluation inline as a special case, same as word/path, please give it here. We have to draw the line somewhere that includes functions in the active side, integers and such on the non-active side, paren!, word! and path! as a special case, and block! as a different special case, but it's a bit arbitrary after that.

All of the mezzanine code is already careful about active values, so it wouldn't be affected either way. If any user R3 code isn't careful about active values, it's a bug. Being careful about active values can mean excluding them by type tests or data flow analysis, so it doesn't necessarily mean using get-words all over the place.

Whatever we choose, lit-path! and lit-word! need to be treated consistently with each other and so do word! and path!, so this ticket needs fixing either way. It would be a good idea for the R3 fix to the behavior of the word! type to stay, as long as path! behavior is consistent with it (reverted to R2 behavior).

If it helps, here is one argument in favor of making lit-words and lit-paths a special case instead of fully active: When they are active, you have to special-treat all values that are screened to be part of any-word! or any-path! in case they might be lit-word! or lit-path!. That makes code a little more awkward at times. If only function types are fully active (are there other active values except functions, lit-word and lit-path?) they can be easily excluded from argument type specs, or screened for by type tests. That would make code a little easier to write.

[rebolbot]

Submitted by: BrianH

A more direct answer to your question, Ladislav, in code for R2:

>> 'a
== a  ; inline evaluation of lit-word converts to word
>> b: quote 'a  b
== a  ; regular evaluation of lit-word value converts to word
>> do quote 'a
== 'a  ; explicit evaluation of lit-word value does not convert to word (inconsistency)

>> 'a/1
== a/1  ; inline evaluation of lit-path converts to path
>> b: quote 'a/1  b
== a/1  ; regular evaluation of lit-path value converts to path
>> do quote 'a/1
== 'a/1  ; explicit evaluation of lit-path value does not convert to path (inconsistency)

That's R2's inconsistency. For R3:

>> 'a
== a  ; inline evaluation of lit-word converts to word
>> b: quote 'a  b
== a  ; regular evaluation of lit-word value converts to word
>> do quote 'a
== a  ; explicit evaluation of lit-word value converts to word

>> 'a/1
== a/1  ; inline evaluation of lit-path converts to path
>> b: quote 'a/1  b
== 'a/1  ; regular evaluation of lit-path value does not convert to path (inconsistency)
>> do quote 'a/1
== 'a/1  ; explicit evaluation of lit-path value does not convert to path (inconsistency)

Now, we could replicate the consistent behavior of R3's lit-words to its lit-paths, or we could replicate the inconsistent behavior of its lit-paths to its lit-words (what was referred to above as "making lit-words and lit-paths a special case" like words and paths are). Either is a valid choice, as long as we make the lit-word and lit-path behavior consistent between each other.

[rebolbot]

Submitted by: Ladislav

Glossary note. Brian coined some notions that should lessen the confusion when discussing evaluations of REBOL values.

Note: in the code below we use the QUOTE function to suppress the inline evaluation where we don't need it to occur.

===Inline evaluation

The first one is the notion of "inline evaluation". By "inline evaluation" is meant an evaluation of a value encountered typically in a REBOL block, when a block is evaluated by DO. Examples:

; REBOL blocks evaluate to themselves when inline-evaluated:
[a block] ; == [a block]

; REBOL paren values do not evaluate to themselves when inline-evaluated:
(1 + 3) ; == 3

===Explicit evaluation

The notion of "explicit evaluation" Brian suggested to use for the case when a REBOL value is supplied as an argument to the DO function. Examples:

; REBOL blocks do not evaluate to themselves when explicitly evaluated:
do quote [1 + 2] ; == 3

===Indirect evaluation

When a REBOL word WORD (of a word! datatype) is evaluated, the result of such an evaluation depends on (the type of) the value the word refers to. We may say, that the value assigned to the WORD is evaluated indirectly. Examples:

; parens evaluate to themselves when indirectly evaluated:
a-paren: quote (1 + 2)
a-paren ; == (1 + 2)

; functions don't evaluate to themselves when indirectly evaluated:
a-function: does ["OK"]
a-function ; == "OK"

[rebolbot]

Submitted by: Ladislav

My preferences, and, after performing related polls, I would say preferences of the majority of respondents as well regarding the inline, explicit and indirect evaluations of lit-paths and lit-words are summarized below (i.e. below are shown the preferred results):

>> 'a
== a
>> b: quote 'a  b
== 'a 
>> do quote 'a
== a
>> 'a/1
== a/1 
>> b: quote 'a/1  b
== 'a/1 
>> do quote 'a/1
== a/1

[rebolbot]

Submitted by: BrianH

Works for me. This would let us safely work with indirectly referenced lit-word and lit-path values without converting them to words and paths accidentally, and it would make direct DO evaluation consistent whether DO is called explicitly (as a function) or implicitly (for inline evaluation). This is similar to the way parens work. It would also mirror the pattern of behavior proposed for path! (#1881) and word! (#1882).

[rebolbot]

Submitted by: BrianH

Ladislav's solution was requested again in #2101.

[rebolbot]

Submitted by: BrianH

Ladislav's solution implemented in rebol/rebol#184