Some properties are not retrieved; others are retrieved inefficiently

[Lexikos]

There are several cases where the extension fails to get a property's value:

the := {__get: (*) => 42}
; @Debug-Output => {the.answer}
OutputDebug the.answer "`n"

c := ComValue(9, ObjPtrAddRef({Value: "actual property value", Value2: ":("}))
; @Debug-Output => {c.Value}
OutputDebug c.Value "`n"
; @Debug-Output => {c.Value2}
OutputDebug c.Value2 "`n"

d := ComObject("Scripting.Dictionary")
; @Debug-Output => {d.Count}
OutputDebug d.Count "`n"

; Not supported by property_get in v2.0.13 or v2.1-alpha.9:

s := "str"
(Any.DefineProp)(s.base, 'Length', {get: StrLen})
(Any.DefineProp)(s.base, '__Item', {get: SubStr.Bind(,,1)})
; @Debug-Output => {s.Length}
OutputDebug s.Length "`n"
; @Debug-Output => {s[2]}
OutputDebug s[2] "`n"

a := ["L"]
; @Debug-Output => {a.__Item[1]}
OutputDebug a.__Item[1]

The last two are not supported in v2.0.13 or v2.1-alpha.9, but if they are supported by the engine, v1.11.0 of the extension still fails to show their values. The rest are supported by property_get and work correctly in other debug clients.

v2.1-alpha.10 will be changing the debugger to enumerate inherited properties, but will omit Base and __Class. When these properties are omitted, v1.11.0 of the extension fails to retrieve their values, even though the engine supports it.

---

The current process for property evaluation is far less efficient than it should be. Due to the presence of dynamic properties, it can also have more side-effects than necessary. I did not fully understand what I was seeing at first, so did extensive debugging.

When a property value is requested, the extension currently appears to do the following:

• evaluate potentially retrieves the stack and resolves variables between brackets. I see no problem here.
• Every time evaluate is called, it sends the context_names command. AutoHotkey returns a constant string (regardless of the stack), but maybe there is some reason not to cache the result with AutoHotkey_H, or maybe not. If not, it is just adding latency for nothing, but this is not the main concern.
• For each context, evaluate starts by calling existsProperty.
  • For a variable x, it returns true without sending any commands.
  • For a property x.y, the loop iterates once and queries x. This causes the engine to enumerate own properties of x and evaluate all enumerated dynamic properties.
  • If y exists as a child of x, the function returns true, the property exists. The property value, which was already evaluated by the engine and returned to the extension, is discarded.
  • If y does not exist as a child of x, fetchInheritedProperty is called to recursively search x.<base>, x.<base>.<base>, etc. Again, this causes the engine to evaluate all enumerated dynamic properties.
  • For a property x.y.z or x.y[1], this process is repeated for the second component if it was determined that y exists within x.
• If existsProperty returned false, the remainder of the iteration is skipped.
• safeFetchProperty is called to retrieve the property, even though existsProperty likely already had its value.
  • The parent property (x in x.y or x.y in x.y[1]) is retrieved to determine its type, even though it was already retrieved by existsProperty.
  • If the parent property is not a COM object, Prototype or Class, fetch the originally requested property and return... at last... or not!
    • If a variable is requested from the Auto-execute thread stack frame, fetchProperty fetches the property (from context 0 first due to how evaluate works) and then discards it.
      • It is predetermined that the property will be discarded, so it would be better to check the caller-provided context first and avoid calling sendPropertyGetCommand if the workaround is to be applied.
      • All of the work done by existsProperty up to this point is wasted. It would be even better to perform the check at the beginning of the iteration in evaluate, where context is retrieved from contexts.
      • I think the referenced bug was fixed by AutoHotkey/AutoHotkey@db6ee5cd (v1.1.34.04 and prior to v2.0). The bug itself has relatively low impact and only affected versions between August 2018 and August 2022, so I would just remove the workaround.
    • "a COM object" currently really means "a v1 COM object wrapper of type VT_DISPATCH which lacks an event sink". v2 ComValue lacks the DispatchType and DispatchIID properties, and v1 COM objects have additional properties if they have an event sink (ComObjConnect).
  • If the object is a Prototype or Class, additional steps are taken to "workaround the issue of getting dynamic properties directly, which causes errors".
    • I have no idea why Class is included, as I don't recall there being any issues getting a property of a class. Classes are essentially just normal objects.
    • I am unsure why Prototype is included, as the only issue I can think of is that dynamic properties cannot be evaluated without a proper instance. In other words, Map.Prototype.Count will correctly fail, as will A_Args.base.Length both in script and by the debugger. If the user is hovering over something like that which attempts to evaluate a property on a prototype, applying the workaround to get a non-error result would be incorrect.
    • The workaround retrieves the parent property again, but this time with depth + 1, presumably so that the child property (if present) will have the correct depth. Because of the extra depth, the number of properties evaluated and returned can be exponentially larger.
    • The fake properties of COM objects (Value, VarType, etc.) can be retrieved with this workaround, but I am unsure that there is ever a need to directly retrieve them. They can be considered like attributes of the parent COM object. They cannot be evaluated by script, so for instance, comObj.Value should just invoke object, like with property_get -n comObj.Value.
    • I think that the workaround could have been applied to RegExMatchInfo in v1, as it has some nested (parameterised) properties which the debugger can only retrieve by requesting the match object itself.
    • If the workaround is deemed necessary, it could still be avoided in 99.99% of cases by just querying for the originally requested property first and skipping all of the other work if it is successful.

---

The essence of the issue is that the extension uses a roundabout method to determine whether a property exists before querying it, instead of just querying the property, which in almost all cases will give the right answer (even if the answer is undefined).

To demonstrate what I mean about evaluation of dynamic properties, this simple case prints 3, but a is 4 if the variable list is updated.

a := 0, z := 0
x := {}
x.DefineProp('a', {get: ((x, &v) => ++v).Bind(,&a)})
x.DefineProp('z', {get: ((x, &v) => ++v).Bind(,&z)})
; @Debug-Output => {x.a}
; @Debug-Breakpoint
Exit

"Dynamic properties" include methods without getters. If {x.a} above is changed to {x.y} (which is undefined), the following dynamic properties are evaluated by invoking the object (with Object::Invoke, an internal virtual method similar in nature to IDispatch::Invoke):

x.a
x.z
x.<base>.Clone
x.<base>.DefineProp
x.<base>.DeleteProp
x.<base>.GetOwnPropDesc
x.<base>.HasOwnProp
x.<base>.OwnProps
x.<base>.<base>.__Init
x.<base>.<base>.Base
x.<base>.<base>.GetMethod
x.<base>.<base>.HasBase
x.<base>.<base>.HasMethod
x.<base>.<base>.HasProp
x.<base>.Clone
x.<base>.DefineProp
x.<base>.DeleteProp
x.<base>.GetOwnPropDesc
x.<base>.HasOwnProp
x.<base>.OwnProps
x.<base>.<base>.__Init
x.<base>.<base>.Base
x.<base>.<base>.GetMethod
x.<base>.<base>.HasBase
x.<base>.<base>.HasMethod
x.<base>.<base>.HasProp
x.<base>.<base>.__Init
x.<base>.<base>.Base
x.<base>.<base>.Base.Clone
x.<base>.<base>.Base.DefineProp
x.<base>.<base>.Base.DeleteProp
x.<base>.<base>.Base.GetOwnPropDesc
x.<base>.<base>.Base.HasOwnProp
x.<base>.<base>.Base.OwnProps
x.<base>.<base>.GetMethod
x.<base>.<base>.HasBase
x.<base>.<base>.HasMethod
x.<base>.<base>.HasProp

Most of these cases just return the function object, but think about what happens if the bases have more dynamic properties. All I'm trying to do is retrieve the value of x.y, which property_get -n x.y gives. Maybe it is defined sometimes, or maybe I made a typo. (Side note: Perhaps there should be an option or format specifier to provide replacement text for unset in the event that a property is undefined. Another side note: Maybe in future you will support hovering over x?.y and similar.)

I had one case where a property was being evaluated 10 times instead of once; or it might have been a separate property that shouldn't have been evaluated at all. I do not remember what combination of inheritance, property getters, debug directives and console, etc. triggered this.

---

If you wish to work around the fact that some properties cannot be retrieved directly, I suggest this approach:

• First, attempt to retrieve the property directly. Usually this works.
• If the property is undefined, drop .name[] or [] and query again with depth = 0 to determine its type. This will avoid executing any property getters.
• If the parent is a type which is known to have this issue, query again with depth = 1 or 2.

I suppose that it would only be needed if hovering over something like match.Pos[2] in v1. Actually, v2 has match.Pos[2] but doesn't return it in the parent property. property_get will be able to retrieve it in v2.0.14 and v2.1-alpha.10.

[zero-plusplus]

Thanks for the detailed debugging.

I am currently creating an evaluation process and will refer to it.

I completely agree that the existsProperty and safeFetchProperty are unnecessary.

Prototype and Class may have been the target of the process because the source of the bug was not fully understood.

I was not aware of a way to work around the problem by setting depth to 0 to get the type.
It would be a good workaround for a specific version as well as a good solution for other problems.

---

I am currently writing the evaluation process (For convenience, I refer to this evaluation machine as AELL for short for AutoHotkey Expression Like Language.), but I made some of the same mistakes because I was using a general evaluation method.

I should only do my own evaluation for things that property_get does not support, such as `a[ “k” “e” “y” ], and only need to do property_get once for everything else.

---

This report also brought a lot of inspiration. I thank you.
I will do my best for a better debugger environment.

[zero-plusplus]

I am working on a transformer to optimise AELL ASTs in order to reduce the use of the property_get command as much as possible.

I then noticed two differences.

When retrieving a child element of a nested object, such as a.b.c, the traditional method of retrieving it incrementally as a, a.b and a.b.c results in the fullname being a.b.c and the name as c.

But if I get a.b.c directly with property_get, both fullname and name are a.b.c.

Is this difference a specification?

[Lexikos]

Property names are not part of the DBGp spec; they are language-specific.

fullname is what you should pass back to property_get -n to query the property.

name has no real meaning. "This is the short part of the name, and is meant to be displayed in the IDE's UI." You can display whatever you want; the DBGp spec has no say in how you implement your UI.

If you are displaying properties in a hierarchal layout such as a tree, you probably want the short names c, b, a. However, you don't get b or a by querying property_get -n a.b.c.

If you were querying a property for a watch list or hover, you're probably going to display the full name a.b.c, which is what property_get -n a.b.c returns in name.

Either way, you know which property you are querying and can display whatever you want without relying on fullname or name. Oftentimes, the property is already being displayed and there is no reason to consider getting its name from the response.

When property_get -n a returns two levels of child properties, name is the property name or index/key, and fullname is derived from the caller-provided property name (a) and the name of each child property. By contrast, when you query property_get -n a.b.c, the engine has only a.b.c and no need to parse the short name from that. If it did choose some suffix as the short name, you would still need to make assumptions about the property name syntax (which is not formally defined) in order to determine the parent name.

[zero-plusplus]

Thank you for the explanation. I am convinced.

I didn't have a problem, but I didn't know if it was a bug or a specification, so I asked the question.

---

PS

I thought it was today's message, but now I realise I missed the notification...