Better-typed string context

[Ericson2314]

The string context current is made up various patterns like !out!/path/to/drv and =/path/to/drv. Are these encodings intrinsically important, or could we replace with a std::variant?

CC @edolstra

[edolstra]

They're not intrinsically important but they avoid another allocation.

[Ericson2314]

@edolstra are you worried about indirection or just having multiple strings? I was thinking something like:

// probably exists already
struct StorePathWithOutput {
    StorePath path;
   std::string output;
}; 

// is that what the `=` thing means?
struct Closure {
    StorePath path;
};

typedef std::variant<StorePathWithOutput, Closure> StringCtxItem;

so more strings with things split, but not more indirection since the variant is stored directly in the set. This seems fine to me?

[edolstra]

Currently contexts are stored in Value::string as a const char * *, which is one heap allocation for the array and one allocation per context string. If we changed this to a const StringCtxItem *, then we get an additional heap allocation for each StorePath::baseName and maybe another heap allocation for output.

[Ericson2314]

So we have a const char[][] (trying to make the arrays clear) and are going to a const StringCtxItem[]. That means before we had heap allocs for:

• each context item, a string context item []
• the context, an array of strings const char [] []

Now we have them for:

• std::strings/StorePaths of each item (drv path, output, etc)
• the contex, an array of variants const StringCtxItem []. Note the the variants are stored directly in the array.

So what exactly is worse about the second option?

• there's no more indirection
• the context array is bigger
• the items are smaller but more numerous
• the total allocated bytes should be roughly the same

Are you just worried about the constant factor space and time overhead of fine-grained allocations?

[stale]

I marked this as stale due to inactivity. → More info

[stale]

I closed this issue due to inactivity. → More info

[Ericson2314]

Since #6237 "deocoded" string contexts are better typed, but we should still do something about the interpreter heap original.