Heapsnapshot has some unrooted objects

[NHDaly]

The heapsnapshots seem to be missing either some edges or maybe some roots, or maybe both, but somehow many objects end up not retained by any roots, which is of course nonsense, since they couldn't have been exported if they're not reachable from a root (since unrooted things are garbage).

You can see that in this simple repro:

julia> using Profile

julia> Profile.take_heap_snapshot("1.11.0-DEV.638.heapsnapshot")
"1.11.0-DEV.638.heapsnapshot"

1.11.0-DEV.638.heapsnapshot.zip

You can see in the screenshot that the roots are only retaining 87% of the heap snapshot. (in another attempt, i got only 5,000 bytes out of 120 MB!)

And you can see here that one random object has retainers that don't terminate in a root:

From randomly clicking around so far, it seems to often be Arrays that aren't parented, and so i wonder if we're missing some edge somewhere for arrays?

[NHDaly]

(@JianFangAtRai from RAI is going to look into this this month.)

[IanButterworth]

vscode just added a built-in .heapsnapshot viewer which looks to have an interesting graph viewer. However trying to open a julia-generated .heapsnapshot file just spins without any indication why (I found that chrome generated ones load immediately).

Perhaps this issue is related?

[JianFangAtRai]

vscode just added a built-in .heapsnapshot viewer which looks to have an interesting graph viewer. However trying to open a julia-generated .heapsnapshot file just spins without any indication why (I found that chrome generated ones load immediately).

Perhaps this issue is related?

Are the sizes of the two files equal or close? Could the file size be an issue here?

[IanButterworth]

#52618 didn't fix the vscode viewer hang unfortnately

File sizes:

• from a chrome webpage: 5 MB (renders instantly)
• from a fresh julia session: 175 MB (hangs indefinitely and reports Error: invalid profile JSON)

2023-12-30 19:27:38.621 [info] ExtensionService#_doActivateExtension ms-vscode.vscode-js-profile-table, startup: false, activationEvent: 'onCustomEditor:jsProfileVisualizer.heapsnapshot.table'
2023-12-30 19:27:38.772 [error] UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "invalid profile JSON".

Asked upstream at microsoft/vscode#199239 (comment)

---

both files validate in jsonlint.com

Headers:
julia's

{
    "snapshot": {
        "meta": {
            "node_fields": [
                "type",
                "name",
                "id",
                "self_size",
                "edge_count",
                "trace_node_id",
                "detachedness"
            ],
            "node_types": [
                [
                    "synthetic",
                    "jl_task_t",
                    "jl_module_t",
                    "jl_array_t",
                    "object",
                    "jl_datatype_t",
                    "String",
                    "jl_sym_t",
                    "jl_svec_t"
                ],
                "string",
                "number",
                "number",
                "number",
                "number",
                "number"
            ],
            "edge_fields": [
                "type",
                "name_or_index",
                "to_node"
            ],
            "edge_types": [
                [
                    "internal",
                    "element",
                    "hidden",
                    "property"
                ],
                "string_or_number",
                "from_node"
            ]
        },
        "node_count": 911769,
        "edge_count": 3024013
    },

Chrome's

{
    "snapshot": {
        "meta": {
            "node_fields": [
                "type",
                "name",
                "id",
                "self_size",
                "edge_count",
                "trace_node_id",
                "detachedness"
            ],
            "node_types": [
                [
                    "hidden",
                    "array",
                    "string",
                    "object",
                    "code",
                    "closure",
                    "regexp",
                    "number",
                    "native",
                    "synthetic",
                    "concatenated string",
                    "sliced string",
                    "symbol",
                    "bigint",
                    "object shape"
                ],
                "string",
                "number",
                "number",
                "number",
                "number",
                "number"
            ],
            "edge_fields": [
                "type",
                "name_or_index",
                "to_node"
            ],
            "edge_types": [
                [
                    "context",
                    "element",
                    "property",
                    "internal",
                    "hidden",
                    "shortcut",
                    "weak"
                ],
                "string_or_number",
                "node"
            ],
            "trace_function_info_fields": [
                "function_id",
                "name",
                "script_name",
                "script_id",
                "line",
                "column"
            ],
            "trace_node_fields": [
                "id",
                "function_info_index",
                "count",
                "size",
                "children"
            ],
            "sample_fields": [
                "timestamp_us",
                "last_assigned_id"
            ],
            "location_fields": [
                "object_index",
                "script_id",
                "line",
                "column"
            ]
        },
        "node_count": 58856,
        "edge_count": 245569,
        "trace_function_count": 0
    },

[NHDaly]

Seems worth looking into. Thanks for the report, Ian! Can you please move this into a separate ticket so we can track it even after Jian's PR closes this issue? Maybe we should also open a ticket on the VSCode repo with one of the snapshot files, since Chrome can load it just fine? Thanks!

…

On Sat, Dec 30, 2023, 5:32 PM Ian Butterworth ***@***.***> wrote: #52618 <#52618> didn't fix the vscode viewer hang unfortnately File sizes: - from a fresh julia session: 175 MB (hangs indefinitely) - from a chrome webpage: 5 MB (renders instantly) — Reply to this email directly, view it on GitHub <#52432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMCIELFWTSPHGJ7DPCTHELYMCXABAVCNFSM6AAAAABAKDMS42VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSGYZTMMBSGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[IanButterworth]

Will do. So #52618 should have closed this issue?

[NHDaly]

I think so, yeah! 👍 Lemme add that to the PR description.

[NHDaly]

Ah, @JianFangAtRai even already had it in PR description already:

This PR is to fix the issue: #52432

I think it just didn't get picked up by GitHub's automations. Closing this ticket now. Thanks again Jian!