core: "Could not find parent of" error in FragmentedTreeWriter for tree.json

[sorawee]

Describe the bug

In #34478, FragmentedTreeWriter was introduced to prevent tree.json from exceeding 512MB. When I play around with the maxNodes value, I find that choosing a low value (5000 for my application) causes the error "Error: Could not find parent of <root node>".

Because our project is private, I unfortunately cannot provide its code. However, I'll try to create a small example that triggers the bug if I get a chance to.

At first glance, do you see any issues that may have caused this, though? @rix0rrr @QuantumNeuralCoder

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Library Version

No response

Expected Behavior

No error

Current Behavior

There's an error as described above.

Reproduction Steps

N/A for now

Possible Solution

No response

Additional Information/Context

No response

AWS CDK Library version (aws-cdk-lib)

2.198

AWS CDK CLI version

2.1011.0 (build cd4ec06)

Node.js Version

v24.0.2

OS

Mac

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

Hi @sorawee,

Thank you for reporting this issue. After investigating the code, I can see that the error occurs in the FragmentedTreeWriter when trying to find the parent of a node during tree fragmentation.

The issue appears when using a maxNodes value of 5000, which is significantly lower than the default value of 500,000. The default was chosen assuming an average node size of 1kB per node, providing a safety margin for the 512MB file size limit.

When using such a low maxNodes value, the aggressive fragmentation appears to cause issues with the parent-child relationship tracking in the FragmentedTreeWriter. Specifically, the error occurs in the addToExistingTree method when trying to find a grandparent node that should exist but isn't found in the parent tracking map.

As a temporary workaround, I would recommend:

1. Try using a higher maxNodes value, closer to the default:

const app = new App({
  context: {
    '@aws-cdk/core.TreeMetadata:maxNodes': 100_000 // or higher
  }
});

I'll bring this up to the team for inputs.

[rix0rrr]

That shouldn’t happen, but can you tell me what you are trying to achieve by changing that parameter?

[sorawee]

So in our application, with the default parameter (maxNodes = 500000), our tree.json has size 314826788, with 68674 split files.

I tried various maxNodes to reduce the size even further.

• With maxNodes = 100000, tree.json has size 26466963, with 78518 split files.
• With maxNodes = 50000, tree.json has size 12907435, with 40242 split files.
• With maxNodes = 10000, tree.json has size 1990020, with 8511 split files.

But with maxNodes = 5000, I got the "Could not find parent of" error.

[rix0rrr]

So you are trying to minimize the sizes of the tree fragment files, while increasing the count of them?

[rix0rrr]

I see in your numbers that smaller files seems to imply fewer files. That is an odd relation that doesn’t make sense to me 🤔. The relationship should be inverse: count * size ~= constant

[rix0rrr]

Ahh, it does make sense, for the following reason:

We are trying to maximize the size of the "main tree" (to maximize the visibility of constructs for unadapted readers), but that has as a side effect that there will be a lot of subtrees.

The alternative, minimizing the subtree count, is probably going to minimize the size of the main tree as well which will reduce the visibility of legacy readers (so is undesirable).

The only way out that I can see is to "pack" small subtrees together in a single file. This doesn't change the number of "trees", but it changes the number of "files" (and that's ultimately what we are interested in).