Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(core): slow synthesis after update to 2.71.0 #25286

Closed
deftomat opened this issue Apr 25, 2023 · 3 comments · Fixed by #25392
Closed

(core): slow synthesis after update to 2.71.0 #25286

deftomat opened this issue Apr 25, 2023 · 3 comments · Fixed by #25392
Labels
@aws-cdk/core Related to core CDK functionality bug This issue is a bug. effort/medium Medium work item – several days of effort p1

Comments

@deftomat
Copy link

deftomat commented Apr 25, 2023 •
edited
Loading

Describe the bug

Looks like large cdk.out directory (>10GB) is causing slow synthesis due to file/directory hash calculation introduced in
#23951.

I was able to revert synthesis times back to normal by just returning static string from computeChecksumOfFolder.

Looks like calculating hash of 10GB of cdk.out content can easily introduce a 10~15 seconds delays.
This is especially an issue as there is no automated cleaning process for this directory.

I'm not sure if reading GBs of data each time we are synthesising app is a good idea, especially when a lot of people probably not using #23951 feature anyway.

Expected Behavior

Synthesis times should not by affected by stale data stored in cdk.out directory.

Current Behavior

The larger the cdk.out directory, the slower the synthesis is.

Reproduction Steps

  • Install aws-cdk 2.71.0
  • Add a huge amount of data into cdk.out
  • Lost a 15 seconds of your life by waiting for synthesis 🙂

Possible Solution

  • Consider to calculate hash from file's lastModified time instead from the file's content. Looks like that should be enough for the given functionality.
  • Consider to use faster hashing alg
  • Do not calculate hash when there is no validation plugin

Additional Information/Context

No response

CDK CLI Version

2.74.0

Framework Version

No response

Node.js Version

18.16.0

OS

macOS 13.3.1

Language

Typescript

Language Version

5.0.4

Other information

No response
@deftomat deftomat added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 25, 2023
@github-actions github-actions bot added the @aws-cdk/core Related to core CDK functionality label Apr 25, 2023
@deftomat
Copy link
Author

deftomat commented Apr 25, 2023 •
edited
Loading

@otaviomacedo ^^^

@pahud pahud added p1 effort/medium Medium work item – several days of effort and removed needs-triage This issue or PR still needs to be triaged. labels Apr 25, 2023
@pahud
Copy link
Contributor

pahud commented Apr 25, 2023

Thank you for your report.

@otaviomacedo otaviomacedo mentioned this issue May 2, 2023
Merged
@mergify mergify bot closed this as completed in #25392 May 2, 2023
mergify bot pushed a commit that referenced this issue May 2, 2023
@otaviomacedo
fix(core): output folder checksum is computed unnecessarily (#25392)
f2294ba 
The output folder checksum is being at least once in every synthesis. This is only necessary if a validation plugin was registered. Also reusing the `FileSystem.fingerprint()` function while I'm at it.

Fixes #25286

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

github-actions bot commented May 2, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/core Related to core CDK functionality bug This issue is a bug. effort/medium Medium work item – several days of effort p1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(core): output folder checksum is computed unnecessarily aws/aws-cdk
2 participants
@pahud @deftomat