From 6c9996318051e04287bbd41540b0194068e16023 Mon Sep 17 00:00:00 2001
From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com>
Date: Tue, 27 Apr 2021 10:52:42 +0000
Subject: [PATCH] chore: patch jsii-rosetta to limit the number of worker
 threads (backport #14389) (#14393)

This is an automatic backport of pull request #14389 done by [Mergify](https://mergify.io).

---


<details>
<summary>Mergify commands and options</summary>

<br />

More conditions and actions can be found in the [documentation](https://docs.mergify.io/).

You can also trigger Mergify actions by commenting on this pull request:

- `@Mergifyio refresh` will re-evaluate the rules
- `@Mergifyio rebase` will rebase this PR on its base branch
- `@Mergifyio update` will merge the base branch into this PR
- `@Mergifyio backport <destination>` will backport this PR on `<destination>` branch

Additionally, on Mergify [dashboard](https://dashboard.mergify.io/) you can:

- look at your merge queues
- generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/
</details>
---
 patches/README.md                 | 14 ++++++++++++++
 patches/jsii-rosetta+1.28.0.patch | 14 ++++++++++++++
 2 files changed, 28 insertions(+)
 create mode 100644 patches/jsii-rosetta+1.28.0.patch

diff --git a/patches/README.md b/patches/README.md
index 46c0f0415dc2f..64be828bc94d0 100644
--- a/patches/README.md
+++ b/patches/README.md
@@ -27,3 +27,17 @@ $ time node_modules/.bin/lerna exec pwd
 lerna success exec Executed command in 219 packages: "pwd"
         1.11 real         0.99 user         0.82 sys
 ```
+
+## jsii-rosetta+1.28.0.patch
+
+jsii-rosetta uses multiple worker threads by default to speed up sample extraction.
+It defaults to spawning workers equal to half the number of cores. On extremely
+powerful build machines (e.g., CodeBuild X2_LARGE compute with 72 vCPUs),
+the high number of workers (36) results in thrash and high memory usage due to
+duplicate loads of source files. This was causing the v2 builds to fail with:
+"FATAL ERROR: NewSpace::Rebalance Allocation failed - JavaScript heap out of memory"
+
+The patch simply limits the top number of worker threads to an arbitrarily-chosen
+maximum limit of 16. We could simply disable the worker threads, but this takes much
+longer to process. With single-threading, rosetta takes ~35 minutes to extract samples
+from the CDK; with 16 workers, it takes ~3.5 minutes.
diff --git a/patches/jsii-rosetta+1.28.0.patch b/patches/jsii-rosetta+1.28.0.patch
new file mode 100644
index 0000000000000..89dc133e267ca
--- /dev/null
+++ b/patches/jsii-rosetta+1.28.0.patch
@@ -0,0 +1,14 @@
+diff --git a/node_modules/jsii-rosetta/lib/commands/extract.js b/node_modules/jsii-rosetta/lib/commands/extract.js
+index e695ea9..539038e 100644
+--- a/node_modules/jsii-rosetta/lib/commands/extract.js
++++ b/node_modules/jsii-rosetta/lib/commands/extract.js
+@@ -104,7 +104,8 @@ exports.singleThreadedTranslateAll = singleThreadedTranslateAll;
+ async function workerBasedTranslateAll(worker, snippets, includeCompilerDiagnostics) {
+     // Use about half the advertised cores because hyperthreading doesn't seem to help that
+     // much (on my machine, using more than half the cores actually makes it slower).
+-    const N = Math.max(1, Math.ceil(os.cpus().length / 2));
++    // Cap to a reasonable top-level limit to prevent thrash on machines with many, many cores.
++    const N = Math.min(16, Math.max(1, Math.ceil(os.cpus().length / 2)));
+     const snippetArr = Array.from(snippets);
+     const groups = util_1.divideEvenly(N, snippetArr);
+     logging.info(`Translating ${snippetArr.length} snippets using ${groups.length} workers`);