Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add helper for walking a tree concurrently and deterministic #3619

Merged
merged 7 commits into from
Feb 3, 2023

Conversation

sokra
Copy link
Member

@sokra sokra commented Feb 3, 2023

Similar to our try_join helper this adds a try_flat_map_recursive_join helper, which allows to async expand a tree structure into all nodes. It will call the async mapper function concurrently to allow parallelism. It will handle circular and duplicate references and return all nodes in a determinstic way (breath-first).

fixes WEB-558

@vercel
Copy link

vercel bot commented Feb 3, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
examples-cra-web 🔄 Building (Inspect) Feb 3, 2023 at 5:44PM (UTC)
turbo-site 🔄 Building (Inspect) Visit Preview Feb 3, 2023 at 5:44PM (UTC)
8 Ignored Deployments
Name Status Preview Comments Updated
examples-basic-web ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-designsystem-docs ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-kitchensink-blog ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-native-web ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-nonmonorepo ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-svelte-web ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-tailwind-web ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)
examples-vite-web ⬜️ Ignored (Inspect) Feb 3, 2023 at 5:44PM (UTC)

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

Benchmark for 0483b8e

Test Base PR % Significant %
bench_hydration/Turbopack RCC/1000 modules 4446.30ms ± 28.41ms 3835.40ms ± 15.22ms -13.74% -11.93%
bench_hydration/Turbopack RSC/1000 modules 4011.55ms ± 13.95ms 3410.40ms ± 13.75ms -14.99% -13.70%
bench_hydration/Turbopack SSR/1000 modules 3734.00ms ± 25.32ms 3332.96ms ± 13.74ms -10.74% -8.77%
bench_startup/Turbopack CSR/1000 modules 2677.25ms ± 13.00ms 2557.63ms ± 16.64ms -4.47% -2.28%
bench_startup/Turbopack RCC/1000 modules 2864.74ms ± 6.34ms 2316.96ms ± 7.25ms -19.12% -18.25%
bench_startup/Turbopack RSC/1000 modules 2800.14ms ± 12.47ms 2227.82ms ± 5.47ms -20.44% -19.33%
bench_startup/Turbopack SSR/1000 modules 2291.84ms ± 5.30ms 1920.50ms ± 3.05ms -16.20% -15.55%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 9317.46µs ± 74.28µs 9403.38µs ± 49.25µs +0.92%
bench_hmr_to_commit/Turbopack RCC/1000 modules 9524.05µs ± 72.36µs 9631.77µs ± 80.58µs +1.13%
bench_hmr_to_commit/Turbopack RSC/1000 modules 493.05ms ± 2.45ms 485.98ms ± 2.14ms -1.43%
bench_hmr_to_commit/Turbopack SSR/1000 modules 9591.21µs ± 76.43µs 9509.28µs ± 79.50µs -0.85%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8353.07µs ± 68.64µs 8387.68µs ± 75.36µs +0.41%
bench_hmr_to_eval/Turbopack RCC/1000 modules 8727.64µs ± 94.34µs 8631.99µs ± 66.73µs -1.10%
bench_hmr_to_eval/Turbopack SSR/1000 modules 8518.13µs ± 80.97µs 8458.45µs ± 80.28µs -0.70%
bench_hydration/Turbopack RCC/1000 modules 4446.30ms ± 28.41ms 3835.40ms ± 15.22ms -13.74% -11.93%
bench_hydration/Turbopack RSC/1000 modules 4011.55ms ± 13.95ms 3410.40ms ± 13.75ms -14.99% -13.70%
bench_hydration/Turbopack SSR/1000 modules 3734.00ms ± 25.32ms 3332.96ms ± 13.74ms -10.74% -8.77%
bench_startup/Turbopack CSR/1000 modules 2677.25ms ± 13.00ms 2557.63ms ± 16.64ms -4.47% -2.28%
bench_startup/Turbopack RCC/1000 modules 2864.74ms ± 6.34ms 2316.96ms ± 7.25ms -19.12% -18.25%
bench_startup/Turbopack RSC/1000 modules 2800.14ms ± 12.47ms 2227.82ms ± 5.47ms -20.44% -19.33%
bench_startup/Turbopack SSR/1000 modules 2291.84ms ± 5.30ms 1920.50ms ± 3.05ms -16.20% -15.55%

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

⚠️ CI failed ⚠️

The following steps have failed in CI:

  • Turbopack Rust tests

See workflow summary for details

crates/turbo-tasks/src/join_iter_ext.rs Show resolved Hide resolved
crates/turbopack-core/src/chunk/mod.rs Outdated Show resolved Hide resolved
@sokra sokra force-pushed the sokra/improve-tree-walking branch from 57afe52 to ba92446 Compare February 3, 2023 16:37
@sokra sokra marked this pull request as ready for review February 3, 2023 16:48
@sokra sokra requested review from a team as code owners February 3, 2023 16:48
@sokra sokra force-pushed the sokra/improve-tree-walking branch from ba92446 to 9c55551 Compare February 3, 2023 17:43
@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2023

Benchmark for 231bde2

Test Base PR % Significant %
bench_hydration/Turbopack RCC/1000 modules 5112.25ms ± 18.80ms 4296.30ms ± 7.06ms -15.96% -15.06%
bench_hydration/Turbopack RSC/1000 modules 4565.68ms ± 13.08ms 3846.29ms ± 14.60ms -15.76% -14.63%
bench_hydration/Turbopack SSR/1000 modules 4118.45ms ± 15.65ms 3638.44ms ± 29.16ms -11.66% -9.55%
bench_startup/Turbopack CSR/1000 modules 3028.68ms ± 8.97ms 2823.87ms ± 18.79ms -6.76% -4.96%
bench_startup/Turbopack RCC/1000 modules 3249.75ms ± 7.22ms 2615.60ms ± 11.40ms -19.51% -18.45%
bench_startup/Turbopack RSC/1000 modules 3153.10ms ± 9.93ms 2485.56ms ± 16.60ms -21.17% -19.61%
bench_startup/Turbopack SSR/1000 modules 2481.66ms ± 7.15ms 2069.09ms ± 4.66ms -16.62% -15.76%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 10.01ms ± 0.12ms 9895.09µs ± 51.28µs -1.11%
bench_hmr_to_commit/Turbopack RCC/1000 modules 9543.81µs ± 116.48µs 9962.33µs ± 199.45µs +4.39%
bench_hmr_to_commit/Turbopack RSC/1000 modules 514.60ms ± 1.54ms 514.24ms ± 1.58ms -0.07%
bench_hmr_to_commit/Turbopack SSR/1000 modules 10.39ms ± 0.08ms 10.25ms ± 0.10ms -1.31%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8696.55µs ± 92.45µs 8471.56µs ± 37.07µs -2.59%
bench_hmr_to_eval/Turbopack RCC/1000 modules 9315.85µs ± 83.70µs 9305.57µs ± 76.92µs -0.11%
bench_hmr_to_eval/Turbopack SSR/1000 modules 9159.66µs ± 89.37µs 8961.36µs ± 100.80µs -2.16%
bench_hydration/Turbopack RCC/1000 modules 5112.25ms ± 18.80ms 4296.30ms ± 7.06ms -15.96% -15.06%
bench_hydration/Turbopack RSC/1000 modules 4565.68ms ± 13.08ms 3846.29ms ± 14.60ms -15.76% -14.63%
bench_hydration/Turbopack SSR/1000 modules 4118.45ms ± 15.65ms 3638.44ms ± 29.16ms -11.66% -9.55%
bench_startup/Turbopack CSR/1000 modules 3028.68ms ± 8.97ms 2823.87ms ± 18.79ms -6.76% -4.96%
bench_startup/Turbopack RCC/1000 modules 3249.75ms ± 7.22ms 2615.60ms ± 11.40ms -19.51% -18.45%
bench_startup/Turbopack RSC/1000 modules 3153.10ms ± 9.93ms 2485.56ms ± 16.60ms -21.17% -19.61%
bench_startup/Turbopack SSR/1000 modules 2481.66ms ± 7.15ms 2069.09ms ± 4.66ms -16.62% -15.76%

@sokra sokra merged commit 2626223 into main Feb 3, 2023
@sokra sokra deleted the sokra/improve-tree-walking branch February 3, 2023 19:31
mehulkar pushed a commit that referenced this pull request Feb 3, 2023
Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
jridgewell pushed a commit to vercel/next.js that referenced this pull request Mar 10, 2023
…turborepo#3619)

Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
sokra added a commit to vercel/next.js that referenced this pull request Mar 13, 2023
…turborepo#3619)

Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 25, 2024
…turborepo#3619)

Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 29, 2024
…turborepo#3619)

Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Aug 1, 2024
…turborepo#3619)

Similar to our `try_join` helper this adds a
`try_flat_map_recursive_join` helper, which allows to async expand a
tree structure into all nodes. It will call the async mapper function
concurrently to allow parallelism. It will handle circular and duplicate
references and return all nodes in a determinstic way (breath-first).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants