perf: non-blocking write of optimized dep files #12603
Conversation
|
|
| if (isDepsOptimizerEnabled(config, false)) { | ||
| depsOptimizerReady = initDepsOptimizer(config) | ||
| // start optimizer in the background, we still need to await the setup | ||
| await initDepsOptimizer(config) |
There was a problem hiding this comment.
@bluwy we need to await the optimizer init here. The setup is very lean and it starts the processing in the background so this shouldn't take more than a few ms. The issue was evident when I tried to implement the wait 500ms when there is a _writing file.
|
We'll need to keep an eye on this one https://github.com/vitejs/vite/actions/runs/4526804238/jobs/7972195375#step:13:94, I don't know if this appears in this PR. It could be related to previous ones. |
| @@ -67,7 +70,7 @@ export function optimizedDepsPlugin(config: ResolvedConfig): Plugin { | |||
| // load hooks to avoid race conditions, once processing is resolved, | |||
| // we are sure that the file has been properly save to disk | |||
| try { | |||
| return await fs.readFile(file, 'utf-8') | |||
| return loadOptimizedDep(file, depsOptimizer) | |||
There was a problem hiding this comment.
Could we stream .contents, or use them here?
There was a problem hiding this comment.
I think we need to make it a string here. For sourcemaps, since they're handled in the transform middleware, we could, but I tried that and it doesn't work since etags are generated via the Buffer instance, and Buffer.isBuffer(new Uint8Array(2)) is false.
|
Thanks for the changes @bluwy! Let's merge this now so we also fix the optimizer setup before the server. |
Description
Keep the esbuild output files in memory so we can avoid waiting for the write of all optimized deps and then read from disk.
Implements a
_writinglock of the deps cache that is checked when reading the cached deps to avoid issues if the user stopped the process in the middle of writing the optimized deps. @dominikg thinks this isn't enough, but given that two processes shouldn't be working with the same cache dir in the same root I think this is enough.What is the purpose of this pull request?