Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid loading babel in forked processes. #182

Closed
jamestalmage opened this issue Nov 10, 2015 · 10 comments
Closed

Avoid loading babel in forked processes. #182

jamestalmage opened this issue Nov 10, 2015 · 10 comments
Assignees

Comments

@jamestalmage
Copy link
Contributor

We are paying a hefty penalty to load Babel once per forked process.

I have played around with a way to share one Babel instances across every forked process (though it does not necessarily need to be limited to a single instance).

https://github.com/jamestalmage/__share_transpile_across_process

With some very naive benchmarks I am seeing < 3 seconds using this method compared to > 16 loading up a Babel instance per fork.

@floatdrop
Copy link
Contributor

@jamestalmage thought of this too. We can transpile files with babel in master process and load transpiled code from cache (as soon as I deal with #118).

@sindresorhus
Copy link
Member

That's a cool idea. I like it.

@jamestalmage
Copy link
Contributor Author

@floatdrop
What is your plan for the cache? Write it all to disk before launching forks?

I'm actually firing up an http server and then using child_process.execSync('curl', ...) to fetch the transpiled code.

@jamestalmage
Copy link
Contributor Author

The major problem with my solution is that it requires actually forking an entirely new process for every fetch, which is not free. Forking an execution of curl vs forking another node process is way better, but still not great.

For the initial test file, we could use IPC (maybe even do static analysis of the Babel AST to find additional require calls).

Dynamic requires will be a problem. That is where I think my strategy will help. If somebody does a lot of dynamic calls there is even a chance my strategy may hurt, so we probably need a way for people to turn it off.

@floatdrop
Copy link
Contributor

@jamestalmage yes, but do it step by step (transpile one test and launch it), not all at once.

@jamestalmage
Copy link
Contributor Author

Gotcha.
Seems reasonable.

@jamestalmage
Copy link
Contributor Author

@sindresorhus We need a good real-world library to performance test this against. I know pageres is mentioned in the readme, but given the nature of that project (I am just guessing), that a lot of time is spent on I/O. Is there one with a high test-file count where I/O is minimal (where improvements to the load time of AVA forks would be most noticeable).

@sindresorhus
Copy link
Member

@jamestalmage https://github.com/sindresorhus/got has a lot of tests and uses a local server for most, so the tests are pretty fast.

@vadimdemedes
Copy link
Contributor

Really good idea! Well done @jamestalmage

@jamestalmage
Copy link
Contributor Author

This was fixed in ~0.10.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants