Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default installed packages in workers #1761

Open
brollb opened this issue Jul 13, 2020 · 1 comment
Open

Default installed packages in workers #1761

brollb opened this issue Jul 13, 2020 · 1 comment

Comments

@brollb
Copy link
Contributor

brollb commented Jul 13, 2020

Although operations can define their own custom dependencies, it might be nice to include more default packages in the operations. This is especially motivating since conda is somewhat slow so installation of these common data science packages may be a bit annoying.

@umesh-timalsina and I have been talking about this somewhat recently and @dustinjoe also brought it up recently here so it is probably about time it got an official issue :)

If we decide to go this route, some candidates for being included in the base environment:

  • scikit learn
  • numpy
  • pandas
@umesh-timalsina
Copy link
Contributor

umesh-timalsina commented Jul 13, 2020

This would be nice. One of the issues I faced while working on #1747 is that the more packages there are in your base environments, the more it takes for conda to export it. It will not be an issue for local and gme computes, but can cause lag problems in for example -> ephemeral computes like sciserver-compute, where base conda environments do not exist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants