Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do some performance enhancements for testing #1059

Closed
mlissner opened this issue Nov 18, 2019 · 3 comments
Closed

Do some performance enhancements for testing #1059

mlissner opened this issue Nov 18, 2019 · 3 comments

Comments

@mlissner
Copy link
Member

Tests take about 13 minutes right now. About eight of that is tests and the rest is getting code, docker images, etc. The fixes are clear, but they won't necessarily be easy:

  • We've got about a decade of Git commits as of now and our repository has grown large. Remove old files from repository #796 is to remove files from our repository. We should do that too, but more important is to nuke anything more than a few years old. I have no idea how to do this, but it'd be great if it weren't too difficult.

  • Running tests in parallel would speed them up. The tricky part is that if two tests use the same data in the database, we're in trouble. Maybe we can split Selenium tests up from the rest? Django has some good tools for this, as does Github Actions.

  • Shrink our docker images? I think we currently have different images for cl-celery and cl-django, but they're basically identical if you look at their Dockerfile files. Could one inherit from the other, say, for a big boost? Secondly, can we use the alpine images to make slim images?

  • Our selenium tests are stupidly slow. I literally put a five minute timeout on them. What is wrong with these that makes them so slow?

@mlissner
Copy link
Member Author

Another thing: Squashing migrations should help. I'm going to give that a try as soon as the PR backlog is flushed.

@divergentdave
Copy link
Contributor

Regarding the first bullet point, doing a shallow clone of the repository would allow you to speed up CI without having to rewrite git history. Travis CI builds do git clone --depth 50 by default. Once actions/checkout@v2 is released, (actions/checkout#70) this will be easy to configure.

@mlissner
Copy link
Member Author

It's funny, it didn't occur to me that was possible when I wrote this up, and then a couple days ago I realized it must be a thing and that it'd be much easier than deleting old commits. Thanks for digging it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants