Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply ZIO SBT Ecosystem and CI Plugins #1096

Open
wants to merge 75 commits into
base: series/2.x
Choose a base branch
from

Conversation

khajavi
Copy link
Member

@khajavi khajavi commented Mar 3, 2023

No description provided.

@sideeffffect
Copy link
Member

sideeffffect commented Mar 5, 2023

I'm all for improving our CI/sbt story (which I know currently is less than ideal)... But wow, the number of GitHub Actions jobs is huge! I'm worried that it wouldn't be reasonable to have such high number, each PR would be very expensive and take a long time to complete.
Perhaps there could be a way to reduce the number. Maybe this can serve as some sort of inspiration https://en.wikipedia.org/wiki/Latin_hypercube_sampling

Or keep the original idea where JVM 17, Scala 2.13 and JVM platform are the default and we diverge from it only in one axis.

@khajavi
Copy link
Member Author

khajavi commented Mar 6, 2023

@sideeffffect
With zio-sbt, we can change the granularity of tests. Currently, I need more granularity to find errors. Then I will make tests sequential.

@khajavi khajavi marked this pull request as ready for review March 6, 2023 11:28
@khajavi khajavi requested a review from a team as a code owner March 6, 2023 11:28
@khajavi khajavi marked this pull request as ready for review March 14, 2023 03:37
@khajavi
Copy link
Member Author

khajavi commented Mar 14, 2023

Okay. The other thing we should be doing here is making sure we are running each test file and each project sequentially. We already run tests in parallel so there is no point in also running different test files or projects in parallel and it can create additional memory pressure that leads to test failures or decreased test performance. In this project I added a couple of command aliases to do that.

I added an option to limit the parallel task on the matrix strategy. Also, grouped tests together based on scala and java versions, to make them more sequential.

@adamgfraser
Copy link
Contributor

@khajavi So it seems like this has resulted in a dramatic increase in CI times. It is taking around 20 minutes for these tests now versus it previously took around 4 minutes after we merged my PR upgrading the ZIO version. Can we please maintain the current execution semantics?

@khajavi
Copy link
Member Author

khajavi commented Mar 14, 2023

@adamgfraser Okay, I will try to reduce the duration of the CI process. If this does not lead to any improvement, we need to consider discontinuing automatic test generation.

@khajavi khajavi marked this pull request as draft March 14, 2023 07:42
@khajavi
Copy link
Member Author

khajavi commented Mar 14, 2023

I'll update this after each try:

  • After setting to only test on Java 17=> decreased from 1h 28m to 1h 1m
  • After setting max metaspace size to 4G => decreased to 1h and 4m

@khajavi khajavi force-pushed the apply-zio-ecosystem-plugin branch 2 times, most recently from e30a15f to e7409ba Compare March 14, 2023 14:10
@khajavi khajavi marked this pull request as ready for review March 25, 2023 08:16
@khajavi
Copy link
Member Author

khajavi commented Mar 25, 2023

@adamgfraser To accomplish this task, I utilized the existing semantic, as you said, and attempted to migrate it to the zio-sbt plugin. This involved applying mostly the same workflows used in the original code to the version.

By doing so, I expect to have the same level of quality as the current workflow execution.

@guizmaii
Copy link
Member

@khajavi, Could you update this PR, maybe, please? I'd be interested in merging these changes 🙂

@khajavi
Copy link
Member Author

khajavi commented Apr 15, 2024

@guizmaii
I'm currently engaged in some other things, but I might revisit this later.
Also, I think this PR is open to any collaboration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants