Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use GitHub Workflows CI for testing #216

Open
ibnesayeed opened this issue May 16, 2020 · 2 comments · May be fixed by #320
Open

Use GitHub Workflows CI for testing #216

ibnesayeed opened this issue May 16, 2020 · 2 comments · May be fixed by #320

Comments

@ibnesayeed
Copy link

I have created IPFS Setup Action to test IPWB with various combinations of different OSes, Python versions, and IPFS versions. I think this repo can benefit from the newly created GH Action.

@ntninja
Copy link
Contributor

ntninja commented May 19, 2020

I somehow wish you had done that a couple of months ago, now that I figured out how to our entire testing cross-platform, switching is not all that appealing: https://github.com/ipfs-shipyard/py-ipfs-http-client/blob/master/.travis.yml

But really what I care most about: What's the performance your solution? Can it compete with TravisCI premium: https://github.com/ipfs-shipyard/py-ipfs-http-client/runs/663698061 ?

Interested in your experiences with this. 👍

@ibnesayeed
Copy link
Author

ibnesayeed commented May 19, 2020

I somehow wish you had done that a couple of months ago, now that I figured out how to our entire testing cross-platform, switching is not all that appealing: https://github.com/ipfs-shipyard/py-ipfs-http-client/blob/master/.travis.yml

I honestly was thinking about doing it long time ago, but could not spare cycles earlier unfortunately. Your Travis build setup is the reason why I suggested it here when I looked at it the other day.

But really what I care most about: What's the performance your solution? Can it compete with TravisCI premium: https://github.com/ipfs-shipyard/py-ipfs-http-client/runs/663698061 ?

I have not used Travis Premium, so can't comment on that, but it is very comparable to their free plan for OSS. In fact, we are running both Travis CI and GH Workflows in our IPWB repo (and still discussing, should we retire Travis CI?). In one of our recent builds Travis CI took 2m 45s and GH Workflows took 2m 37s. The test matrix runs in parallel, so the time of multiple jobs does not add up linearly.

There are some other factors to consider:

  • The code and tests happen on the same platform
  • No need to pass access token across services or install applications that would have unnecessary access on the repo
  • Steps pipeline support, i.e., output of one step can be fed into the other to run certain steps conditionally without custom state management or scripts

These were CI platform comparisons, but one additional benefit that would come from using the IPFS Setup Action is built-in SemVer resolution (i.e., using version prefixes like >, <, =, and ^ and partial version specification). Which means, if we want to run tests on the latest patch releases of IPFS 0.4.x and 0.5.x, then we don't need to update build config each time new releases are out, while still being able to pin to a specific versions if we want to.

If you want to compare build times and other aspects in this repo, perhaps we can add a workflow file (in addition to existing CI setup) for trial and see how it goes. If Travis CI seems work better then GH Workflow can be abandoned.

@kasteph kasteph linked a pull request May 16, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants