-
Notifications
You must be signed in to change notification settings - Fork 435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use appraisal
gem in CI
#1308
Use appraisal
gem in CI
#1308
Conversation
Looking good, thanks for wrapping this up! 🎉 While re-reading #1230, I noticed there was one last thing to decide. Do we want to:
Happy to hear what you think @joelhawksley @boardfish @BlakeWilliams |
I'm leaning towards that first option. I think we should prioritize ensuring that running the tests works the same way in dev as it does in CI. It may be a good idea in the long run to mirror @excid3's fork to @github so we can ensure that version of the code remains publicly available until @excid3's PR is merged. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice this is great! I'm a big fan of appraisal :)
.github/workflows/ci.yml
Outdated
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
rails_version: ['5.2.6', '6.0.4.4', '6.1.4.4', '7.0.1', 'main'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a heads-up a repo admin will need to adjust the "required checks" after merging in order to accommodate the different labels here.
@joelhawksley would you mind giving us your feedback on this? (See my latest comment above). I think we should try to merge this soon. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Spone IMO we need to be cautious of depending on a long-running upstream PR on Appraisal.
Let's take another approach, or work to get the upstream PR merged.
FWIW, I've just gone through the fun process of trying to get Lookbook CI working with Appraisal - I took a slightly different approach (and with a smaller matrix of ruby/rails versions for now) but I finally got it working. Just in case it's useful to have an example which uses the matrix approach but doesn't require the Appraisal fork: https://github.com/allmarkedup/lookbook/blob/main/.github/workflows/ci.yml |
There is a new PR on Appraisal that could fix the issue and allow us to move forward with this approach. If it does not, I'll take some time to mirror @allmarkedup's method. |
Appraisal maintainer here! 👋 I've just merged that PR mentioned above (thoughtbot/appraisal#202), which I'm hoping can help you out here. In a few days I'll cut a new release. We'd been a bit stuck with a good solution to that problem, but let us know over there if you're having problems. |
Awesome @nickcharlton! I'll try it out next week. Thanks for the heads up! |
Gemfile
Outdated
@@ -17,6 +17,10 @@ group :test do | |||
gem "selenium-webdriver", "~> 4" | |||
end | |||
|
|||
group :development, :test do | |||
gem "appraisal", github: "thoughtbot/appraisal" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm targeting main
for now, but we'll be able to use the next version when it's released.
I thought it was time to get this in shape for v3.0.0 :) Do not forget this after merging: #1308 (comment) Thanks a lot @boardfish for your help on this! |
Do you want to review this again @joelhawksley @camertron? Else I think I'll merge this in the coming days. |
@Spone I'd prefer we wait for the Appraisal release instead of depending on a branch. |
@boardfish thoughts on testing this against thoughtbot/appraisal#204 (comment)? Perhaps we don't need to wait for 3.0? |
Yeah, that sounds good! |
@boardfish cool, would you be up for bringing this PR up to date? |
Sure! I can have a crack at that sometime in the next week. |
d1d26ad
to
edb5fb7
Compare
edb5fb7
to
a9b4c22
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Can you merge main?
a9b4c22
to
fd21097
Compare
Ugh sorry @boardfish, could you fix these merge conflicts? I'm not sure how to resolve the difference in the Gemfile 😓 |
7146d23
to
6891bba
Compare
# :nocov: | ||
raise "`render_preview` expected a described_class, but it is nil." if described_class.nil? | ||
|
||
"#{described_class}Preview" | ||
# :nocov: | ||
else | ||
self.class.name.gsub("Test", "Preview") | ||
end | ||
result = result.constantize | ||
rescue NameError | ||
raise NameError, "`render_preview` expected to find #{result}, but it does not exist." | ||
end | ||
# :nocov: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why were these changes necessary? I'd really prefer to avoid adding these comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try dropping them, but runs were failing due to coverage errors. I think this was the cause, though I'm not sure if anything in this PR in particular would've triggered it.
- uses: actions/cache@v3 | ||
with: | ||
path: vendor/bundle | ||
key: gems-build-rails-${{ matrix.rails_version }}-ruby-${{ matrix.ruby_version }}-${{ hashFiles('**/Gemfile.lock') }} | ||
bundler-cache: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😎
07fa887
to
beb9857
Compare
@joelhawksley Looks like we do still get coverage-related failures without those nocov comments. Not sure why, though – maybe there's a discrepancy in the running environment between the runs on I feel like we should merge this and follow up to resolve that. What do you reckon? |
@boardfish I'm happy to merge and revisit given the complexity of this PR. |
Awesome. I'll get rid of those merge conflicts and merge this in myself later, then 👍 |
beb9857
to
56cf867
Compare
Appraisal allows us to test with different versions of gems. In addition to GitHub Actions' matrix functionality, we can use this to test against different combinations of Ruby and Rails versions. As part of this, we've also made the following changes to our CI pipeline: Rename Appraisal rails-head to rails-main Loosen version constraints to minor versions Use setup-ruby's caching mechanism Means we don't need to configure vendor/bundle as the path here. We should probably do this for other workflows too. Don't trigger CI twice on PRs Co-authored-by: Simon Fish <si@mon.fish>
56cf867
to
d9c202c
Compare
Supersedes #1230
Uses
appraisal ~> 2.5
to run against an array of Rails minor versions from6.1
onwards. Also moves us away from manually caching gem dependencies, as that can now be handled by thesetup-ruby
action.