Skip to content

[SPARK-46626][DOCS] Bump jekyll to 4.3.3 to enable support for Ruby 3.3.0 #44628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nchammas wants to merge 3 commits into from
Closed

[SPARK-46626][DOCS] Bump jekyll to 4.3.3 to enable support for Ruby 3.3.0 #44628

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9b056ea
update download links; don't use sudo
nchammas Jan 8, 2024
0fdc1c2
loosen jekyll requirement and update to 4.3.3
nchammas Jan 8, 2024
d863980
provide some guidance in Gemfile
nchammas Jan 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,12 @@

source "https://rubygems.org"

# Keep these specifications as flexible as possible and leave it to Bundler
# to pin versions in the lock file.
# To update the lock file, run `bundle update`.
# Version constraint reference: https://guides.rubygems.org/patterns/#declaring-dependencies
gem "ffi", "1.15.5"
gem "jekyll", "4.3.2"
gem "jekyll", "~> 4.3"
Copy link
Contributor Author

@nchammas nchammas Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to migrate all of the dependency specifications here to use the pessimistic operator, but I leave that to a future PR.

gem "rouge", "3.26.0"
gem "jekyll-redirect-from", "0.16.0"
gem "webrick", "1.8.1"
18 changes: 9 additions & 9 deletions docs/Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GEM
remote: https://rubygems.org/
specs:
addressable (2.8.5)
addressable (2.8.6)
public_suffix (>= 2.0.2, < 6.0)
colorator (1.1.0)
concurrent-ruby (1.2.2)
Expand All @@ -11,11 +11,11 @@ GEM
eventmachine (1.2.7)
ffi (1.15.5)
forwardable-extended (2.6.0)
google-protobuf (3.24.2)
google-protobuf (3.25.1)
http_parser.rb (0.8.0)
i18n (1.14.1)
concurrent-ruby (~> 1.0)
jekyll (4.3.2)
jekyll (4.3.3)
addressable (~> 2.4)
colorator (~> 1.0)
em-websocket (~> 0.5)
Expand Down Expand Up @@ -48,28 +48,28 @@ GEM
mercenary (0.4.0)
pathutil (0.16.2)
forwardable-extended (~> 2.6)
public_suffix (5.0.3)
rake (13.0.6)
public_suffix (5.0.4)
rake (13.1.0)
rb-fsevent (0.11.2)
rb-inotify (0.10.1)
ffi (~> 1.0)
rexml (3.2.6)
rouge (3.26.0)
safe_yaml (1.0.5)
sass-embedded (1.63.6)
google-protobuf (~> 3.23)
sass-embedded (1.69.7)
Copy link
Contributor

@cloud-fan cloud-fan May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nchammas , I hit an error when building docs for the 4.0 preview release

Your bundle is locked to sass-embedded (1.69.7) from rubygems repository https://rubygems.org/ or installed locally, but that version can no longer be found in that
source. That means the author of sass-embedded (1.69.7) has removed it. You'll need to update your bundle to a version other than sass-embedded (1.69.7) that hasn't been
removed in order to install.

I don't know the context of picking sass-embedded version here, which version should we use?

cc @dongjoon-hyun

Copy link
Contributor

@cloud-fan cloud-fan May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have doc building job in PR pre-merge tests, I'm wondering how this can be broken. Or is it because the test environment is different from the release environment defined by https://github.com/apache/spark/blob/master/dev/create-release/spark-rm/Dockerfile ?

Copy link
Contributor Author

@nchammas nchammas May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version was selected by Bundler. Try running bundle update to bump the versions in the lock file and try again.

Would you like me to open a PR to bump the Ruby dependencies?

Copy link
Contributor

@cloud-fan cloud-fan May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, we upgraded bundle at 1e3892f#diff-78c95b3441eb17ae275e1651c656a3614738ea96c7a13df92c95b0b987b6c497R77 but not in the docker file. let me try it.

Copy link
Contributor Author

@nchammas nchammas May 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruby 2 is EOL, so we should not be using Ruby 2.7 anywhere. I can also address this if you like, though it should not be a blocker. A simple refresh of the Gem lock file should be enough for now.

Copy link
Contributor

@cloud-fan cloud-fan May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a blocker for producing a 4.0 release...

@nchammas can you help to make necessary updates to the docker file so that it can build docs? Thanks!

Copy link
Contributor Author

@nchammas nchammas May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure thing. Will look now.

I will need your help with testing, however, since non-committers cannot run all the steps of the release image.

Copy link
Contributor

@cloud-fan cloud-fan May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I'll test directly with your PR. Just ping me when it's ready. Thanks!

Copy link
Contributor Author

@nchammas nchammas May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#46512

google-protobuf (~> 3.25)
rake (>= 13.0.0)
terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3)
unicode-display_width (2.4.2)
unicode-display_width (2.5.0)
webrick (1.8.1)

PLATFORMS
ruby

DEPENDENCIES
ffi (= 1.15.5)
jekyll (= 4.3.2)
jekyll (~> 4.3)
jekyll-redirect-from (= 0.16.0)
rouge (= 3.26.0)
webrick (= 1.8.1)
Expand Down
11 changes: 5 additions & 6 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,13 @@ whichever version of Spark you currently have checked out of revision control.

The Spark documentation build uses a number of tools to build HTML docs and API docs in Scala, Java, Python, R, and SQL.

You need to have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and
[Python](https://docs.python.org/2/using/unix.html#getting-and-installing-the-latest-version-of-python)
installed. Make sure the `bundle` command is available, if not install the Gem containing it:
You need to have [Ruby][ruby] and [Python][python] installed. Make sure the `bundle` command is available. If not, install it as follows:

[ruby]: https://www.ruby-lang.org/en/documentation/installation/
[python]: https://www.python.org/downloads/

```sh
$ sudo gem install bundler
$ gem install bundler
```

After this all the required ruby dependencies can be installed from the `docs/` directory via the Bundler:
Expand All @@ -45,8 +46,6 @@ $ cd docs
$ bundle install
```

Note: If you are on a system with both Ruby 1.9 and Ruby 2.0 you may need to replace gem with gem2.0.

To generate the Python or R docs, you'll need to [install Pandoc](https://pandoc.org/installing.html).

### SQL and Python API Documentation (Optional)
Expand Down