ARROW-14840: [R][CI] test-ubuntu-20.10-docs nightly build failing due to R install issue #11815

thisisnic · 2021-11-30T17:36:58Z

I'm not 100% sure why this fixes this fail, but downgrading the Ubuntu version to 20.04 seems to fix it, and here's my reasoning.

So, apt-key, which is used as part of the R bit of the docs build is now deprecated[1] and Ubuntu 20.10 (Groovy Gorilla) is no longer a supported release as of 22nd July 2021 [2] which can apparently cause issues subsequently trying to do stuff via apt[3].

Reading the R docs for Ubuntu packages[4], it says there that "Package for the current R 4.1. release* are available for most stable Desktop releases of Ubuntu until their official end of life date" which maybe explains why the URL for R packages works fine for focal[5] but results in a 404 error for groovy [6].

The way of getting the signing key for updating packages changed [4] to no longer use apt-key [7] - I have no idea if this had any effect either, but to be on the safe side I have updated that too.

[1] https://manpages.debian.org/testing/apt/apt-key.8.en.html
[2] https://wiki.ubuntu.com/Releases
[3] https://news.itsfoss.com/ubuntu-20-10-end-of-life/
[4] https://cran.r-project.org/bin/linux/ubuntu/
[5] https://cran.r-project.org/bin/linux/ubuntu/focal-cran40/
[6] https://cran.r-project.org/bin/linux/ubuntu/groovy-cran40/
[7] http://web.archive.org/web/20210618164400/https://cloud.r-project.org/bin/linux/ubuntu/

github-actions · 2021-11-30T17:37:21Z

https://issues.apache.org/jira/browse/ARROW-14840

thisisnic · 2021-11-30T17:37:38Z

@github-actions crossbow submit test-ubuntu-21.04-docs

github-actions · 2021-11-30T17:38:40Z

Revision: 8804c56

Submitted crossbow builds: ursacomputing/crossbow @ actions-1218

Task	Status
test-ubuntu-21.04-docs

thisisnic · 2021-11-30T17:47:17Z

@github-actions crossbow submit test-ubuntu-20.04-docs

github-actions · 2021-11-30T17:48:23Z

Revision: 518cb8a

Submitted crossbow builds: ursacomputing/crossbow @ actions-1219

Task	Status
test-ubuntu-20.04-docs

thisisnic · 2021-11-30T17:48:55Z

Alternatively, we could move to 21.04 after #11786 has merged

jonkeane · 2021-11-30T19:40:40Z

Whoa fantastic debugging and puzzle-piece-putting together here.

So, apt-key, which is used as part of the R bit of the docs build is now deprecated[1] and Ubuntu 20.10 (Groovy Gorilla) is no longer a supported release as of 22nd July 2021 [2] which can apparently cause issues subsequently trying to do stuff via apt[3].

This now makes so much sense now. I'm fine with waiting for the Clang fix and then bumping to 21.04 (or 21.10??). Also, we could add CLANG_TOOLS: 9 or CLANG_TOOLS: 11 to the task and bump now like we do for a few others

FWIW, https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-1219-azure-test-ubuntu-20.04-docs looks unrelated to the R package stuff (though I suspect the R package issues masked when this was introduced):

+ sphinx-build -b html -j 2 /arrow/docs/source /build/docs
Running Sphinx v3.5.4

Extension error:
Could not import extension sphinx_tabs.tabs (exception: No module named 'sphinx_tabs')
2
Error: `docker-compose --file /home/vsts/work/1/s/arrow/docker-compose.yml run --rm -e SETUPTOOLS_SCM_PRETEND_VERSION=7.0.0.dev219 ubuntu-docs` exited with a non-zero exit code 2, see the process log above.

thisisnic · 2021-11-30T20:38:57Z

@github-actions crossbow submit test-ubuntu-20.04-docs

github-actions · 2021-11-30T20:40:02Z

Revision: 9ab33be

Submitted crossbow builds: ursacomputing/crossbow @ actions-1222

Task	Status
test-ubuntu-20.04-docs

kou · 2021-11-30T21:02:15Z

dev/tasks/tasks.yml

    ci: azure
    template: docker-tests/azure.linux.yml
    params:
      env:
-        UBUNTU: "20.10"
+        UBUNTU: "20.04"


test-ubuntu-default-docs already uses Ubuntu 20.04.

Good point! Given that this additional job was introduced in #10430 and things have moved on since then, I've removed the extra job. Or should I add it back in and pin it to a different version? It'd be good to know your thoughts too @jonkeane as you were the author of the PR I mentioned above and I don't fully understand everything that's going on there.

IIRC, I introduced test-ubuntu-default-docs because we started being unable to build docs on 20.04 (and that inability wasn't noticed cause we were only testing 20.10 before this)

It looks like the 20.10 build was changed from a much older one in #9933.

I'm not sure what the purpose of having a default build and then a separate build on a non-default version is. The one reason I can think of is that we want the non-default version to be a test of a new(ish) ubuntu version, to check for issues there. If that is the case (and we want to maintain that), we probably should build on 21.04 or 21.10. But I'm also totally fine removing the non-default docs build if it's more hassle than is helpful to maintain it.

OK, sounds like it made sense at the time but no longer serves a specific purpose, so unless there's any other reason to keep it in, let's leave it out for now (and add it back in if necessary later).

We chose Ubuntu 20.10 to reduce build time by using system gRPC: #9933 (review)

I'm OK with removing the docs job with non default UBUNTU version.

Could you also remove UBUNTU=20.10 from dev/release/post-09-docs.sh and ci/docker/ubuntu-20.10-cpp.dockerfile file?

Aha, thanks for clarifying that @kou! Just to check my understanding, did you mean remove the reference to 20.10 in dev/release/post-09-docs.sh but entirely remove the file ci/docker/ubuntu-20.10-cpp.dockerfile as no other tasks depend on this now?

ci/docker/linux-apt-docs.dockerfile

Co-authored-by: Sutou Kouhei <kou@cozmixng.org>

…0.10 in release notes

…4840

jonkeane

Looks good to me

ursabot · 2021-12-01T17:11:11Z

Benchmark runs are scheduled for baseline = 5227f24 and contender = 736a69e. 736a69e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.18% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

… to R install issue I'm not 100% sure why this fixes this fail, but downgrading the Ubuntu version to 20.04 seems to fix it, and here's my reasoning. So, apt-key, which is used as part of the R bit of the docs build is now deprecated[1] and Ubuntu 20.10 (Groovy Gorilla) is no longer a supported release as of 22nd July 2021 [2] which can apparently cause issues subsequently trying to do stuff via apt[3]. Reading the R docs for Ubuntu packages[4], it says there that "Package for the current R 4.1. release* are available for most stable Desktop releases of Ubuntu until their official end of life date" which maybe explains why the URL for R packages works fine for focal[5] but results in a 404 error for groovy [6]. The way of getting the signing key for updating packages changed [4] to no longer use apt-key [7] - I have no idea if this had any effect either, but to be on the safe side I have updated that too. [1] https://manpages.debian.org/testing/apt/apt-key.8.en.html [2] https://wiki.ubuntu.com/Releases [3] https://news.itsfoss.com/ubuntu-20-10-end-of-life/ [4] https://cran.r-project.org/bin/linux/ubuntu/ [5] https://cran.r-project.org/bin/linux/ubuntu/focal-cran40/ [6] https://cran.r-project.org/bin/linux/ubuntu/groovy-cran40/ [7] http://web.archive.org/web/20210618164400/https://cloud.r-project.org/bin/linux/ubuntu/ Closes apache#11815 from thisisnic/ARROW-14840 Authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Jonathan Keane <jkeane@gmail.com>

Update docs build to run in 21.04 and update to no longer use apt-key

8804c56

Change to 20.04 instead as having clang issues

518cb8a

pitrou requested a review from kszucs November 30, 2021 17:58

Add sphinx-tabs to doc dockerfile

9ab33be

kou reviewed Nov 30, 2021

View reviewed changes

thisisnic added 3 commits November 30, 2021 21:11

Alphabetise ordering

1cfdfcf

Remove redundant job

e734997

alphabetise other list

53e687c

kou reviewed Nov 30, 2021

View reviewed changes

ci/docker/linux-apt-docs.dockerfile Outdated Show resolved Hide resolved

thisisnic and others added 3 commits November 30, 2021 21:31

Update ci/docker/linux-apt-docs.dockerfile

4092a64

Co-authored-by: Sutou Kouhei <kou@cozmixng.org>

Remove redundant dockerfile and note to build ubuntu-docs on UBUNTU 2…

69c79df

…0.10 in release notes

Merge branch 'ARROW-14840' of github.com:thisisnic/arrow into ARROW-1…

a2134ee

…4840

jonkeane approved these changes Dec 1, 2021

View reviewed changes

jonkeane closed this in 736a69e Dec 1, 2021

kou mentioned this pull request Dec 1, 2021

[Website] Update documentations for 6.0.1 apache/arrow-site#167

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-14840: [R][CI] test-ubuntu-20.10-docs nightly build failing due to R install issue #11815

ARROW-14840: [R][CI] test-ubuntu-20.10-docs nightly build failing due to R install issue #11815

thisisnic commented Nov 30, 2021 •

edited

Loading

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

jonkeane commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

kou Nov 30, 2021

thisisnic Nov 30, 2021

jonkeane Nov 30, 2021

thisisnic Nov 30, 2021

kou Nov 30, 2021

thisisnic Dec 1, 2021

kou Dec 1, 2021

jonkeane left a comment

ursabot commented Dec 1, 2021 •

edited

Loading

ARROW-14840: [R][CI] test-ubuntu-20.10-docs nightly build failing due to R install issue #11815

ARROW-14840: [R][CI] test-ubuntu-20.10-docs nightly build failing due to R install issue #11815

Conversation

thisisnic commented Nov 30, 2021 • edited Loading

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

thisisnic commented Nov 30, 2021

jonkeane commented Nov 30, 2021

thisisnic commented Nov 30, 2021

github-actions bot commented Nov 30, 2021

kou Nov 30, 2021

Choose a reason for hiding this comment

thisisnic Nov 30, 2021

Choose a reason for hiding this comment

jonkeane Nov 30, 2021

Choose a reason for hiding this comment

thisisnic Nov 30, 2021

Choose a reason for hiding this comment

kou Nov 30, 2021

Choose a reason for hiding this comment

thisisnic Dec 1, 2021

Choose a reason for hiding this comment

kou Dec 1, 2021

Choose a reason for hiding this comment

jonkeane left a comment

Choose a reason for hiding this comment

ursabot commented Dec 1, 2021 • edited Loading

thisisnic commented Nov 30, 2021 •

edited

Loading

ursabot commented Dec 1, 2021 •

edited

Loading