ARROW-12262: [Doc] Enable S3 and Flight in docs build

[pitrou]

Also use Ubuntu 20.04 by default, instead of 18.04.

[pitrou]

The Python Flight API docs still fail generating because of:

# python
Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow
>>> import pyarrow.flight
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/pyarrow/flight.py", line 18, in <module>
    from pyarrow._flight import (  # noqa:F401
ImportError: /usr/local/lib/libarrow_flight.so.400: undefined symbol: _ZN4absl14lts_2020_09_2319str_format_internal13FormatArgImpl8DispatchINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEbNS2_4DataENS1_24FormatConversionSpecImplEPv
>>> 

# c++filt _ZN4absl14lts_2020_09_2319str_format_internal13FormatArgImpl8DispatchINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEbNS2_4DataENS1_24FormatConversionSpecImplEPv
bool absl::lts_2020_09_23::str_format_internal::FormatArgImpl::Dispatch<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(absl::lts_2020_09_23::str_format_internal::FormatArgImpl::Data, absl::lts_2020_09_23::str_format_internal::FormatConversionSpecImpl, void*)

[pitrou]

For the C++ and Python side at least, it would be easier to use a Conda-based build... but I don't know how that would work for R :-/

[pitrou]

I wonder if we can use Ubuntu's libgrpc++-dev package instead of building from source. As a bonus, the build would be faster...

[kszucs]

I wonder if we can use Ubuntu's libgrpc++-dev package instead of building from source. As a bonus, the build would be faster...

You could try it by adding libgrpc to the ubuntu-20.04-cpp.dockerfile

[pitrou]

Ok, it doesn't seem to work:

• we need a more recent libprotobuf than 20.04 provides
• our CMake build can't find Ubuntu's gRPC or c-ares libraries anyway :-/

[github-actions]

https://issues.apache.org/jira/browse/ARROW-12262

[kou]

We can use system gRPC with Ubuntu 20.10.
Our packages for Ubuntu 20.10 use system gRPC.

[pitrou]

Our packages for Ubuntu 20.10 use system gRPC.

That seems to work. However, c-ares still needs to be built from source. Is there a reason for that? I see that the Flight library links to the system c-ares...

# ldd /usr/local/lib/libarrow_flight.so.400.0.0 | grep ares
	libcares.so.2 => /lib/x86_64-linux-gnu/libcares.so.2 (0x00007f3d253bb000)

[kou]

Ah, it's a bug of our CMake script.
We need to find system c-ares by pkg-config too.

Do you want me to create a patch for it?

[pitrou]

That would be nice, yes :-)

[kou]

OK. I'll work on it.

[pitrou]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: afd1324

Submitted crossbow builds: ursacomputing/crossbow @ actions-285

Task	Status
test-ubuntu-20.10-docs

[pitrou]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: 971b698

Submitted crossbow builds: ursacomputing/crossbow @ actions-286

Task	Status
test-ubuntu-20.10-docs

[pitrou]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: 73993bd

Submitted crossbow builds: ursacomputing/crossbow @ actions-297

Task	Status
test-ubuntu-20.10-docs

[pitrou]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: 9ccb811

Submitted crossbow builds: ursacomputing/crossbow @ actions-299

Task	Status
test-ubuntu-20.10-docs

[kou]

c-ares improvement: #9977

[kszucs]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: 1720a07

Submitted crossbow builds: ursacomputing/crossbow @ actions-316

Task	Status
test-ubuntu-20.10-docs

[pitrou]

Crossbow failed unrelatedly in the Java build :-/

[kou]

@github-actions crossbow submit test-ubuntu-20.10-docs

[github-actions]

Revision: 1720a07

Submitted crossbow builds: ursacomputing/crossbow @ actions-319

Task	Status
test-ubuntu-20.10-docs

[pitrou]

Ok, I don't think the Java failures should block this PR (we may also disable building the Java docs if that's required to upload the 4.0.0 docs). Will merge.