ORC-1635: Download orc-format from dlcdn.apache.org instead of archive.apache.org

[progval]

What changes were proposed in this pull request?

Download orc-format from dlcdn.apache.org instead of archive.apache.org

Why are the changes needed?

https://archive.apache.org/ discourages heavy use, and its rate limits can cause CI systems building Apache ORC to be banned.

How was this patch tested?

It builds from a clean repo

Was this patch authored or co-authored using generative AI tooling?

no

[deshanxiao]

Thanks @progval Pending CI.

[dongjoon-hyun]

Hi, @progval and @deshanxiao .

Are you sure with this?

IIRC, since Apache Download is only serving the latest versions, we cannot use it in the source code. The released branches are going to be broken easily when we release a new orc-format.

[dongjoon-hyun]

Here are the example. You can see that 404 Error for Apache Spark 3.5.0.

$ curl --head https://dlcdn.apache.org/spark/spark-3.5.1/spark-3.5.1.tgz
HTTP/2 200
server: Apache
last-modified: Thu, 15 Feb 2024 11:39:51 GMT
etag: "21ae2b9-6116a15e24d57"
access-control-allow-origin: *
content-type: application/x-gzip
via: 1.1 varnish, 1.1 varnish
accept-ranges: bytes
age: 21
date: Mon, 26 Feb 2024 17:48:04 GMT
x-served-by: cache-hel1410020-HEL, cache-sjc10040-SJC
x-cache: MISS, HIT
x-cache-hits: 0, 0
x-timer: S1708969685.562024,VS0,VE45
content-length: 35316409

$ curl --head https://dlcdn.apache.org/spark/spark-3.5.0/spark-3.5.0.tgz
HTTP/2 404
server: Apache
content-type: text/html; charset=iso-8859-1
via: 1.1 varnish, 1.1 varnish
accept-ranges: bytes
date: Mon, 26 Feb 2024 17:48:11 GMT
age: 0
x-served-by: cache-hel1410022-HEL, cache-sjc1000130-SJC
x-cache: MISS, MISS
x-cache-hits: 0, 0
x-timer: S1708969691.332510,VS0,VE511
content-length: 196

[dongjoon-hyun]

Let me close this to prevent any accidental merging first.

Feel free to reopen this if you have any other ways, @progval .

[douardda]

Maybe we could put both URLs since the ExternalProject URL entry allows several entries.

[dongjoon-hyun]

Maybe we could put both URLs since the ExternalProject URL entry allows several entries.

+1 for the suggestion, @douardda . You want to put dlcdn first as a cache, right?

[douardda]

Maybe we could put both URLs since the ExternalProject URL entry allows several entries.

+1 for the suggestion, @douardda . You want to put dlcdn first as a cache, right?

yes

[progval]

Submitted at #1830

[douardda]

Submitted at #1830

thanks!