Host icu downloads locally (icu download is breaking)

[jbergstroem]

We've seen quite a bit of these through the buildbots:

** Error occurred while downloading
 <http://download.icu-project.org/files/icu4c/55.1/icu4c-55_1-src.zip>
Traceback (most recent call last):
  File "configure", line 1068, in <module>
    configure_intl(output)
  File "configure", line 967, in configure_intl
    localzip = icu_download(icu_full_path)
  File "configure", line 836, in icu_download
    nodedownload.retrievefile(url, targetfile)
  File "tools\configure.d\nodedownload.py", line 38, in retrievefile
    msg = ConfigOpener().retrieve(url, targetfile, reporthook=reporthook)
  File "C:\Python27\lib\urllib.py", line 289, in retrieve
    "of %i bytes" % (read, size), result)
urllib.ContentTooShortError: retrieval incomplete: got only 13001029 out of 27025781 bytes

There's a few routes to explore here to improve the actual download; but one thing we can do from an infra point of view could be setting up a proxy_store location match on our local nginx that would pull files from icu-project.org and then allow us to use our cdn to cache it. Thoughts?

[rvagg]

The simple fix I have in mind is to put the .zip in a specific place on the build machines (/home/iojs/icu4c-55_1-src.zip would work for everything non-Windows as it stands now) and do a cp as part of the Jenkins scripting to drop it in to place. These failures are becoming too frequent—another variation is a failed checksum once its downloaded. Also, it adds precious minutes to the build process which we could do without.

/cc @nodejs/intl

[jbergstroem]

I have a patch adding a download-path to configure. Good enough?

[rvagg]

can you use that to point to a directory and it'll check if the .zip is in that directory? that'd be perfect because it'd solve for the case where we don't have the .zip in place or it's not the right version.

[jbergstroem]

Yep. I'd also like to switch to curl/wget. On windows curl is aliased to Invoke-RestMethod which accepts -Outfile /path/to/$foo. This way we can at least have resume support on !Windows

[rvagg]

this is getting out of hand, the OSX release slaves have been dying part way through ICU download, latest incarnation was

 Fetch: : 26.4MB total, 10.4MB downloaded  Slave went offline during the build
ERROR: Connection was broken: java.io.EOFException
    at org.jenkinsci.remoting.nio.NioChannelHub$3.run(NioChannelHub.java:613)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
...

[orangemocha]

FYI, @srl295 and I have been pondering whether it would be feasible to just host it in git. Not much progress on that investigation, but it's on our plate.

[jbergstroem]

@orangemocha as in, bundle in deps/? I'm -1 to growing our git repo for a currently optional build-time dependency.

[srl295]

Ahh, let me see if I can get the sourceforge mirror to work again. There's an issue somewhere to have a backup URL.

[srl295]

@jbergstroem ICU 54 and 55 are now redirects to sourceforge CDN, please let me know if this helps.

[srl295]

@rvagg:

can you use that to point to a directory and it'll check if the .zip is in that directory?

You can already do that! --with-icu-source=<somepath>/icu.zip

edit also it will check if its icu4c-xx-src.zip file already exists in deps/ and use that.

[srl295]

The "use backup URLs" issue is now nodejs/node#2746

[rvagg]

edit also it will check if its icu4c-xx-src.zip file already exists in deps/ and use that.

That's what I'm after! I'm going to hook this up to some of the Ci machines and get them to do a cp just before configure

[srl295]

Perfect.

El martes, 8 de septiembre de 2015, Rod Vagg notifications@github.com
escribió:

edit also it will check if its icu4c-xx-src.zip file already exists in
deps/ and use that.

That's what I'm after! I'm going to hook this up to some of the Ci
machines and get them to do a cp just before configure

—
Reply to this email directly or view it on GitHub
#190 (comment).

[rvagg]

We really need to do something about this folks, it holds up every release now, it's been a huge frustration to both @Fishrock123 and I for each release we've pushed through, many many start/stop cycles and the problems with the Windows linker hanging for minutes and causing the next build to fail if you start it too soon only exacerbates the problem. If I need to take some action please let me know, I'm unsure what the state of this is.

See https://ci.nodejs.org/job/iojs+release/nodes=win2008r2-release-ia32/192/console for the latest drama, happens most frequently on Windows.

[rvagg]

^ that was 4.1.1 btw, a wasted hour trying to get a successful build even started

[rvagg]

 Fetch: ' 26.4MB total, 26.4MB downloaded   
 ** Error occurred while downloading
 <http://download.icu-project.org/files/icu4c/55.1/icu4c-55_1-src.zip>
Traceback (most recent call last):
  File "configure", line 1075, in <module>
    configure_intl(output)
  File "configure", line 974, in configure_intl
    localzip = icu_download(icu_full_path)
  File "configure", line 843, in icu_download
    nodedownload.retrievefile(url, targetfile)
  File "tools\configure.d\nodedownload.py", line 38, in retrievefile
    msg = ConfigOpener().retrieve(url, targetfile, reporthook=reporthook)
  File "C:\Python27\lib\urllib.py", line 289, in retrieve
    "of %i bytes" % (read, size), result)
urllib.ContentTooShortError: retrieval incomplete: got only 27021421 out of 27025781 bytes

Same error, persistent on both Windows boxes, I need to get v4.1.2 out today as announced for fixing the DoS bug and I can't get past this. Halp! @nodejs/intl @nodejs/build

[rvagg]

Cleaned out workspaces, emptied trash and rebooted those machines and they both suddenly work again, perhaps this is a temporary workaround, a pretty terrible one but it's something.

[srl295]

Rod, thought this was solved. Let's chat about this as soon as possible.
edit did anything come of the plan in #190 (comment) ?

[srl295]

This will improve matters once it lands: nodejs/node#3200 - if we land this one before ICU 56 is landed, then manually caching ICU-54 and ICU-55 on the buildslaves will be sufficient.

[jbergstroem]

I believe we're now hosting these on all release-machines. We should improve our ansible-stack by downloading these as part of the setup, though.

[joaocgreis]

All windows test machines have this now.

C:\>dir node-icu
 Volume in drive C has no label.
 Volume Serial Number is 8AD6-8BDC

 Directory of C:\node-icu

12/09/2015  01:44 PM    <DIR>          .
12/09/2015  01:44 PM    <DIR>          ..
12/09/2015  01:42 PM        26,945,277 icu4c-54_1-src.zip
12/09/2015  01:43 PM        27,025,781 icu4c-55_1-src.zip
12/09/2015  01:44 PM        27,128,416 icu4c-56_1-src.zip
               3 File(s)     81,099,474 bytes
               2 Dir(s)  82,721,615,872 bytes free

[jbergstroem]

@joaocgreis these are only relevant for release machines (for now)