Add working appveyor config for 64+32 bit, current Travis build #406
Conversation
|
OK that's the tar xvf changed over to your suggestion. I think the appveyor and travis files have to stay in the root dir, the other couple I've moved out to a ./ci directory. Not sure what's up with the cudnn problem, I've switched over to the new dependency but still get this: [ERROR] /C:/projects/javacpp-presets/cuda/src/main/java/org/bytedeco/javacpp/helper/cudnn.java:[25,34] cannot find symbol |
|
Does this happen only on Windows or on Linux as well? |
|
Ah think I realised what it is.. the .m2 dir is set to be cached in appveyor and travis (may or maynot be a good idea in itself), I think there's an old build of javacpp or something else lurking in there probably. It built fine on a fresh windows elsewhere. So, I think ignore that one. For the rest, I think it's ready to go (depending if you want to do anything about hdf5 in caffe now or revisit it later) |
|
Ok, good! I've made the changes for Caffe, so it now depends on HDF5... Also, there's a couple of |
|
Those are in, and the caffe stuff merged back too. Think that covers it all? Does win32 build OK for you? I had a quick look through the difference in env settings, but can't see anything obvious. Wonder if the cudnn lib/dll is really only working with 64 bit now? |
|
CUDA isn't available for windows-x86, it should just produce an empty JAR, but it looks like I forgot to take that into consideration when adding a helper for CUDA. I'll have to fix this. Thanks for letting me know! Other than that, let's switch to Visual Studio 2015, because it's going to be needed for LLVM (pull #407), MXNet (pull #309), and probably others in the future. I've pushed the few necessary changes in 7dfdee3 Let's also rename Another thing, because Caffe's build now depends on HDF5, we'll need to add on the same line, am I correct? Finally, what is BTW, in that file and in Thanks for everything! I'll be merging that after this round :) |
|
I'll post when it's done (bit confusing as I have to commit to see if it builds, which makes it look a bit like the PR is ready). The Travis stuff was still outstanding as I was focussing on getting appveyor out of the way first. But yeah, no harm getting that into a better shape too (there'll still be more to come on there though, but should be a decent start point). |
|
Looks like there is a command to avoid timeout from log silence: |
|
Yeah mixed results so far with that.. its basically just a function much
like I put in the prog.sh script, but without the flexibility I needed
(can't remember the main issue off the top of my head, think it might have
been getting the exit status back of the build subprocess. Ah, I think it
was to do with trying to redirect output to a file, but needing their
function to still output to stdout, I don't think it supports that (guess
intention is for process that don't output much due to long compile steps,
rather than having their output redirected).
There's a few other issues that seem to have cropped up since I last did a
pull:
- tensorflow is looking for cudnn 5.1 rather than 6.0, and doesn't look
like they can live together cleanly (different header files)
- no luck with flycapture right now, maybe something to do with new
version, though I've tried old and new
- tesseract is now breaking both linux and windows builds, something up
with the new version?
Those and the cuda x32 issue are the only red cases on appveyor now I
think.
On the Travis side, openblas generates alot of output and fails due to
that, seems to be a gcc output line per file, which is huge. Switching to
cmake doesn't help much, its slightly tidier but still alot of lines of
output progress. Was cases like these that I had the prog.sh in mind for -
it's just one line of code, can probably add it directly into .travis file
…On 10 April 2017 at 15:34, Samuel Audet ***@***.***> wrote:
Looks like there is a command to avoid timeout from log silence:
https://docs.travis-ci.com/user/common-build-problems/#
Build-times-out-because-no-output-was-received
So we could try that along with bazel build --show_result 0
--noshow_progress .... The less hack we have to use, the less potential
future headaches IMO :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#406 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AMYRJ3AJVrO-eAMsmRrrPJ0Jej17-xdjks5ruj4MgaJpZM4Mw2tQ>
.
|
|
OK, Appveyor side is looking pretty good I think, just cuda32bit and tesseract breaking, and thats switched to vs2015. On Travis side, its in reasonable state, I think the hardcoded stuff is gone, prog.sh is gone, there's too much output from the gcc build as mentioned, but that needs a bit more thought (as its run inside docker, lifes a bit more complicated). But seems a reasonable state to get this in if you feel happy enough with it, as there might still be some teething problems |
|
Great, thanks! That some builds are not passing is not a problem. It's precisely to spot these issues in advance that we want to use CI. It's actually a very good sign that it's doing what we want it to do! 👍 |
Hopefully nothing here looking too significant. Main issue where I've had to change existing files is to reduce down the size of the logs generated during a build, both travis and appveyor have 4mb limit. The HDF5 build during caffe totally blows this out, they have lots of warnings enabled. For the moment, I've redirected it to a log file so its there for investigation if there's an issue. Maybe longer term solution a bit like what I've started with a couple of the others, is some redirection if its called from CI, to dump to a $PROJ.log file. Travis and I think Appveyor let you upload artefacts, and it works with Travis to push the logs to a AWS S3 storage. But, that adds a whole load of complication around access permissions that would be nicer to avoid.
For the rest of noise reduction, I've just taken off verbose extractions of tar files, saves alot of output that probably doesn't add much debug use (if there's an error in the file, I think you'd still get a pointer to it).
Something up with cuda build currently too, but I think maybe genuine issue, something missing in javacpp?
Anyway, maybe worth giving these a try, check I've re-encrypted logins properly. Seems to deploy snapshots during the build OK, would be good to check 32 bit build is as expected.