USE_BINARYBUILDER=0 on master

[smartalecH]

Does USE_BINARYBUILDER=0 only work for "full" release tarballs, and not for the git master branch? From the docs, it sounds like it should indeed work... however the current master branch ignores the flag and always attempts to download the binaries.

[DilumAluthge]

It is intended to work on master; if it doesn't, then it's a bug.

Can you provide:

1. Your operating system
2. Your CPU architecture
3. The exact commit of master that you tried to build
4. The exact make invocation that you used
5. Did you have a Make.user file? If so, can you post the contents of it?

[smartalecH]

Got it, thanks!

Your operating system

CentOS 8 (CentOS 7 also fails)

Your CPU architecture

x86_64, Intel Broadwell

The exact commit of master that you tried to build

ab262e73104994e05c149d5b21f1ba71139b6fca

The exact make invocation that you used

I tired make with a Make.user. That didn't work, so I tried make USE_BINARYBUILDER=0. That didn't work either.

Did you have a Make.user file? If so, can you post the contents of it?

Yes,

USE_BINARYBUILDER=0

[vtjnash]

I tried to do a fresh checkout of Julia with that flag to test locally, and I don't see any binary-builder flags being attempted to download. However, I did detect 4 broken rules, so I also had to disable the builds for CSL, LLD, ZLIB, and P7ZIP to get anywhere after downloading the sources:

USE_BINARYBUILDER=0
USE_SYSTEM_CSL=1
USE_SYSTEM_LLD=1
USE_SYSTEM_ZLIB=1
USE_SYSTEM_P7ZIP=1

[smartalecH]

so I also had to disable the builds for CSL, LLD, ZLIB, and P7ZIP to get anywhere after downloading the sources:

So that's indeed problematic. But I think I also misunderstood something -- are the deps sources themselves not bundled? I was under the impression that with USE_BINARYBUILDER=0, I shouldn't need to download anything?

[StefanKarpinski]

No, we don't bundle any dependency sources. If you disable BB, then the makefiles will download all dependency sources from their origin locations and build them for you.

[smartalecH]

Great, thanks @StefanKarpinski!

In the docs, I'm looking at the following:

julia/doc/src/devdocs/build/build.md

Lines 19 to 26 in 23160a1

	dependencies](#required-build-tools-and-external-libraries). If you
	prefer to build all the dependencies on your own, or are building on a system that cannot
	access the network during the build process, add the following in `Make.user`:
	```
	USE_BINARYBUILDER=0
	```

I'm on a system that doesn't have access to the network during the build process. But I want to build from master, so that I can leverage a recent bugfix not yet backported to the "full" release tarballs (#46976). I was under the impression that USE_BINARYBUILDER=0 would take care of this for me, per the above blurb. Do I have a workaround? Perhaps a command that can fetch the sources beforehand (while I do have network access) and then build as if this were a "full" release?

Thanks for the help!

[smartalecH]

Perhaps a command that can fetch the sources beforehand (while I do have network access) and then build as if this were a "full" release?

I found this useful command:

julia/Makefile

Line 537 in 67b8ac0

@$(MAKE) -C deps getall DEPS_GIT=0 USE_BINARYBUILDER=0

which can be used to fetch all the dependency sources before I start the build process (where I lose network access). While it fetches most sources, it does throw the following error:

make: *** No rule to make target 'get-lld', needed by 'getall'.  Stop.

Although it looks like #47589 fixes this?

[ViralBShah]

I believe this works now. Please reopen if still broken.