Use a special D docker image for CircleCi

[wilzbach]

The Docker image is automatically built and updated: https://github.com/wilzbach/dlang-docker-circleci
I picked ubuntu:17.04 on purpose as currently the DMD and DRuntime testsuite doesn't work on a PIE-hardened system.
Hence, the build should fail.

This will need something like #7420 or #7427

[dlang-bot]

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

[wilzbach]

Hmm any Docker experts in the house?

When this image is run on CircleCi I get weird permission errors - which don't happen locally:

> ls -l /dlang/dmd-2.078.0   
ls: cannot access '/dlang/dmd-2.078.0/linux': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/freebsd': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/samples': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/html': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/src': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/man': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/windows': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/solaris': Permission denied
ls: cannot access '/dlang/dmd-2.078.0/osx': Permission denied
total 16
-r-xr-xr-x 1 root root  600 Jan  2 19:53 README.TXT
-r-xr-xr-x 1 root root  782 Jan  4 08:03 activate
-r-xr-xr-x 1 root root  894 Jan  4 08:03 activate.fish
d????????? ? ?    ?       ?            ? freebsd
d????????? ? ?    ?       ?            ? html
-r-xr-xr-x 1 root root 1338 Jan  2 19:53 license.txt
d????????? ? ?    ?       ?            ? linux
d????????? ? ?    ?       ?            ? man
d????????? ? ?    ?       ?            ? osx
d????????? ? ?    ?       ?            ? samples
d????????? ? ?    ?       ?            ? solaris
d????????? ? ?    ?       ?            ? src
d????????? ? ?    ?       ?            ? windows

> docker run -i -t dlang2/dmd-circleci:2.078.0
> ls -l /dlang/dmd-2.078.0      
total 16
-r-xr-xr-x 1 root root  600 Jan  2 19:53 README.TXT
-r-xr-xr-x 1 root root  782 Jan  4 02:49 activate
-r-xr-xr-x 1 root root  894 Jan  4 02:49 activate.fish
dr-xr-xr-x 4 root root   32 Jan  2 21:20 freebsd
dr-xr-xr-x 3 root root   33 Jan  2 21:20 html
-r-xr-xr-x 1 root root 1338 Jan  2 19:53 license.txt
dr-xr-xr-x 6 root root   58 Jan  2 21:20 linux
dr-xr-xr-x 4 root root   30 Jan  2 21:20 man
dr-xr-xr-x 3 root root   17 Jan  2 21:20 osx
dr-xr-xr-x 3 root root   15 Jan  2 21:20 samples
dr-xr-xr-x 4 root root   32 Jan  2 21:20 solaris
dr-xr-xr-x 5 root root   62 Jan  2 21:20 src
dr-xr-xr-x 3 root root   17 Jan  2 21:20 windows

The image is built by the modifying the Docker base script (https://github.com/wilzbach/dlang-docker/blob/master/circleci/dlang.docker) - the custom patching is needed as AFAICT they only push their built images do DockerHub, not the base layer.

(tested with image: dlang2/dmd-circleci@sha256:ee5c105f593b03a295c2a010a2e74561fa2aad19e9016fbe1bc86d6306f83edd)

[jondegenhardt]

@wilzbach Regarding the permissions issue - I don't know if it is the same problem, but I had permissions issue when using the dlanguage/ldc docker image on Travis-CI. On Travis, you are running on their base OS (ubuntu 14.04), and invoke the docker image via docker run. The source files are passed on the docker run line, and the output files are generated by the docker image. However, the docker image runs as a different user, and the files it produces are not readable from the base image. What I did to fix this was use a Travis instance with sudo capability and reassign the file permissions afterward. I did this recursively, for the entire directory tree. To see what I did, look at the tsv-utils: .travis.yml file. Line 69-70 saves the UID/GID, lines 129-130 issue the docker run command and the sudo chown that resets the file permissions. Lines 113-115 do something similar, but actually run the docker built exe on the travis base platform (testing that the exe built with Ubuntu 16.04 will run on Ubuntu 14.04).

[wilzbach]

Nice - this is finally failing:

runnable/test42.d(4184): Deprecation: identity comparison of static arrays implicitly coerces them to slices, which are compared by reference
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test423Foo7__ClassZ' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '__dmd_personality_v0' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test425foo6dFZ12__critsec260G48g' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test423C158callbackMFiZv' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test423C157__ClassZ' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test425foo16FZv' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc against '_D6test426Test227__ClassZ' which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC
/usr/bin/ld: error: test_results/runnable/test42_0.o: requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC

So we just need to fix the testsuite and then include this PR - or cherry-pick this PR into the PR that fixes the testsuite.

(I will work a bit more on this as I would like the -fPIC to run as a third job for optimal code coverage)

[wilzbach]

Regarding the permissions issue - I don't know if it is the same problem, but I had permissions issue when using the dlanguage/ldc docker image on Travis-CI.

I think it's a different one, for some weird reason all I need to do to fix them is:

sudo ls -l /dlang/dmd-2.078.0/linux && sudo ls -l /dlang/dmd-2.078.0/src

Also I tried using CircleCi 2.0 workflows as this seems that's the only way to use different docker images.
However, this leads to CircleCi reporting each job back as a different GH status event:

[wilzbach]

Cherry-picked this to #7420 -> closing.