Replace glsl-to-spirv with shaderc

[djcsdy]

Replaces glsl-to-spirv with shaderc.

glsl-to-spirv is deprecated and the author states: "use shaderc-rs instead".

PR checklist:

• make succeeds (on *nix)
• make reftests succeeds
• tested examples with the following backends: dx12, vulkan
• rustfmt run on changed code

Note: make reftests is failing for dx12 on master, so it’s failing on this branch too. It passes for vulkan.

[djcsdy]

Urgh. The CI build is failing on Windows due to the 260 character path limit 🤢.

[djcsdy]

This PR now also installs ninja on Travis Windows before the build.

ninja is an optional build-time dependency of shaderc. It is recommended on Windows because without it the shaderc build process generates files with very long pathnames, which can cause build failures.

[msiglreith]

Hi and thanks for the PR!
How is the experience for a user who wants to start using gfx-rs by trying out an example or building code on top of existing ones? Are there further build systems required to get started, which might especially a blocker on windows systems?

[djcsdy]

This adds a dependency on Python on all platforms, and on Windows sometimes adds a dependency on ninja.

I am using Windows myself and I didn’t encounter significant problems. The standard Windows distribution of Python 3 from https://www.python.org/downloads/windows/ works fine. Of course Python has to be on the PATH. The Python installer can do this for you – it is optional but IIRC turned on by default.

Ninja is required on Windows if the path to the gfx source tree is longer than about 30 characters. This is because, without Ninja, the shaderc build process uses a different project generator that generates excessively long path names, which can cause a build failure on Windows. I didn’t experience this problem on my machine because I had the project checked out at a shorter path, but as you can see it blew up on Travis and it was not immediately obvious what the solution was. I think it would be a good idea for the shaderc project to just add a hard dependency on ninja at build time on Windows, since at least then it could produce a helpful error message.

I’m not aware of any other additional dependencies compared to what this project already depended on – shaderc depends on cmake and a C compiler, but those were already required.

[djcsdy]

I proposed a PR to shaderc which would make Ninja mandatory when building for Windows with MSVC: google/shaderc-rs#47

[fu5ha]

IMO we should not adopt shaderc for the examples but we should in warden. We can make a note that shaderc is a better choice for a real project, but since glsl-to-spirv works for the quite simple use case we need it for, and honestly doesn't really break until you get a fair bit deeper, it "just works", and it doesn't double or more the build times.... I think it is worth it to leave in.

[ghost]

There's some earlier discussion about this subject in #2546.

I agree that shaderc build times are not great. I'm also not a fan of the additional dependencies to Python and ninja. Was CMake already a dependency for gfx-hal? Where is this dependency?

We might be able to avoid these dependencies by wrapping glslang ourselves and invoking it directly instead of going through shaderc.

[djcsdy]

I thought CMake was a dependency of gfx already, but I guess I was mistaken. I installed CMake for some Rust project, but I guess it wasn’t this one.

[ghost]

Looking into it, glsl-to-spirv does have CMake as a dependency, but since it includes pre-built binaries on Windows, it's only used on the other platforms.

[knappador]

Static lib options and crate behavior have been made more normal and default behaviors are expanded in google/shaderc-rs#46

The expanded static options just make it easier to pick a path that builds really fast with few dependencies. The old --no-defaults option was funky (but still supported, just deprecated).

I don't think this makes life easier on gfx since it's pretty heavily focused on expanding options for Linux CI. @djcsdy if you want support on getting the build script working on Windows or have anything you want but didn't get, just say so.

Appveyor is currently advertising a build failure, but I don't have access to the failure info to go fix it. Time for other-platform people to ✨ ✨

[djcsdy]

@knappador The build does work on Windows. Check my last commit.

[djcsdy]

@knappador Scratch that sorry, I misunderstood. I see you are referring to google/shaderc-rs#46, which is currently failing on Windows.

[knappador]

@djcsdy I'm fixing a line that got regressed as soon as I can shove it into history realtime github chat

[knappador]

@djcsdy okay tests passed on the CI servers. No idea what platforms or coverage they have. I don't think you need ninja to do the static lib build paths. I pretty much just cargo-culted forward what was there for the --no-defaults legacy path in build.rs for non-Linux builds

[knappador]

Shaderc's crate split has landed with some additional options to get pre-built binaries linked. I'm available to guide integration attempts and land additional PR's against shaderc-rs if more build variations are desired.

[imkow]

functionality side, shaderc looks fine, but I think it's not ready for cross platform support. And it depends on too many 3rd party stuff.

[djcsdy]

I don’t see any path forward for me here.

If someone else wants to pick this up, please do. But consider me thoroughly discouraged.

[kvark]

@djcsdy sorry about the way this PR progressed (and essentially ended up unmerged)!
Please don't hesitate to visit our chat to talk or just provide feedback.