Port raycast module to Embree 4

[akien-mga]

Godot version

4.1.beta (5983785)

System information

All

Issue description

The raycast module uses the thirdparty Embree library, currently version 3.13.5.

Upstream released Embree 4 as a new major, compatibility breaking release, and we need to adapt our usage of the library API if we want to use it.

The current latest version is 4.1.0. Release notes:

• https://github.com/embree/embree/releases/tag/v4.0.0
• https://github.com/embree/embree/releases/tag/v4.1.0

#73631 was an attempt at adding support for both majors at the buildsystem level, but due to the API changes the embree4 variant wouldn't compile at all so I abandoned it.

Doing this upgrade is not urgent, and once evaluating it we might find reasons not to (e.g. I see 4.1.0 just re-added support for Linux ARM64 which 3.13.5 already had, so there might be other such downgrades for our use case). Anyone is welcome to have a look.

[Calinou]

Can I go ahead and attempt this? it probably won't go well but I might as well try.

Sure, feel free to open a pull request for this 🙂

However, in the long run, we aim to replace Embree with a simpler BVH/software raytracing implementation such as https://github.com/madmann91/bvh or https://github.com/lighttransport/nanort. This will allow decreasing binary size and compile times significantly, as Embree is a large, complex library. OpenImageDenoise was removed from core for similar reasons (it was moved into a separate CLI program and eventually a dynamic library).

Replacing Embree entirely is more difficult than updating to Embree 4 though.

[quirkylemon]

How long will it be until Embree is replaced?

[Calinou]

How long will it be until Embree is replaced?

There's no ETA for replacing Embree, so I don't know.

[quirkylemon]

So currently it looks like we need to manually update cpp_files[] and include_dirs[] in godot_update_embree.py would it be helpful to include the script I am writing to automatically make these lists?

[akien-mga]

Probably not - those lists are written manually because we want to have precise control over which files we build, and not build any file that we don't need. It's the same for all thirdparty libraries, we trim them down to only include and compile what we will actually use.

Unless of course your script can automate figuring out which is the minimal subset of files in upstream embree that Godot needs, but in my experience that's best done with human reasoning and changes from release to release, hence the manual listing.

[quirkylemon]

This project is to hard for me right now. so I am going to give up.

[Chubercik]

However, in the long run, we aim to replace Embree with a simpler BVH/software raytracing implementation such as https://github.com/madmann91/bvh or https://github.com/lighttransport/nanort. This will allow decreasing binary size and compile times significantly, as Embree is a large, complex library.

What about performance?

(source: https://github.com/madmann91/bvh?tab=readme-ov-file#performance)

From what I understand, Embree is the fastest solution out of the (popular) available ones (at least it was in version 3, no idea how 4 compares, but I wouldn't expect much of a regression in this area).

[Calinou]

@Chubercik It's true that Embree is generally faster across the board, but smaller binary size and maintainability is something that tends to be valued a lot in Godot.