Mipmap Bleeding

[Uradamus]

Godot version:
3.2 Beta 1

OS/device including version:
Manjaro Linux 64bit KDE

Issue description:
Enabling mipmaps for a texture causes very terrible bleeding artifacts that get worse the farther out they go. If I had to guess, the code for mipmap generation is sampling the wrong pixels, or perhaps too many pixels, either way it seems to be including surrounding pixel values into the averaging which is causing the very sloppy results.

Steps to reproduce:
I created a test atlas broken up into 256x256 pixel tiles and a large subdivided plane with each 2x2m face mapped to the same texture tile. The tile in question has a 250x250 pixel green area in the center which the UVs were snapped to the outer edges of, with 3 pixels of white along each side to bring the total up to 256x256. The surrounding texture tiles I filled with red.

Typically 1-2 pixels of bleed with filtering enabled is completely acceptable and expected, hence the 3 pixel white margin for testing. Anything beyond that is completely unacceptable if things are setup properly, and they will show up as red artifacts in this setup thanks to the surrounding red texture tiles.

With mipmapping, as long as you stick to power of 2 sized sub-tile textures and have them perfectly grid aligned, it's expected that mipmaps would be generated cleanly, as they should be only sampling the pixels within the texture region that the newer smaller versions will be representing, typically going down a power of 2 size with each step, so it should only be sampling the 4 pixels needed to generate the corresponding new pixel for the next step down.

Minimal reproduction project:
Test.zip

The following are some screenshots showing off the results with 4 different sets of flag settings, as can be seen it is only the ones with mipmaps enabled that cause the red artifacts, while filtering stays well within the 3 pixel white margins.

1 - filtering & mipmaps on

2 - only mipmaps on

3 - only filtering on

4 - both disabled

[Calinou]

Duplicate of #27837. As a workaround, you can enable anisotropic filtering on the texture to make it less visible.

[Uradamus]

@Calinou - Anisotropic helps a tiny bit, but it's a band-aid on a hatchet wound. The referenced bug is a lot less clear and from what I read it's basically being treated as solved as it's likely to get after the anisotropic suggestion came up. I just pumped up the margin to 8 pixels per side, so only 240x240 usable area in the center of each tile now (losing over 12% area to bleed margins) and it still looks like hot garbage - even with anisotropic on.

This is a really bad look for the engine and should be a far higher priority than it has been so far. I'm not even sure I want to bother with it anymore after how spectacularly it's failed on such a basic quality test.

Are there any plans to add something like UDIM support? That would probably be the ideal replacement for classic atlases. Would then be able to have each texture tile on it's own image, but still gain the advantage of an atlas where each texture has it's own UV space. That seems to be something that isn't possible with texture arrays atm, but the documentation is still really lacking on that front, so maybe I'm missing something.

[Calinou]

This is a really bad look for the engine and should be a far higher priority than it has been so far. I'm not even sure I want to bother with it anymore after how spectacularly it's failed on such a basic quality test.

This is mostly an issue in 3D games that use "atlased" world textures, which aren't very common outside of voxel games (from what I've seen). This may explain why it's considered low-priority.

Are there any plans to add something like UDIM support?

UDIM support sounds difficult to add in a real-time rendering engine. Blender is still having trouble with it, even though they have more people working on the renderer compared to Godot.

[Uradamus]

Atlases are very useful for things like environments, especially when working with a tile-based setup, which is where I'm starting with this project (Think of something a bit like the Pokemon Let's Go game maps from last year, 3D but done in a grid based tile setup with moderately low poly tiles and hand painted textures).

There are a lot of simple tiling textures that really add up on the draw calls when each of them have to have their own material. Something as simple as say a tree tile, would need materials for the ground, the trunk and the foliage, so 3 draw calls instead of 1. And that is far from uncommon, a huge chunk of the tiles I'll be making need multiple textures. Using an atlas I can use a single material for most of the environment, (plus one for semi-transparent stuff and maybe another one or two for animated bits like water or grass/foliage). It can generally be a huge performance boost. The reason why it may seem less common could be because a lot of devs are content to just keep throwing hardware at the problems instead of addressing performance concerns and a lot probably just don't even know any better.

But speaking of Blender, that has absolutely no troubles with mipmap bleeding. I cranked it up to 16 and couldn't see a single artifact except when leaving the image interpolation on linear, but using closest, cubic or smart (goes between bicubic and bilinear based on view distance) all handled that test project like a champ real time in the viewport. I know Sketchfab has also come up with solutions for this sort of thing, they have a nearest with mipmap option from what I recall that would be the goto for something like this, and I'm sure stuff like Unity and UE4 have their own working solutions for this sort of thing.

I don't mind going crazy with the materials when I'm just making something to render out in Blender, but for a game, it's worth putting in the effort and consideration to do everything within reason to minimize draw calls and atlases can play an important role in that for a lot of games.

Godot is just doing something wonky with the sampling that makes a mess of it, I tried having a look at the image.cpp file, but there are like close to a 190 uses of the word mipmap and its all scattered all over the place, would take someone with a lot more familiarity or time than I have to figure out what's going wrong in there.

Something that could be interesting to look into is importing Face Maps from recent versions of Blender (or Polygroups as they are known in a lot of other packages). That could make up for the limited number of UV maps supported. Basically a Face Map works similar to a vertex group, but for faces and each face can only be in at most 1 Face Map, so they are ideal for masking and selections. If we could use those in conjunction with texture arrays to map certain indexes to different groups of faces, it would solve a lot of the same problems as UDIMs for this sort of thing. Though also just allowing an arbitrary number of UV maps would also be a solution and would likely be the easiest to implement.

[NeoSpark314]

@Calinou asked me about this issue in another thread:
I don't know of any easy way to fix the issue of MipMapping + Texture Atlas. The best solution I know of if you have square same-sized tiles in your atlas is to first compute the Mip Maps manually for each tile (with wrap around) down to one pixel and then put these into the atlas mip-map. And then you have to make sure in the shader that your texture-lookup never filters across tile borders in the mip-map (for this you need to know in which tile you are to properly clamp the uv-coordinates and also control the TextureLoD lookup and also potentially implementing filtering yourself in the shader). Something like what I just described was written in more detail here: https://0fps.net/2013/07/09/texture-atlases-wrapping-and-mip-mapping/
If the target hardware supports texture arrays this would be probably the preferred solution as was already mentioned above somewhere.
Since the solution needed will usually depend a lot on how the atlas is created and will be specific for the user (except maybe the minecraft style case :-) one potential solution strategy to the problem would be to allow users to provide some logic/code for custom mip-map generation; or perhaps this might not be necessary if godot supports a texture format that can already contain mip-maps (.dds .ktx and .basis can all contain precomputed mip-maps).

[Calinou]

@NeoSpark314 Godot allows loading DDS images with custom mipmaps, I just tested it:

Demo project: test_custom_mipmaps.zip

The included DDS image is uncompressed, but you can also use DXT compression and Godot will be able to import it.

You can use GIMP 2.10 to load and save DDS images with custom mipmaps by choosing Use existing mipmaps in the Export dialog.

[NeoSpark314]

Thats cool; This allows then to build an external pipeline that generates the correct mip maps for the atlas and stores them in the .dds. On godot side you would need only to implement atlas-aware UV texture coordinate transforms/clamping so you don't filter outside of the tile but I think this should be possible in the godot shading language.

[Uradamus]

I kinda wonder if the AtlasTexture resource meant for 2D could be of use here. Since it has that filter_clip property. Right now they don't seem to work at all with spatial materials, but if that were changed, perhaps they could help with this situation.

I was thinking about getting into messing around with premade mipmaps with DDS, but a sorta annoying thing is that I couldn't figure out a way to get textures that were automatically imported with the model into the import window, and the use existing mipmap thing didn't seem to be among the settings that could be accessed in the inspector through the material settings.

At this point though, I'm almost tempted to give it another year or two before trying again, and just focus on some 2D game ideas and non-game 3D projects. There is so little in terms of learning material for the 3D side, the documentation is still kinda weak, and its obvious a lot of areas are being neglected until the Vulkan backend is ready. Just seems really not ready enough by a long shot and I'm too inexperienced with the engine in general to really want to try to blaze any trails here.