Book 3: possible bug causing extra-hot pixels?

[Walther]

A thought I haven't been able to verify in the code yet: I wonder if it is possible that the Book 3 indirect lighting & material changes have something that might cause occasional "hotter pixels" generated than the approaches in the previous books.

With the random, unweighted scattering approach in the first two books, the images as a whole are somewhat noisy with super-low sample counts and improve by increasing the sample count, as expected. However, even from fairly low sample counts onward, the noise looks fairly smooth and uniform across the image.

With the direct-lighting-only, with the probability density functions pointing at the light source as per early parts of Book 3, you get ultra-smooth images, even with super-low sample counts. There's no visible noise pretty much anywhere. Of course this comes with the caveat of lacking any indirect lighting.

With the mixture-pdf lighting, and the other changes in materials at the end of the Book 3, there seems to be a drastic change in the noise profile. It seems like there's suddenly a significant amount of isolated extra-hot pixels, that were not present in previous renders. These can be white or colored, and they "jump out" visually a lot more than the noise in the previous images.

I'm attaching a zip of a progression, rendering a Cornell box with one Lambertian box and one metal box, at samples-per-pixel counts of 1 through 1024, doubling between each image. Observe the hot pixels at various sampling levels: white pixels, red pixels, green pixels, looking distracting in their surroundings. Curiously enough, even at 1024 samples per pixel, you can see those surprise extra-hot pixels here and there. (Sadly due to implementation, these are not a progression of the same exact render adding more iterations; images have been generated as separate runs with a different spp parameter provided as an argument)
progression.zip

For comparison, I checked out my codebase to an older commit - pre-book3 - and did a similar series of renders at varying spp levels. In this zip, you don't really see hot pixels standing out at beyond say, 8 samples per pixel. The noise profile is uniform across the board, and at least to my personal taste, even aesthetically pleasing from say, 128 spp forward.
progression2.zip

---

While I understand that denoising can be an extensive topic unto its own, and not in the scope of this book series, I do wonder if there could be some bug somewhere in the Book 3 causing the drastic change in noise profile. At least to my personal taste, a uniformly-noisy surface does not look bad, but hot pixels easily distract and become a focus for the eye.

This is probably at least somewhat adjacent to #637

[Walther]

Thinking out loud:

To investigate this further, i added some conditional logging to my ray color function: if a component of the final computed color was larger than a threshold, print an error message. I noticed that even for surprisingly large values, the print can get triggered.

While my implementation is in another language, I managed to reproduce it in this repo too with a simple change to the end of the ray_color function:

    // TheRestOfYourLife/main.cc at commit 5c312de55e33fa97b0885b00ec86c908f7adba0b
    auto final_color = emitted + srec.attenuation * rec.mat_ptr->scattering_pdf(r, rec, scattered) * ray_color(scattered, background, world, lights, depth - 1) / pdf_val;
    if (final_color.x() > 30.0 || final_color.y() > 30.0 || final_color.z() > 30.0)
    {
        std::cerr << "\nA color component was larger than 30.\n";
    }
    return final_color;

When running ./theRestOfYourLife > testfile-30.ppm, this ends up printed quite a bit! Considering the light has color values of 15, I'm surprised how color components of value over 30 can be present, considering the materials should be attenuating or reflecting the light - which should result in less-than-or-equal behavior, if i'm not mistaken 🤔

Perhaps something can occasionally cause too-big values - causing these hot pixels?

[trevordblack]

I suspect that the reason for the extra hot pixels is because a 50-50 weight mixture of surface and light is waaaay too much. Something closer to 90-10 surface-light.

But the color values of 30 for a light of value 15 is a new one. I'll have to dig in and try to figure it out.
Thanks. sigh

[trevordblack]

Following up on this.

The reason that we get values that are greater than the max brightness of the light (indeed, that we want values that are greater than the max brightness of the light) is because the / pdf_value will increase the result.

The values of pdf is between 0 and 1, so we get a greater number when we divide, a slight modification to your code above:

    if (final_color.x() > 30.0 || final_color.y() > 30.0 || final_color.z() > 30.0)
    {
        std::cerr << "\nA color component was larger than 30.\n";
        std::cerr << "The mixture PDF value for the light was: " << pdf_val;
    }

Bores this out:

Scanlines remaining: 584
A color component was larger than 30.
The mixture PDF value for the light was: 0.00568558
A color component was larger than 30.
The mixture PDF value for the light was: 0.0989286
A color component was larger than 30.
The mixture PDF value for the light was: 0.109055
A color component was larger than 30.
The mixture PDF value for the light was: 0.0982704

It's fine for a reflection to be a higher value than the actual brightness, we use the pdf to hyperbolically approach the true value

[trevordblack]

We can start modifying the percentage of rays that are steered toward the lights versus reflected:

From within mixture_pdf

vec3 generate() const override {
  if (random_double() < 0.5) // Change this number
    return p[0]->generate();
  else
    return p[1]->generate();
}

We see what we get with a 50-50 split:

Then we can steer 10% of the rays toward the lights (ceiling quad and sphere):

Or 90% of the rays toward the lights:

You'll notice that the number of fireflies is greatly determined by the percentage of rays that get sent to the light, and because the pdf solved for any given ray can be very small, a single ray directed toward a light can have a very low pdf, which means it can have a very high luminance.

Regardless of how we choose to weight this mixture of pdfs (50-50, 90-10, 99-1, etc.), because these are mixtures of pdfs, this approach will always converge on the truth (only more or less quickly).

That said, you are correct that adding the mixture pdf for the light is adding a whole bunch of fireflies. The solution is one of

1. Be more refined with your pdf mixtures (is an art)
2. Remove fireflies directly by averaging them out with nearby pixels (more of a kludge)
3. Just increase sample count to infinity (impractical)

Getting rid of fireflies is a tremendously deep vein to explore, but out of reach for this book series, I think.

Does that all address your issue?

[trevordblack]

This, by the way, is the reason that so many renderers use shadow rays as the default (as opposed to mixture pdfs)

[Walther]

Thank you for the detailed response! ❤️

because the pdf solved for any given ray can be very small, a single ray directed toward a light can have a very low pdf, which means it can have a very high luminance

This is indeed what I didn't take into account: the final division by pdf_value in the colorization function can make the values large.

[hollasch]

This would be great in the discussions section. Maybe I'll at least add a pointer to this issue — it would be a shame to lose this good commentary.

[hollasch]

Added pointer from discussion #926