Blue light turns violett in Camera View

[davidAnPrintell]

Hi!

I faced some unexpected color behavior, if there is a blue light source indoor.
Im worried that there will be other color grading issues.

Expected Behavior

The camera view renders the expected color, like other AR cameras or the native camera.

Current Behavior

The camera view renders a violet color.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. Run a sample Project (tested with Augmented Images and environment-lights) on an android device ( tested on an Samsung Galaxy S8 and S20).
2. Turn a blue light source on

Context

Couldnt reproduce the issue with archived Sceneform 1.15.0.
Expected Behavior Screenshot is the sample AR-Core Augmented Images project (OpenGL Rendering)

Current Behavior testet with

• SDK version: Android-Version 12

[ThomasGorisse]

Hi,

Can you try to disable light estimation?

[davidAnPrintell]

Thanks for your quick reply.
I also had this issue with all the available configs in the sample light estimation app.

My Configs:

config.updateMode = Config.UpdateMode.LATEST_CAMERA_IMAGE
config.focusMode = Config.FocusMode.AUTO
config.planeFindingMode = Config.PlaneFindingMode.DISABLED
config.instantPlacementMode = Config.InstantPlacementMode.DISABLED
config.augmentedFaceMode = Config.AugmentedFaceMode.DISABLED
config.lightEstimationMode = Config.LightEstimationMode.DISABLED
config.augmentedImageDatabase = imageDatabase
session.configure(config)
arSceneView?.setSession(session)

[ThomasGorisse]

Please also call arSceneView.lightEstimationConfig = LightEstimationConfig.DISABLED

[davidAnPrintell]

To better reproduce the issue, i just filmed some blue images from google images. Result:

First Img is my app with following configs:

                    config.updateMode = Config.UpdateMode.LATEST_CAMERA_IMAGE
                    config.focusMode = Config.FocusMode.AUTO;
                    config.planeFindingMode = Config.PlaneFindingMode.DISABLED
                    config.augmentedImageDatabase = imageDatabase
                    session.configure(config)
                    arSceneView?.lightEstimationConfig = LightEstimationConfig.DISABLED
                    arSceneView?.setSession(session)

Second Img is https://github.com/SceneView/sceneform-android/tree/master/samples/augmented-images

Thrid Img is https://github.com/google-ar/arcore-android-sdk/tree/master/samples/augmented_image_java

[ThomasGorisse]

@RGregat @grassydragon Do you think it could be a Filmic color mode issue?

[grassydragon]

Since the inverseTonemapSRGB function is used in the camera material the camera image can be also affected by the color issues. According to Romain, the inverse tonemapping operation should be exact for the Filmic tonemapping. However, something could have changed in Filament or Sceneform because the problem with the ViewRenderable color has been reported recently: #252

[ThomasGorisse]

@davidAnPrintell Would you mind creating a discussion (not an issue) on the Filament repo and linking/mentioning this issue?

[ThomasGorisse]

BTW, thanks for your detailed issue.

[davidAnPrintell]

Link for Diskussion: google/filament#5136

[romainguy]

It's a known side effect of inverse tone mapping. You are seeing a hue shift due to out of gamut colors. We already have an open bug for this and it's not entirely trivial to fix. In the meantime make sure you are using the FILMIC tone mapper.

[ThomasGorisse]

Thanks.

I confirm that the workaround is still applied here:
https://github.com/SceneView/sceneform-android/blob/master/core/src/main/java/com/google/ar/sceneform/SceneView.java#L386-L393

[grassydragon]

If I use the deprecated toneMapping(ColorGrading.ToneMapping.FILMIC) here, the colors are rendered as expected:

sceneform-android/core/src/main/java/com/google/ar/sceneform/SceneView.java

Line 391 in 5b5ae1f

.toneMapper(new ToneMapper.Filmic())

[romainguy]

Yeah I need to find a good way to offer inverse tone mapping based on the selected tone mapper in ColorGrading.

[ThomasGorisse]

@davidAnPrintell Can you try calling this?

arSceneView.renderer!!.filamentView.colorGrading =
            ColorGrading.Builder().toneMapping(ColorGrading.ToneMapping.FILMIC)
                .build(EngineInstance.getEngine().filamentEngine)

[davidAnPrintell]

Thanks a lot. Its solves the issue

[davidAnPrintell]

I'll close this issue, since its a known filament bug.
Thanks for the help

[ThomasGorisse]

@grassydragon @RGregat Did we merged it?

[grassydragon]

@grassydragon @RGregat Did we merged it?

No, should we merge it and wait for the related Filament issue to be resolved?

[romainguy]

Note that toneMapping(ColorGrading.ToneMapping.FILMIC) will not go anywhere. It's marked deprecated because I'd like to move our users to " better" tone mappers, but we won't remove it from our APIs. You can safely use it.

Ideally I'd like everyone to use the generic tone mapper (terrible name, great tone mapper) with luminance scaling turned on.

[romainguy]

Except you have luminance scaling off :p But luminance scaling would need to be invertible, that's another story.

[ThomasGorisse]

Except you have luminance scaling off :p But luminance scaling would need to be invertible, that's another story.

Sorry I'm tired. Answer edited

[ThomasGorisse]

So we can't use the generic tone mapping even with luminanceScaling(true) to solve the blue issue?

[romainguy]

The problem is how inverseTonemap() works. It was designed for FILMIC, and will only be somewhat of an approximation with other tone mappers.

[ThomasGorisse]

@grassydragon I'm not good enough in fragment shader. Do you think there could be any other way than using the inverse tonemap for the camera stream here:

sceneform-android/core/assets-sources/materials/sceneform_camera_material.mat

Line 23 in 03409ab

material.baseColor.rgb = inverseTonemapSRGB(color.rgb);

[grassydragon]

I think only if we use the linear tone mapper or there is a way to disable the tone mapping for particular renderables (the camera stream and view renderables).

[romainguy]

Just use FILMIC and inverseTonemapSRGB() :)

[ThomasGorisse]

@grassydragon Can you PR it?
// Change the ToneMapper to FILMIC to avoid some over saturated
// colors, for example material orange 500.

renderer.getFilamentView().setColorGrading(
        new ColorGrading
                .Builder()
                .toneMapping(ColorGrading.ToneMapping.FILMIC)
                .build(EngineInstance.getEngine().getFilamentEngine())
);

[ThomasGorisse]

Surely not related but this blue issue made me thought about some color issues I had during lights estimation work.
I always though that it was due to ARCore more recognizing blue than other colors but could it be related to some Filmic color accentuation differences?

As you can see, red and green are less present in the reflections than other colors (Maybe it's only main light estimation issue but I try my luck)

[romainguy]

Seems like a light estimation issue. Tone mapping would not be responsible for removing green from the lighting.

[ThomasGorisse]

I just made the test again and confirm it's an ARCore "issue".
@devbridie, any idea why would red and green be less recognized?
On the right column, the ARCore estimated environment reflections applied to the Skybox. Main light color also applied.