Improve drop shadow effect accuracy #2523
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This applies a constant scale factor to the distance and softness (aka blur or radius) properties on drop shadow effects so that the end result more closely matches what is shown in After Effects and on other platforms. See airbnb/lottie-ios#2175 for similar changes made to the iOS Lottie implementation.
allenchen1154
commented
Jul 24, 2024
lottie/src/main/java/com/airbnb/lottie/parser/DropShadowEffectParser.java
Outdated
Show resolved
Hide resolved
gpeal
reviewed
Jul 24, 2024
There was a problem hiding this comment.
Thanks for doing this investigation. How confident are you in these values? It might help to make an animation that has a drop shadow on a circle and a larger concentric stroked circle that represents where the shadow should end to ensure that it is correct. Have you tried anything like that?
lottie/src/main/java/com/airbnb/lottie/parser/DropShadowEffectParser.java
Outdated
Show resolved
Hide resolved
lottie/src/main/java/com/airbnb/lottie/animation/keyframe/DropShadowKeyframeAnimation.java
Outdated
Show resolved
Hide resolved
lottie/src/main/java/com/airbnb/lottie/animation/keyframe/DropShadowKeyframeAnimation.java
Outdated
Show resolved
Hide resolved
|
@gpeal Good idea, for the distance scale factor we could use a animation that has a shadow with no blur, and use a reference point as you suggested that the shadow should touch at a predetermined distance. For the softness scaling we may need something like a paint swatch in the background? |
@allenchen1154 Yeah, something like that. Are you familiar enough with After Effects to make something like that? |
|
I'm not, but I'll work with a motion designer to create the samples! |
Passes the `parentAlpha` from the content and layer to the drop shadow so that it can multiply by its opacity value. This removes the `isDirty` check which was skipping the `applyTo()` logic if none of the drop shadow's values had changed.
allenchen1154
changed the title
|
I've added the test files we discussed and I'm reasonably confident in the scaling factors. |
gpeal
approved these changes
Jul 30, 2024
lottie/src/main/java/com/airbnb/lottie/animation/keyframe/DropShadowKeyframeAnimation.java
Show resolved
Hide resolved
|
Actually, after looking through the snapshot tests, are a few others that look a bit off/clipped as well. Do you want to take a look? I can send you any of the prod animations from the snapshot tests if you want any of them. web web web (lottiefiles/the_final_frontier) |
gpeal
pushed a commit
that referenced
this pull request
Oct 19, 2024
## High-level summary This PR introduces a large change to how drop shadows are rendered, introducing an `applyShadowsToLayers` flag which, by analogy to `applyOpacitiesToLayers`, allows layers to be treated as a whole for the purposes of drop shadows, improving the accuracy and bringing lottie-android in line with other renderers (lottie-web and lottie-ios). Several different codepaths for different hardware/software combinations are introduced to ensure the fastest rendering available, even on legacy devices. The calculation of shadow direction with respect to transforms is improved so that the output matches lottie-web and lottie-ios. Image layers now cast shadows correctly thanks to a workaround to device-specific issues when combining `Paint.setShadowLayer()` and bitmap rendering. Even in non-`applyShadowsToLayers` mode, correctness is improved by allowing the shadow-to-be-applied to propagate in a similar way as alpha. This allows some amount of visual fidelity to be recovered for animations or environments where enabling `applyShadowsToLayers` is not possible. A number of issues that caused incorrect rendering in some other cases have been fixed. ## Background ### Drop shadows in Lottie Lottie specifies drop shadows as a tuple of (angle, distance, radius, color, alpha), with each element being animatable. The consensus behavior for the rendering of a layer with a drop shadow, which seems to be mostly respected in lottie-web and lottie-ios, seems to be: 1. Evaluate the values at the current frame for angle (`theta`), distance (`d`), radius (`r`), color with alpha (`C`). 2. Apply the layer transform and render the layer normally to a surface `So` (original layer). 3. Copy `So` to new surface `Ss` (shadow). 4. Apply a gaussian blur of radius `r' = c * r` to `Ss`, where `c` is some platform-specific constant intended to normalize blur implementations between platforms. (Ours is 0.33, lottie-web's is 0.25; see #2541). 5. Tint `Ss` with the color and combine the alpha by applying the following for each pixel `P`: `P.rgb = C.rgb * P.a; P.a = C.a * P.a`. 6. Now the shadow is ready on `Ss`, and needs to be drawn into its final position. 7. Convert from polar coordinates `theta` and `d` into `dx` and `dy`, with the 0 position at 12 o'clock: `dx = d * cos(theta - pi/2); dy = d*sin(theta - pi/2)`. 8. Draw `Ss` onto `Si` (intermediate surface) with a translation of `(dx, dy)`. 9. Draw `So` (original layer) onto `Si` with identity transform. 10. Compose `Si` into the framebuffer using the layer's alpha and blend mode. Some non-obvious consequences of the definition above: - The angle, distance, and radius are relative to the layer post-transform, not pre-transform. That is, rotating the layer (via its transform) still keeps the same screen-space direction of the shadow, and scaling the layer (via its transform) still keeps the same screen-space shadow blur radius. - The drop shadow is not based on any derived outline, so a layer's drop shadow can be seen through its non-fully-opaque pixels. At the same time, reducing the alpha of a pixel in a layer reduces its alpha in the drop shadow. - A layer's shadow and the layer do not blend on top of each other on the final canvas in case the layer has a blend mode or alpha. Instead, the shadow and the layer are alpha-blended with each other, and the result is then composited onto the canvas. - In case the layer has a normal blend mode, this is equivalent to alpha-blending the layer's shadow and then the shadow onto the canvas separately. ### Drop shadows in lottie-android currently lottie-android's current implementation of drop shadows differs in important ways: 1. **Shadows are applied per-shape.** This means that a case like a shape with both fill and stroke has incorrect shadows, since both the fill and the stroke render a separate shadow on top of each other. 2. **Precomp layer shadows are ignored.** This means that a precomp cannot cause any of its child shapes to cast a shadow. This is a consequence of the current implementation of (1). 3. **Image layers do not render correct shadows,** due to the minefield that is the support matrix (or in Android's case, a more apt name would be a support tensor) of Android's graphics stack - `setShadowLayer()` simply doesn't work for images consistently. (See the last image in #2523 (comment).) ## Contributions of this PR This PR introduces the following improvements and additions. 1. **Move the drop shadow model from individual content elements to layers,** and add some missing keypath callbacks. This is a prerequisite for handling drop shadows on a layer level. 2. **An `OffscreenLayer` implementation,** which serves as an abstraction that can replace `canvas.saveLayer()` for off-screen rendering and composition onto the final bitmap, but with the important distinction that it can also handle drop shadows, and possibly use hardware-accelerated `RenderNode`s and `RenderEffects` where available. - To use an `OffscreenLayer`, call its `.start()` method with a parent canvas and a `ComposeOp`, and draw on the *returned canvas.* Once finished, call `OffscreenLayer.finish()` to compose everything from the returned canvas to the parent canvas, applying alpha, blend mode, drop shadows, and color filters. - `OffscreenLayer` makes a dynamic decision on what to use for rendering - a no-op, forward to `.saveLayer()`, a HW-accelerated `RenderNode`, or a software bitmap, depending on the requested `ComposeOp` and hardware/SDK support. - The hope is that `OffscreenLayer` becomes a useful abstraction that can be extended to e.g. support hardware blurs, multiple drop shadows, or to support mattes in a hardware-accelerated fashion where possible. 3. **The `applyShadowsToLayers` flag** which, by analogy to `applyOpacityToLayers`, turns on a more accurate mode that implements the drop shadow algorithm described above. - `OffscreenLayer` is used to apply alpha if `applyOpacityToLayers` is enabled, and to apply shadows if `applyShadowsToLayers` is enabled. The cost is paid only once if both alpha and drop shadows are present on a layer. - Not all `saveLayer()` calls in the code have been rewritten to use `OffscreenLayer` - the blast radius is minimized. `OffscreenLayer` is presently used only to apply alpha and drop shadows, and blend mode and color filters are still applied in `BaseLayer` using `saveLayer()` directly. 4. **More accurate shadow transformations.** Previously, the angle and distance were pre-transform, and only the radius was post-transform (contrary to step (2) of the algorithm). We correct this to match other renderers. 5. **More complete shadow handling even when `applyShadowsToLayers` is `false`:** we plumb the shadow through `.draw()` and `drawLayer()` calls similarly to alpha, and this allows us to render per-shape shadows on children of composition layers too. 6. ***Workaround for drop shadows on image layers.** - The workaround relies on `OffscreenLayer` as well, and image layers now render shadows properly in all cases. 7. **Fixes to a few subtle issues** causing incorrect rendering in other cases. (will be marked using PR comments, I might have forgotten some) ## Open questions * **Should `applyShadowsToLayers` be `true` by default?** Some codepaths, such as when rendering purely via software, can be slow if shadow-casting layers are exceedingly large. But, the performance is still acceptable, and in the vast majority of cases everything is quite snappy. * **Have I introduced any regressions?** The snapshot tests should answer this. * **How does this perform on older devices?** `applyShadowsToLayers` plus an old device should trigger the purely-software shadow rendering mode. Simulating this in condition manually yields accurate results, and the performance seems surprisingly good, but it's unclear what will happen on a lower-end phone. There's also always the possibility of some device subtlety being missed. I don't have access to an older Android device. ## Testcases These files now match between lottie-web and lottie-android: [drop_shadow_comparator.json](https://github.com/user-attachments/files/16997070/drop_shadow_comparator.json) [simple_shadow_casters_ll2.json](https://github.com/user-attachments/files/16997084/simple_shadow_casters_ll2.json) The files from this earlier PR still all render the same: #2523, with the exception of the fix for image layer bug, which fixes the rendering of the Map icon as mentioned in the comment of that PR. This file has been used as a perf stress test with many <255 opacity precomps, some stacked inside each other, that must all be blended separately: [precomp_opacity_killer.json](https://github.com/user-attachments/files/16997261/precomp_opacity_killer.json)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes make several improvements to how drop shadow effects are displayed:
Paint.setShadowLayer()to more closely match how they are displayed in After Effects. See Improve conversion from After Effects shadow softness value toCALayer.shadowRadiusvalue lottie-ios#2175 for similar changes on iOS.Distance test file:
Softness test file:
Example of Image layer shadow bug, right side is how it should look (capture from After Effects):
