Strange StyleboxFlat regression with high corner radius

[EIREXE]

Godot version

3.4 beta, 95162ca

System information

Windows 11, GLES3, Vega 56

Issue description

There is some strange regression that I can't entirely pinpoint, here's the jist of it:

Corner radius seems different from what is shown in the editor to the actual final look in the game: (not a bug actually)

Additionally, if the corner radius is very high it creates this strange line on the top left corner>

This wasn-t present in earlier versions of 3.x at least in 03c41fa

Steps to reproduce

Create a StyleboxFlat with a high corner radius.

Minimal reproduction project

No response

[Calinou]

This wasn-t present in earlier versions of 3.x at least in 03c41fa

#51442 (and its 3.x version #51589) was merged just after that commit. Can you reproduce this if you disable antialiasing in the StyleBoxFlat? For 45-degree beveled angles, you don't really need antialiasing.

cc @Geometror

[EIREXE]

This wasn-t present in earlier versions of 3.x at least in 03c41fa

#51442 (and its 3.x version #51589) was merged just after that commit. Can you reproduce this if you disable antialiasing in the StyleBoxFlat? For 45-degree beveled angles, you don't really need antialiasing.

cc @Geometror

Disabling AA does indeed fix that

[Calinou]

Given how this is a corner case (no pun intended), it may be better to document this in the class reference if we can't fix it without causing undesirable side effects for common use cases (rounded corners).

[EIREXE]

Given how this is a corner case (no pun intended), it may be better to document this in the class reference if we can't fix it without causing undesirable side effects for common use cases (rounded corners).

For sure, that would make sense

[Geometror]

There is some strange regression that I can't entirely pinpoint, here's the jist of it:

Corner radius seems different from what is shown in the editor to the actual final look in the game:

Maybe I overlooked something or misunderstood you, but I think that isn't a regression.
The stylebox preview in the inspector has the same corner radius (bottom left) as the stylebox in the game (58 pixels), but since the prefix stylebox has a greater height, the chamfer does not reach the top left corner. However, the other corner with the radius set to 1024 looks the same because the actual corner radius is clamped to the stylebox's height.

Nevertheless, the strange line is indeed a regression. When modifying the fake AA code in StyleBoxFlat I forgot to change the corner adaption part. I hope I can open a PR soon to fix that (when I figured out how to remove a weird overlap when using a transparent border color).
I managed to eliminate the strange lines for fully opaque borders:

But as Calinou said, this is a corner case in which the fake AA doesn't even work correctly (also present in 3.3; only the outer side of the border has a fake AA gradient, but in this case it has the length 0), so I would recommend to disable it. Here's an idea: A shear property in StyleBoxFlat would provide an option to achieve this type of effect in a better way (more flexible, less "hacky").

[EIREXE]

There is some strange regression that I can't entirely pinpoint, here's the jist of it:
Corner radius seems different from what is shown in the editor to the actual final look in the game:

Maybe I overlooked something or misunderstood you, but I think that isn't a regression.
The stylebox preview in the inspector has the same corner radius (bottom left) as the stylebox in the game (58 pixels), but since the prefix stylebox has a greater height, the chamfer does not reach the top left corner. However, the other corner with the radius set to 1024 looks the same because the actual corner radius is clamped to the stylebox's height.

Nevertheless, the strange line is indeed a regression. When modifying the fake AA code in StyleBoxFlat I forgot to change the corner adaption part. I hope I can open a PR soon to fix that (when I figured out how to remove a weird overlap when using a transparent border color).
I managed to eliminate the strange lines for fully opaque borders:

But as Calinou said, this is a corner case in which the fake AA doesn't even work correctly (also present in 3.3; only the outer side of the border has a fake AA gradient, but in this case it has the length 0), so I would recommend to disable it. Here's an idea: A shear property in StyleBoxFlat would provide an option to achieve this type of effect in a better way (more flexible, less "hacky").

I see, this makes sense

[Calinou]

A skew option sounds good to me – feel free to open a proposal for it 🙂

Edit: Proposal opened: godotengine/godot-proposals#3301