[core] Line annotation disappears and/or clips to tile boundaries from zoom 3 onwards

[brunoabinader]

In macosapp example, press ⌘L and zoom in until a piece of the example annotation disappears:

I suspect #5514 could be the cause - need to investigate/bisect further to pinpoint the issue. From my experiments, #5517 does not fix the issue.

👀 @yhahn @1ec5 @mourner

[brunoabinader]

Also reproducible in glfw-app:

Here you can see also an interesting artifact: the line annotations are somehow clipped when crossing tile boundaries:

[brunoabinader]

Fix added in #5517.