Mixing Text and non-Text content in UIs

[viridia]

What problem does this solve or what need does it fill?

Currently Bevy UI text nodes are segregated from other kinds of UI nodes - text spans can only live inside of a Text component. I understand the reasons for this - it makes the layout engine simpler, since things like word-wrap calculations only apply to Text components and not UI components generally.

However, there are some use cases where you would like to mix text and non-text content in a single context. A classic example is a button that has both an icon and a text caption. Yes, you can make a button that has a dedicated "slot" for an icon, but only if the button component is specifically built for this. The same logic holds for menu items, dialog titles, basically anything that accepts a text string - it's better if these can accept a variety of renderable elements.

What solution would you like?

I'm not sure what's the right solution. We all know what the web does, but that's an ambitious target, Bevy is never going to have parity with Firefox when it comes to layout. The two most interesting web cases are flex and inline - that is, either the text and images are laid out in a row with gaps between them, or they are laid out fluidly with word-wrap.

What alternative(s) have you considered?

The flex use case can be done by wrapping everything in an extra UI component. The word-wrap case I'm not sure how you would do.

Additional context

This is likely to be relevant in the context of future Bevy UI / Scene Graph assets.

[ickshonpe]

Text rendering just draws glyph sprites from the list of glyphs in TextLayoutInfo. For a simple implementation, maybe there is some way we could add the image to the list as a really big glyph and it would just work.

I think once #9415 and #9923 are merged the glyph_brush_layout text wrapping implementation should work without any more problems (fingers crossed). There are a still number of other critical open issues with Text atm that won't be resolved by the cosmic text adoption: #9278 (should be fixed by #9471), #7714, #6967, and the limitations with font size, texture atlases and scaling etc (I think there is an issue somewhere but can't find it).

It is already possible (albeit a bit inefficiently) to mix text and images. The idea is that you set FlexWrap::Wrap on the parent node, split up your text into individual words and then spawn a separate TextBundle for each word. Then you can insert an image bundle in between the text bundles and it will be laid out along with the text as though it is just another word.

[viridia]

I should mention that my real motivation for wanting this is not what I wrote in the OP - I was trying to make the best possible argument rather than accurately describing my needs.

The real use case I'm concerned about is UI templating: that is, making it easy to write "React-like" components. In HTML, text and non-text elements can be freely intermixed, and the browser just does the right thing, depending on whether the containing element has an "inline", "block" or "flex" layout. Most coders find this fairly natural.

However, in current Bevy, a templating language for UI would require the user add an extra wrapper around text spans in their templates. This means that the templating language now has to understand two different container contexts, each having different constraints on what kind of children they can contain. "Text" objects can only contain text spans, and "Node" or "Element" objects cannot contain text spans, and can only contain block-level elements. This is not much of a hardship for templates that are embedded within Rust source code using macros, but it adds complexity to UI layouts which are parsed via asset loaders - it means that now that your loader needs not just a type system, but two type systems, to represent a Bevy UI layout.

You could of course create a templating language that has both text spans and blocks, and then partition the result into inline and non-inline segments, automatically adding the wrappers as needed, but if you are going to do all that work, you might as well just add it to the UI framework itself, so that everyone can reap the benefits and not just UIs that are generated from templates.