Extensible blocks

[mwort]

Is your feature request related to a problem?

The blocks provided by coderedcms are great but they are difficult to extend consistently. E.g. adding a code block to the CONTENT_STREAMBLOCKS is currently only possible by overriding the body in all of the page models where this is required.

Describe the solution you'd like

#43 hinted on a solution that has so far not been implemented (correct me if I'm wrong). It would probably require all *_STREAMBLOCK globals in blocks/__init__.py to be converted to functions that will assemble the list of blocks depending on settings during the model construction. The setting could be a dictionary with entries pointing to a module with global list variables, e.g. something like this:

ADDITIONAL_BLOCKS = 'website.blocks'  # eg. contains CONTENT_STREAMBLOCKS

Describe alternatives you've considered

See example with body above.

Additional context

I can give this a go and push a PR if you think this would be the right way to do this.

[vsalvino]

I agree - we tried to make these blocks more dynamic at first, but it creates a lot of problems with migrations, because any change to a streamfield must trigger a django migration.

I would be interested to see what your solution looks like. Can't guarantee it will be merged but I would definitely give it a thorough review. Up until now we have not come up with a good way to easily add more blocks other than overriding the body field per model, or adding creating an abstract model and then inheriting from that everywhere.

[ayushin]

@mwort I'd like to see that PR :-)

[ayushin]

@vsalvino I completely understand the concern, however it is better to give that flexibility to a developer (perhaps with a warning) and leave the choice up to them if they know what they are doing.

I imagine a good approach would be to have CODEREDCMS_EXTRA_CONTENT_BLOCKS or similar in settings.

[ayushin]

Or something like this?

def register_layout_streamblock(name):
    def inner(block):
        if name not in get_layout_streamblocks():
            get_layout_streamblocks().append((name, block(CONTENT_STREAMBLOCKS)))
        return block
    return inner

[vsalvino]

Yes - what you have is exactly right, however the problem is that it breaks Django migrations (it will created endless migrations) due to how the wagtail streamfield blocks are actually part of the database migrations - so they cannot be dynamic. This is the number 1 issue we have struggled with in trying to build coderedcms to be dynamic in any fashion.

So further research into the Wagtail streamfield, and Django migrations is required.

[ayushin]

So I am hitting this issue very hard three-fold:

1. Adding own blocks - can be worked around by "creative" re-using existing blocks fields (i.e. subtitle|safe for some extra html field) and using custom extra templates

2. Integrating https://pypi.org/project/wagtailstreamforms/ (perhaps an idea anyway to have that in coderedcms by default?) - i need this included in CONTENT_STREAMBLOCKS to be useful.

3. And the worst by adding an extra language with wagtail-modeltranslation - as it creates migrations in both coderedcms and wagtailstreamforms every time a new language is added to LANGUAGES

@vsalvino

So far the cleanest solution I see is keeping the abstract models in coderedcms and moving all the managed models into project_template together with the settings such as HTML_STREAMBLOCKS from blocks/__init__.py

This would keep the blocks definitions inside of the coderedcms but all the migrations will be in the project and would allow clean and controlled upgrages.

[vsalvino]

Regarding your list:

1. To add your own blocks, override the "body" field on your models. You can build your own streamfield from our various blocks as defined in coderedcms/blocks/__init__.py, and mix in your own custom blocks as well.

2. Stream forms are already integrated: https://docs.coderedcorp.com/cms/stable/features/page_types/stream_forms.html

3. What you are describing is an issue which can be temporarily worked around by being "careful" with migrations - but this is by far our biggest and hardest bug right now. See

• Make built-in models abstract #56
• Official theme support #44

[ayushin]

1. Yes, but I would like to to insert my blocks inside of bootstrap column (two levels down from the body) - do I miss something?

2. Talking about https://github.com/labd/wagtailstreamforms - you can add these forms to any page inside of any streamfield, is it same?

3. Yes....

[vsalvino]

For custom blocks, you'd have to define the entire streamfield as so (see 'my_custom_block'). And then set that as the body field of your model.

from coderedcms.blocks import *
from 3rd_party.blocks import MyBlock


# Re-implement a mega-streamblock from codered defaults
MY_STREAMBLOCKS = [
    ('hero', HeroBlock([
        ('row', GridBlock(CONTENT_STREAMBLOCKS + [('my_custom_block', MyBlock())])),
        ('cardgrid', CardGridBlock([
            ('card', CardBlock()),
        ])),
        ('html', blocks.RawHTMLBlock(icon='code', classname='monospace', label=_('HTML'))),
    ])),
    ('row', GridBlock(CONTENT_STREAMBLOCKS)),
    ('cardgrid', CardGridBlock([
        ('card', CardBlock()),
    ])),
    ('html', blocks.RawHTMLBlock(icon='code', classname='monospace', label=_('HTML'))),
]


class WebPage(CoderedWebPage):
    body = StreamField(MY_STREAMBLOCKS, null=True, blank=True)

[ayushin]

This worked. Combined with ./manage.py sync_translation_fields I'm now finally back to the mainstream coderedcms package!

[ayushin]

OK, so what's the latest recommendation to add custom STREAMBLOCKS to non-pages, like snippets and content walls?

I'm using https://github.com/labd/wagtailstreamforms with great results and would like to embed them into content walls too.

@vsalvino Is it (wagtailstreamforms) something to replace the form pages with?

[ayushin]

I am trying to inherit ContentWall with ContentPopup(ContentWall) and then override content_walls on all my pages, but I got stuck with this:

django.core.exceptions.FieldError: Local field 'content' in class 'ContentPopup' clashes with field of the same name from base class 'ContentWall'.

Any ideas of another solution to be able to add custom streamfield blocks to content walls? 🙇‍♂️

[ayushin]

OK, so revisiting this, will open a new issue for wagtailstreamforms, I've been banging my head against the wall for quite some time but any mechanism with EXTRA_*_STREAMFIELD_BLOCKS would lead to unmanageable migrations in each coderedcms installation and breakage on subsequent upgrade.

I'm thinking the solution could be moving al the CODEREDCMS streamfield migrations to a separate app or website, haven't given it a deeper think yet, just occurred to me as I type it.

I'm really struggling with it because I'd like to use wagtailstreamforms in the snippets (reusable content and content walls) and there's no good solution for it.

#368

[vsalvino]

If you are trying to add a form to content wall... I would recommend creating the form page separately, then use the page preview block within the content wall, pointing to your new form page.

Yes, you are right about adding custom block types, it does create migration problems. The solution is that coderedcms needs to refactor its concrete models containing a streamfield to be abstract models... this is a design flaw that will unfortunately require a major update to sort out (#56)

[awhileback]

Hello everyone! Thanks for the great boilerplate for Wagtail, vsalvino and crew. Google sent me here when looking for how to integrate a third party block package into the StructBlock provided with coderedcms as BaseBlock.

I'm thinking the solution could be moving al the CODEREDCMS streamfield migrations to a separate app or website, haven't given it a deeper think yet, just occurred to me as I type it.

That's precisely what I did when working through similar issues (mainly with integrating third party blocks), I could't think of a better way. Blocks without formatting can just be included (such as alternate editors, markdown, etc) but when I wanted to embed uploaded media, as mentioned in #15, the only way I can think of to include such a block with the advsettings class attached is to create a new StructBlock from the included BaseBlock that includes advsettings, and then add the third party media block to the StructBlock.

In any case all of this will require a migration, so breaking coderedcms out into an app seemed like the most serviceable way to do so.

A lot of this seems to be inherent limitations of wagtail, I suppose. There's not a trivial way to add third party integration to the admin like base-Django.

FWIW, if other people also stumble here from google and want a way to do this, here's how I did it...

The coderedcms advsettings (custom css options) are included from a StructBlock called BaseBlock in blocks/base_blocks.py, so that is the one you want to sub-class for other struct blocks that you want the custom css to attach to.

In my case, wagtailmedia like so...

from .base_blocks import BaseBlock, LocalMediaValues
from wagtailmedia.blocks import AbstractMediaChooserBlock

class LocalMediaBlock(BaseBlock):
    """
    Embed uploaded audio/videos.
    """
    local_media = AbstractMediaChooserBlock(
        label=_('Media'),
        help_text=_('Embed an uploaded audio/video file.')
    )
    
    class Meta:
        template = 'coderedcms/blocks/local_media_block.html'
        icon = 'fa-audio'
        label = _('Embed Local Media')
        value_class = LocalMediaValues

The value_class must be specified because by default, the data of child objects of StructBlocks are not present in StreamField output, as explained in the wagtail docs about custom StreamField blocks

That class, in my case LocalMediaValues looks like... (put it where you like, I put it in with the other BaseBlocks)...

from wagtail.core import blocks

class LocalMediaValues(blocks.StructValue):
    def url(self):
        media_url = self.get('local_media')
        return media_url.file

    def type(self):
        media_type = self('local_media')
        return media_type.type

    def duration(self):
        media_duration = self.get('local_media')
        if media_duration.duration:
            return media_duration.duration
        else:
            pass

    def thumb(self):
        media_thumb = self.get('local_media')
        if media_thumb.thumbnail:
            return media_thumb.thumbnail
        else:
            pass

    def width(self):
        media_width = self.get('local_media')
        if media_width.width:
            return media_width.width
        else:
           pass

    def height(self):
        media_height = self.get('local_media')
        if media_height.height:
            return media_height.height
        else:
            pass

    def tags(self):
        media_tags = self.get('local_media')
        if media_tags.tags:
            return media_tags.tags
        else:
           pass

There's probably a neater way to iterate over the model and generate a context list here, but this works. You need to define a method for each model field for the class you included in the StructBlock's QuerySet that you need access to. You can 'get' the field you created in your StructBlock, and then return the sub-field from the child block's model for the value.

You can see what fields are going to be available in the response object from the StructBlock's QuerySet in the wagtailmedia model. The only thing missing here is a method to get at the filename's extension for the html tag defining the controls.

A template tag fixes that, in coderedcms/templatetags/coderedcms_tags.py...

@register.filter
def extension(value):
    extension = value.split(".")[-1]
    if extension:
        return extension
    else:
        pass

Then in a block template, you can access your values like so...

{% extends "coderedcms/blocks/base_block.html" %}
{% load website_tags %}
{% block block_render %}
{{ value.local_media.extension }}
{% if value.local_media.type == 'audio' %}
<{{ value.local_media.type }} controls>
        <source src="{{ value.local_media.url }}" type="{{ value.local_media.type }}/{{ value.local_media.url|extension }}">
</{{ value.local_media.type }}>
{% elif value.local_media.type == 'video' %}
<{{ value.local_media.type }} width="{{ value.local_media.width }}" height="{{ value.local_media.height }}" controls>
        <source src="{{ value.local_media.url }}" type="{{ value.local_media.type }}/{{ value.local_media.url|extension }}" />
</{{ value.local_media.type }}>
{% endif %}
{% endblock %}

You'll have to season the HTML to taste, obviously. Whatever functions you specified in your value_class should work here as {{ value.your_structblockfield.yourfunctionname }}. Extending the coderedcms/templates/blocks/base_block.html template will add the optional custom CSS class/id/whatever to a prepended div.

From here, all you have to do is include the option in the coderedcms StreamField init lists. I'm a few versions back but unless that's changed, they're in the coderedcms/blocks/__init__.py file.

from .content_blocks import (  # noqa
....
LocalMediaBlock # (the name of your Structblock class)
...
)

HTML_STREAMBLOCKS = [
...
('local_media', LocalMediaBlock(icon='fa-music')), 
]

All of this has triggered a migration for the field you created in the new StructBlock, so you will have to makemigrations/migrate and you are now maintaining a fork of coderedcms. As Ayushin mentioned above I simply broke out the installation into an app to avoid issues. It should also be noted that absent a proper deconstruct method for deleting this thing you've created, you cannot trivially add/remove custom StructBlocks when there are pages depending on them without running into migration issues (the wagtail docs cover this as well). It would be wise to ensure you have everything working the way you want on SQLite and not screw around with this stuff on a live database, lest you find yourself unable to migrate, make migrations, or revert migrations due to a StreamField error about a class or field being missing because you decided you didn't like the name.

[vsalvino]

Best solution for now would be to create an official doc on how to make your own streamfield (see my comment above).