Enterprise-wide use cases for the DSL

[simonbrowndotje]

I'm working for the same organisation as @chrisjschultz. Recently I've come up with an alternative approach to address our Enterprise-wide use case, which doesn't involve forking the DSL parser, or using !include to build the Enterprise-wide workspace. Instead, separate workspace DSL files are parsed individually, and then merged. The benefit is that our individual system workspaces are compatible with standard Structurizr products. They're also more conventional, with everything in one file, not spread out over DSL fragments.

I pitched the idea to Chris yesterday, and am keen to get your feedback too. You can see it here: https://github.com/iambic69/structurizr-tools

There are a couple of specific points I'd like your thoughts on:

• Whether you'd consider adding this functionality to WorkspaceUtils at some point?
• Whether this could be a good use of the, now deprecated, 'enterprise' syntax? I.e. using element Location = External, rather than an 'External' tag. This is something Chris suggested when I showed him yesterday.

@iambic69 @chrisjschultz

Thanks for the info. No need to add this to the WorkspaceUtils, but I will happily create a page about enterprise-wide modelling and add a link to your repo. I have my own approach that I've showing at conferences over the past few months too (https://github.com/structurizr/examples/tree/main/dsl/system-catalog ... it similarly parses DSL files owned by each team, albeit with a central catalog to avoid reconciliation/merging issues), so it would be nice to centralise the various approaches to using the DSL at scale, because they all have trade-offs.

The "enterprise" concept isn't fit for purpose (many teams want to model multiple enterprises), so I've added support for nested groups instead. See https://github.com/structurizr/dsl/tree/master/docs/cookbook/groups#nested-groups for an overview.

[iambic69]

Thanks for your speedy and considered reply, Simon.

Pulling together something describing the various approaches being used to scale Structurizr DSL would be very valuable, and yes, there are always trade-offs. I like the system catalogue / workspace extends approach you've described. I can see it'll work well where you have common, shared model elements higher in the hierarchy, and add more detail as you descend the hierarchy.

In our case, we want to model relationships across our models, often at the more detailed container-to-container level, so I can see we'd end up pushing more and more detail to the root of the hierarchy; which mirrors the classic problem with OO single inheritance, of course.

The trade-off with my merge() approach is repetition of external element declarations. Inspired by a comment Chris made, I'm toying with a refinement that might help: having the maintainer of a workspace put the external declarations they want to 'export' for use in other workspaces into a sibling DSL fragment. Then, rather than authors of other workspaces having to individually declare the external elements they depend on, they can simply !include the fragment. This is analogous to using a #include in C to pull in a header file. I'll add this to my structurizr-tools repo, so it's clearer.

Regarding enterprise, I just liked the cheeky idea of repurposing (and perhaps renaming) this feature as a neat way to indicate what was definitive (owned by the workspace) and what was declarative (a reference to something to be found in another workspace), instead of using a tag. The concept of the element's location (internal/external) would transpose nicely to my alternate usage: internal/external to the workspace. I can quite understand that you probably wouldn't want to do that. ;)

I can either stick with my External tag, or as you suggest, use a suitably-named group.

[simonbrowndotje]

In our case, we want to model relationships across our models, often at the more detailed container-to-container level

I'm curious ... why do you want to do this? It's not something I recommend, and I actually wrote a Twitter thread about this exact topic yesterday.

[iambic69]

I don't want to speak out of turn as @chrisjschultz has primarily driven the modelling effort using Structurizr within the organisation, but here is my take on some of the reasons why:

• The organisation has a design team that works across all of the application systems being modelled, rather than specific teams focussed on each system
• The Enterprise-wide use case was an early focus, and included modelling physical aspects such as network communication taking place between containers in different network zones
• We hadn't seen an argument presented against modelling at this level of fidelity

Your thoughts, Chris...?

[chrisjschultz]

I didn't get a chance to reply over the weekend.

I have had several attempts over the years to build an enduring view of systems and the integration between them.

Enterprise Architect is used quite a lot in Wellington, we even managed to get a fairly large team of BAs and designers to adopt source control in an attempt to build a versioned model of multiple systems and maintain an accurate view over time.

This failed for lots of reasons:

• UML doesn't tend to be used consistently as it is wide ranging
• The model becomes hugely complex as so many "things" are linked to one another, and it becomes cumbersome under its own weight
• People get sick of trying to maintain the model and diagrams, and end up just using UML to draw pictures

The benefits of C4/Structurizr are that its a simple, constrained model aimed at documenting a software system. People can grasp the concepts quickly and get on with doing what the need.

When I was building up the initial model here with my couple of mods, I was able to generate components directly off source code, which meant I could flesh out quite a lot of our systems quite quickly. Developers were then able to pick up and build collaborations of containers/components across a number of our systems into a single view. We rarely modify a single system in any of our projects, so having views that describe collaborations across systems is necessary.

The environment we work in at the moment is a critical utility and we have dedicated "high security" realm. Understanding what passes the boundary between low and high is important. Having a model that can represent this level of detail is very useful for us.

Structurizr proved very useful as:

• we are able to properly branch and merge our model for each project and maintain a correct view of what production is
• we can build logic around the model and then perform validation (e.g. validate specific tags exist, as an example)
• publish the outputs from CI/CD to a common/shared location
• integration of asciidoc into the same pipeline and generate document snippets for objects present in a diagram to create companion documentation to go with diagrams. We haven't properly adopted asciidoc across the teams, but even generating "appendix" documents to can be referred to from other documents is useful as it avoids copying and pasting diagrams into documents and trying to maintain them.

I thought about this after our internal meeting on Friday, I think all that would need to change in Structurizr is:

• parsing needs to defer resolution of relationships until the completion of parsing of all DSL (this means objects are instantiated into the model, and then relationships between them are then added to the model (my ugly directory includes/workspace fragment solution achieves this, example below). This would remove the need to duplicate objects between workspaces, but still allow a workspace to "stand alone"
• allow !include to optionally dedupe files to avoid circular !includes

The above would mean that A -include-> B -include-> A would resolve and views in each of the workspaces can resolve objects from other workspaces.

Example of what my mods allow for (needs to run under our Maven build to generate diagrams and documents:

    model {

        enterprise "esb" {
            !include systems/esb.dsl
        }

        !include systems
        !include people
        !include relationships

    }

Interested on any thoughts about my ramblings :D

[chrisjschultz]

There is of course still the issue of people; they are stand alone entities as well, so if the same person is defined in multiple workspaces,

At this point, I thought I should review your earlier links and noted that in your system catalog you did exactly that, I note you include people in the parent workspace.

[arthis]

Hi @chrisjschultz ,

I have precisely the same need here for the same reasons. What did you end up doing?

Do you still use c4 model with structurizr? Are you back using EA?

I did not come across Simon's reasons explaining why modelling the relationships between containers is bad. If you have any post url, that would be nice.

[simonbrowndotje]

I did not come across Simon's reasons explaining why modelling the relationships between containers is bad.

@arthis ... see #225 (comment)

[arthis]

For anyone wondering the same, I found the reason why : https://static.simonbrown.je/gotoams2024-c4-model-misconceptions-misuses-mistakes.pdf.
It starts slide 131.

[simonbrowndotje]

As a follow-up to this discussion ... I've added an example of one approach to enterprise-wide modelling with the Structurizr DSL, which you can find at https://github.com/structurizr/examples/tree/main/enterprise (it's example 1 ... where the DSL is used to define a system catalog, with individual teams extending this to describe their own software systems and an automatically generated system landscape).

[iambic69]

Thanks Simon, for the examples and for introducing me to Backstage; it looks very interesting!

[simonbrowndotje]

You're welcome. I'd urge some caution with Backstage ... despite the hype, it's anecdotally expensive to run (especially for smaller teams) and doesn't really provide any built-in support for architecture diagrams.

[chipbite]

Great thread, the examples are great.

I am trying to come up with a way to work with our solutions level (composed of several systems) using the C4 toolset, posted a question about it as a separate topic here: #240

Would appreciate any suggestions or insights.