[TASK] Update the architecture diagram

[gulfaraz]

Who is your contact person for this task?

@gulfaraz

User Story

As an external dev
I want to understand the IBF system
So that I can use IBF locally

What needs to be done

Preview Give feedback

1. Create an architecture diagram for IBF
2. Remove old architecture diagram from README, wiki, docs
3. Add new architecture diagram to wiki

Acceptance Criteria

Preview Give feedback

1. Reviewed by other devs
2. Update the wiki with explanation of the diagram

Additional context

Current architecture diagram

Why you need an Architecture Diagram?

[diderikvw]

Hi @gulfaraz , I had a look at the architecture diagram. I understand the gist of it, and have some suggestions:

1. Use the high-level "IBF System" diagram we created together last week and shared in Teams as "starting point", so how the product is defined in there is recognizable here as well. As components and names.
2. Be clear of the "visual language" that you use. What is a circle? What is a rectangle? I suggest to use UML, as that is a clearly defined and simple visual language. In this case: a UML Component Diagram as Structural Diagram would fit, I think.
3. Additional to a more conceptual diagram, consider something like a UML Deployment Diagram as well. So the Azure infrastructural architecture is also visualized and can be easily digested (not all Azure resources in detail, but high-level). I can point to an example for 121. Note that we do not put this on an open GH repo, as our Azure config is somewhat secret, no need to share that publicly, also a security risk.
4. Do you know C4 model? I have never used it, but it looks very promising. I would like to use it for some software project some day. Could be IBF System :) So that would be an alternative to UML Component and Deployment Diagrams.
5. For the Pipelines, make it more explicit of the actual pipelines we have. I am not sure if this needs to be reflected then in both diagrams. Perhaps that is overkill.
6. What does the circle "Trigger Model" mean practically? Is that e.g. en App Service running code from GH that is used to create forecasts for all data sources?
7. Nice to use mermaid, is it possible to do something with the buttons? As now they overlay part of the diagram:
8. 

[gulfaraz]

I understand the gist of it

Thank you for the compliment.

Use the high-level "IBF System" diagram we created together last week and shared in Teams as "starting point", so how the product is defined in there is recognizable here as well. As components and names.

This is not possible in the architecture diagram. The high-level IBF System diagram from last week is conceptual. It does not reflect the underlying software architecture. We have a Glossary to supplement IBF documentation to address this discrepancy.

On 2, 3, and 4.

I used generic symbols where possible but I understand it's not clear. I have not used UML and C4 but feel free to pick it up and improve on the existing diagram.

On 5 and 6.

I did my best to capture the outline of the pipeline. I don't understand the details of the pipeline like what is a trigger model. I don't know which pipelines are in production. If you need additional details, feel free to update the diagram.

Nice to use mermaid, is it possible to do something with the buttons? As now they overlay part of the diagram

Not that I know of.

---

I'm aware there are many possible improvements. This issue was created because the old diagram needed an update.

🙏 I appreciate your time and effort to review this work.

[gulfaraz]

@jannisvisser reviewed the software architecture diagram in a call. We agree that @diderikvw's suggestions are valid but do not fit in the allocated timebox.

We updated the current diagram to as state that is sufficient for this task. Feel free to create follow-up tasks for further improvements.

[diderikvw]

@jannisvisser @gulfaraz
I personally see one value of creating software architecture diagrams as a way of a better joined understanding of how the system works (as-is), or how we want it to work (to-be).

When I read:

I did my best to capture the outline of the pipeline. I don't understand the details of the pipeline like what is a trigger model. I don't know which pipelines are in production. If you need additional details, feel free to update the diagram.

I believe updating this diagram is a great opportunity to align as a team, understand better what we are doing, and become more effective together.

I find that much more valuable than "ticking the box" of having this diagram updated, without alignment, with even the creator(s) not understanding what is in it.

And does @BlaiseSelvan not need to accept an issue before it goes to done?

I made a quick start of an alternative diagram, which does align with what we created last week. IMO a UML component diagram can be conceptual, while also explaining more about how the software is actually structured.

See:
https://miro.com/app/board/uXjVLUZOptU=/

[diderikvw]

@p-phung @jannisvisser @gulfaraz
My suggestion is to do an "IBF Pipeline 101" show&tell ask-me-anything session. Phuoc can then tell us more, show us around, we can ask questions, and from that understanding we can adapt/improve the diagram we have. IMO this diagram is sort-of the "mother of all diagrams" for IBF, so good to align on it as team and have a shared conceptual understanding.

Time-wise, maybe better to do this after the "big release" end of October? So early November?

[p-phung]

@diderikvw can we schedule something in the week of 11 November?

[diderikvw]

Yes, I will close this item and create a Spike for the 101 session. FYI @BlaiseSelvan