Refactor cobalt into using an ECS

[epage]

Cobalt's pure OOP architecture is causing problems

• Hard to refactor processing stages
• Hard to get all of the relevant state around to the relevant stages
• A lot of work to add parallelism, even with rayon, due to the current scattering of code

Ideally any new architecture will

• Make it easier for cobalt serve to reduce the amount of re-calculation on a filesystem notification

Proposal

Choice ECS

I am leaning towards specs due to its popularity.

Options

• A list of some ECSs
• Froggy
• Pyro
  • Discussion
• Spin or own, customized to the needs of cobalt
• And out of left field, salsa which is incremental computation and not an ecs

Comparisons

• Benchmark
  • Original benchmarks

General Flow

Open Questions

• Do we filter out drafts from ever being added to the ECS?
  • Not filtering simplifies the adding code
  • Filtering simplifies the processing cde

Systems

• Generate permalink
• Generate excerpts
• Generate content
• Generate pages

How we can add partial recalculation later

The filesystem watch can keep a change counter

• On change, the filesystem watch updates the path's generation with the latest counter
• Each system checks if the source's generation is older than the destination's and stops processing
• Improvement?: Each system can check if the destination actually changed and not write to it, avoiding unnecessary re-calculations further down the pipeline

An interesting balance in this is not hurting the one-shot case's performance too much. We'll need to add benchmarks (in their own PR) and separately see how switching to an ECS, adding extra parallelism, choosing the ECS storage, and adding partial recalculation impact the performance.

[Geobert]

I'm really interested in this subject as I'm quite new to ECS, trying to get the gist of it.

Filtering out drafts will make previewing our post harder, won't it?

I need to read the links before giving my opinion on if we should do ours or use one of the linked exemples

[epage]

I'm really interested in this subject as I'm quite new to ECS, trying to get the gist of it.

The eye opening explanation for me was that it effectively is an in-memory, native database.

Filtering out drafts will make previewing our post harder, won't it?

I'm referring to the default case. It would still support the option to publish drafts

[Geobert]

Other systems: 

• Generate RSS
• Copy assets?
• Syntax highlighting?

Later:

• Shortcode processing

What are the Components and The Entities?