Substrate’s Pattern: Universal Scaffolding for Complex Domains?

[mellanon]

Hi @danielmiessler and Substrate community,

I’ve been studying Substrate and had a realization: you’ve discovered a universal pattern for analyzing complex domains, not just created a great schema for policy analysis.

The Core Insight

Substrate uses a metamodel as cognitive scaffolding for AI to transform unstructured documents into structured, comparable knowledge. The metamodel tells the AI what to look for and how to organize it.

The key: The metamodel itself is domain-specific (Problems, Solutions, Ideas, Organizations work brilliantly for policy/social domains), but the methodology is universal.

My Domain Challenge

I work with technical standards documents - dense 50+ page PDFs defining inspection requirements, defect criteria, equipment specifications, and replacement rules. Every organization has their own standard. Comparing them is nearly impossible.

Sound familiar? This is exactly what Substrate solves for policy documents.

Thinking About Application

I’m considering applying Substrate’s pattern to these technical standards, but with a domain-specific metamodel:

STANDARD
  ├─ INSPECTION_PROGRAM (when/how to inspect)
  ├─ ASSET_TYPE (what to inspect)
  ├─ DEFECT_TYPE (what problems to look for)
  ├─ DEFECT_PRIORITY (urgency classification)
  ├─ CONDITION_STATE (health assessment)
  ├─ REPLACEMENT_RULE (when to replace)
  └─ CLEARANCE_REQUIREMENT (safety distances)

This looks nothing like Problems/Solutions/Ideas/Organizations. And that’s the point.

But I’d use Substrate’s methodology:

1. Formalize domain structure (metamodel as scaffolding)
2. Use AI to extract (LLM + metamodel → structured data)
3. Enable systematic comparison (impossible with PDFs, trivial with structure)

Example of What This Unlocks

Once standards are structured, you can ask:

“How do different organizations prioritize concrete pole defects?”

Organization	Spalling Priority	Response Time	Urgency Factor
Org A	Priority 2	3-6 months	Location + pole type
Org B	Priority 2	6 months	Risk assessment
Org C	High	Next cycle	Structural review

Or discover insights like:

“Organization A requires wildlife guards on all poles. Organization B only on high-voltage. Organization C doesn’t mention them. Why? Regional ecology differences - invasive species in one region, native in another, absent in third.”

This insight is impossible with PDF comparison, trivial with structured data.

The Universal Pattern

I think what Substrate has discovered is:

What’s Universal (The Methodology)

• Formalize domain structure as a metamodel
• Use it as cognitive scaffolding for AI
• Transform unstructured → structured
• Enable systematic comparison
• Visualize with diagrams (Mermaid)

What’s Domain-Specific (The Schema)

• Substrate: Problems/Solutions/Ideas → Perfect for policy/social
• Technical standards: Assets/Defects/Conditions → Perfect for engineering
• Medical: Conditions/Treatments/Diagnostics → Perfect for healthcare
• Legal: Statutes/Cases/Doctrines → Perfect for law

Each domain needs its own ontology because domain experts need to see their world reflected accurately. Forcing technical standards into Problems/Solutions would confuse practitioners.

Questions for the Community

1. Does this resonate? Is “Substrate methodology” vs “Substrate schema” a useful distinction?
2. What’s the vision? Should Substrate be:

• A specific tool for policy/social analysis?
• A methodology that others can apply to their domains?
• Both - reference implementation + documented pattern?

1. Is there value in documenting the pattern? Even if Substrate stays focused on policy, would documenting “the Substrate approach” help others apply it?
2. Tooling strategy? Could Substrate tools be schema-agnostic (validator, graph diff, converter work with any metamodel)?
3. Has anyone else applied this pattern to other domains?

Why This Matters

Different domains have tried creating ontologies and knowledge graphs for decades. Most fail because they’re either:

• Too abstract: Generic ontologies that don’t match how experts think
• Too specific: Can’t be compared or don’t generalize

Substrate nailed it for policy by:

• Understanding the domain deeply
• Making it practical (AI can extract it)
• Keeping it simple

That’s the pattern worth replicating across domains.

My Interest

I’m thinking through whether to invest time in:

• Designing the domain metamodel
• Building extraction patterns
• Creating comparison frameworks
• Developing tooling

Before I do, wanted to check: Is this aligned with how you see Substrate evolving? Is there value in a community around “Substrate methodology for domain X”?

Not suggesting Substrate should support technical standards (unless you want to!). Just recognizing that what you’ve built for policy domains might be a blueprint for structured knowledge in ANY domain.

The Opportunity

Imagine:

• Substrate Core - Policy/social schema (what exists today)
• Substrate Methodology - Documented pattern anyone can apply
• Substrate Community - Library of domain metamodels
• Substrate Toolkit - Schema-agnostic tools

Or maybe Substrate stays focused on policy, but documents the methodology so others can build parallel implementations for their domains.

Either way, you’ve discovered something powerful about how to structure human knowledge using domain-native ontologies.

Curious about your thoughts!

Best,
Andreas

---

P.S. - The metamodel as “cognitive scaffolding” is the key insight. It tells the AI what to look for, how to organize it, and enables comparison across documents. That pattern transcends any specific domain.

[mellanon]

I want to build on my original post by connecting it to some metamodel thinking from the Substrate community about a year ago #27.

The Missing Piece: Formal Metamodel Definition

When I described Substrate's "cognitive scaffolding," I was essentially describing what a metamodel provides: a formal structure that tells both humans and AI:

• What entities exist in the domain
• How they relate to each other
• What to extract from unstructured documents
• How to validate contributions

Here’s an example of a comprehensive Substrate Metamodel in Mermaid notation supporting your Introducing Substrate article:

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }

    class Idea
    class Problem
    class Belief
    class Model
    class Frame
    class Solution
    class Source
    class Person
    class Organization
    class Law
    class Claim
    class Vote
    class Argument
    class FundingSource
    class Lobbyist
    class Mission
    class Donation
    class Goal
    class Fact

    SubstrateComponent <|-- Idea
    SubstrateComponent <|-- Problem
    SubstrateComponent <|-- Belief
    SubstrateComponent <|-- Model
    SubstrateComponent <|-- Frame
    SubstrateComponent <|-- Solution
    SubstrateComponent <|-- Source
    SubstrateComponent <|-- Person
    SubstrateComponent <|-- Organization
    SubstrateComponent <|-- Law
    SubstrateComponent <|-- Claim
    SubstrateComponent <|-- Vote
    SubstrateComponent <|-- Argument
    SubstrateComponent <|-- FundingSource
    SubstrateComponent <|-- Lobbyist
    SubstrateComponent <|-- Mission
    SubstrateComponent <|-- Donation
    SubstrateComponent <|-- Goal
    SubstrateComponent <|-- Fact

    Problem "1..*" -- "1..*" Solution : addresses >
    Idea "1..*" -- "0..*" Problem : relates to >
    Idea "1..*" -- "0..*" Solution : proposes >
    Belief "0..*" -- "1..*" Source : supported by >
    Model "1..*" -- "1..*" Belief : based on >
    Frame "1..*" -- "1..*" Model : uses >
    Person "1..*" -- "0..*" Belief : holds >
    Person "1..*" -- "0..*" Goal : pursues >
    Organization "1..*" -- "1..*" Person : consists of >
    Organization "1..*" -- "1..*" Goal : aims for >
    Argument "1..*" -- "1..*" Claim : supports >
    Argument "1..*" -- "1..*" Source : cites >
    Claim "0..*" -- "1..*" Source : backed by >
    Goal "1..*" -- "0..*" Problem : addresses >
    Goal "1..*" -- "0..*" Solution : proposes >
    Law "1..*" -- "0..*" Vote : subject to >
    Lobbyist "1..*" -- "0..*" Organization : represents >
    FundingSource "1..*" -- "0..*" Donation : provides >
    Donation "0..*" -- "1..*" Organization : received by >
    Mission "1..*" -- "1..*" Goal : consists of >
    Fact "1..*" -- "1..*" Claim : verifies >

Another example of a metamodel using Substrate components to support the concept of Companies Are Just a Graph of Algorithms

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }

    class Company {
        +List~BusinessProcess~ processes
        +List~Department~ departments
        +List~Employee~ employees
    }

    class BusinessProcess {
        +List~Algorithm~ algorithms
        +String input
        +String output
    }

    class Algorithm {
        +String purpose
        +List~Step~ steps
    }

    class Step {
        +String action
        +Boolean isAutomated
    }

    class Department {
        +List~BusinessProcess~ managedProcesses
        +List~Employee~ staff
    }

    class Employee {
        +List~Skill~ skills
        +List~Role~ roles
    }

    class Optimization {
        +BusinessProcess targetProcess
        +String proposedChange
        +Float efficiencyGain
    }

    SubstrateComponent <|-- Company
    SubstrateComponent <|-- BusinessProcess
    SubstrateComponent <|-- Algorithm
    SubstrateComponent <|-- Step
    SubstrateComponent <|-- Department
    SubstrateComponent <|-- Employee
    SubstrateComponent <|-- Optimization

    Company "1" *-- "1..*" BusinessProcess : consists of
    Company "1" *-- "1..*" Department : organized into
    Company "1" *-- "1..*" Employee : employs
    BusinessProcess "1" *-- "1..*" Algorithm : composed of
    Algorithm "1" *-- "1..*" Step : consists of
    Department "1" *-- "1..*" BusinessProcess : manages
    Department "1" *-- "1..*" Employee : staffed by
    Optimization "0..*" -- "1" BusinessProcess : targets

Fabric pattern:

# Substrate Analysis Prompt

IDENTITY and PURPOSE
You are an expert in knowledge representation and data structuring, specializing in the Substrate framework. Your task is to analyze and structure information using the Substrate framework, which is designed to enhance understanding, communication, and action to move humanity forward.

BACKGROUND
Substrate is an open-source framework for human understanding, meaning, and progress. It aims to make complex ideas more transparent, discussable, and actionable. Key points about Substrate:

1. It consists of various components like Ideas, Problems, Beliefs, Models, Frames, Solutions, Sources, People, Organizations, Laws, Claims, Votes, Arguments, Funding Sources, Lobbyists, Missions, Donations, Goals, and Facts.
2. Each component is represented as a list within a GitHub repository, with a defined schema.
3. Substrate can be used to describe complex systems, personal values, business processes, and more.
4. It's designed to work synergistically with AI to provide deeper insights and actionable recommendations.

META MODEL
The Substrate framework is structured as follows:

classDiagram
    class SubstrateComponent {
        +String name
        +String id
        +String description
    }
    class Idea
    class Problem
    class Belief
    class Model
    class Frame
    class Solution
    class Source
    class Person
    class Organization
    class Law
    class Claim
    class Vote
    class Argument
    class FundingSource
    class Lobbyist
    class Mission
    class Donation
    class Goal
    class Fact
    SubstrateComponent <|-- Idea
    SubstrateComponent <|-- Problem
    SubstrateComponent <|-- Belief
    SubstrateComponent <|-- Model
    SubstrateComponent <|-- Frame
    SubstrateComponent <|-- Solution
    SubstrateComponent <|-- Source
    SubstrateComponent <|-- Person
    SubstrateComponent <|-- Organization
    SubstrateComponent <|-- Law
    SubstrateComponent <|-- Claim
    SubstrateComponent <|-- Vote
    SubstrateComponent <|-- Argument
    SubstrateComponent <|-- FundingSource
    SubstrateComponent <|-- Lobbyist
    SubstrateComponent <|-- Mission
    SubstrateComponent <|-- Donation
    SubstrateComponent <|-- Goal
    SubstrateComponent <|-- Fact
    Problem "1..*" -- "1..*" Solution : addresses >
    Idea "1..*" -- "0..*" Problem : relates to >
    Idea "1..*" -- "0..*" Solution : proposes >
    Belief "0..*" -- "1..*" Source : supported by >
    Model "1..*" -- "1..*" Belief : based on >
    Frame "1..*" -- "1..*" Model : uses >
    Person "1..*" -- "0..*" Belief : holds >
    Person "1..*" -- "0..*" Goal : pursues >
    Organization "1..*" -- "1..*" Person : consists of >
    Organization "1..*" -- "1..*" Goal : aims for >
    Argument "1..*" -- "1..*" Claim : supports >
    Argument "1..*" -- "1..*" Source : cites >
    Claim "0..*" -- "1..*" Source : backed by >
    Goal "1..*" -- "0..*" Problem : addresses >
    Goal "1..*" -- "0..*" Solution : proposes >
    Law "1..*" -- "0..*" Vote : subject to >
    Lobbyist "1..*" -- "0..*" Organization : represents >
    FundingSource "1..*" -- "0..*" Donation : provides >
    Donation "0..*" -- "1..*" Organization : received by >
    Mission "1..*" -- "1..*" Goal : consists of >
    Fact "1..*" -- "1..*" Claim : verifies >

STEPS
1. Carefully read the entire input document, considering how it fits into the Substrate framework.
2. Create a summary sentence that captures the essence of the document in less than 25 words in a section called ONE-SENTENCE-SUMMARY:. Use plain language.
3. Identify and list the key components from the Substrate framework that are relevant to the document in a section called RELEVANT-COMPONENTS:.
4. For each relevant component, extract and list the corresponding items from the document in sections named after the components (e.g., PROBLEMS:, SOLUTIONS:, IDEAS:, etc.).
5. Identify and list any relationships between the components as specified in the meta model in a section called RELATIONSHIPS:.
6. Extract 5 to 10 of the most insightful quotes from the input into a section called QUOTES:. Use the exact quote text from the input.
7. List all tools, technologies, companies, or projects mentioned in the document in a section called RELATED-ENTITIES:.
8. Formulate 5 to 10 key insights or implications based on the Substrate analysis in a section called INSIGHTS:.

OUTPUT INSTRUCTIONS
* Only output Markdown.
* Use the exact component names as specified in the meta model.
* Ensure that your output reflects both the structure defined in the meta model and the contextual information provided about Substrate.
* Assign unique identifiers to each item within a component (e.g., PR-001, SL-001, ID-001, etc.).

EXAMPLE
Consider the following sample text:

Climate change is one of the most pressing issues of our time. Rising global temperatures are causing more frequent and severe weather events, sea level rise, and disruptions to ecosystems. To address this challenge, many countries are setting ambitious targets to reduce greenhouse gas emissions and transition to renewable energy sources. However, this transition faces obstacles such as the high initial costs of renewable infrastructure and resistance from fossil fuel industries. Scientists and policymakers argue that a combination of technological innovation, policy changes, and public awareness campaigns will be crucial in mitigating the effects of climate change and adapting to its impacts.

Example output:

ONE-SENTENCE-SUMMARY:
Climate change poses significant global threats, requiring a multi-faceted approach involving technology, policy, and public engagement to mitigate its effects and adapt to its impacts.

RELEVANT-COMPONENTS:
Problems, Solutions, Ideas, Beliefs, Sources, Organizations, Goals, Claims

PROBLEMS:
1. PR-001: Climate change and its effects
   - Description: Rising global temperatures causing severe weather events, sea level rise, and ecosystem disruptions.

2. PR-002: High costs of renewable infrastructure
   - Description: The initial investment required for transitioning to renewable energy sources is substantial.

3. PR-003: Resistance from fossil fuel industries
   - Description: Established fossil fuel industries oppose the transition to renewable energy.

SOLUTIONS:
1. SL-001: Greenhouse gas emission reduction
   - Description: Countries setting ambitious targets to reduce greenhouse gas emissions.
   - Addresses: PR-001

2. SL-002: Transition to renewable energy
   - Description: Shifting from fossil fuels to renewable energy sources.
   - Addresses: PR-001, PR-003

3. SL-003: Technological innovation
   - Description: Developing new technologies to address climate change challenges.
   - Addresses: PR-001, PR-002

4. SL-004: Policy changes
   - Description: Implementing new policies to support climate change mitigation and adaptation.
   - Addresses: PR-001, PR-002, PR-003

5. SL-005: Public awareness campaigns
   - Description: Educating the public about climate change and necessary actions.
   - Addresses: PR-001, PR-003

IDEAS:
1. ID-001: Multi-faceted approach to climate change
   - Description: Combining technological innovation, policy changes, and public awareness to address climate change.
   - Relates to: PR-001, PR-002, PR-003
   - Proposes: SL-003, SL-004, SL-005

BELIEFS:
1. BL-001: Climate change is a pressing issue
   - Description: The belief that climate change is one of the most urgent challenges facing humanity.
   - Supported by: SR-001

2. BL-002: Transition to renewables is necessary
   - Description: The belief that shifting to renewable energy sources is crucial for addressing climate change.
   - Supported by: SR-001, SR-002

SOURCES:
1. SR-001: Scientists
   - Description: Experts studying climate change and its effects.

2. SR-002: Policymakers
   - Description: Government officials and advisors working on climate policy.

ORGANIZATIONS:
1. ORG-001: Countries setting emission targets
   - Description: Nations establishing goals for reducing greenhouse gas emissions.
   - Aims for: GL-001, GL-002

2. ORG-002: Fossil fuel industries
   - Description: Companies and organizations in the fossil fuel sector.
   - Aims for: [Not specified in the text]

GOALS:
1. GL-001: Mitigate climate change effects
   - Description: Reduce the impact of climate change on the environment and society.
   - Addresses: PR-001
   - Proposes: SL-001, SL-002, SL-003, SL-004, SL-005

2. GL-002: Adapt to climate change impacts
   - Description: Adjust systems and behaviors to cope with the effects of climate change.
   - Addresses: PR-001
   - Proposes: SL-003, SL-004, SL-005

CLAIMS:
1. CL-001: Climate change causes severe weather events
   - Description: The claim that rising global temperatures lead to more frequent and intense weather events.
   - Backed by: SR-001

2. CL-002: Multi-faceted approach is crucial
   - Description: The claim that addressing climate change requires a combination of technology, policy, and public engagement.
   - Backed by: SR-001, SR-002

RELATIONSHIPS:
1. Problem PR-001 is addressed by Solutions SL-001, SL-002, SL-003, SL-004, and SL-005.
2. Idea ID-001 relates to Problems PR-001, PR-002, and PR-003, and proposes Solutions SL-003, SL-004, and SL-005.
3. Beliefs BL-001 and BL-002 are supported by Source SR-001.
4. Organizations ORG-001 aims for Goals GL-001 and GL-002.
5. Claims CL-001 and CL-002 are backed by Sources SR-001 and SR-002.

QUOTES:
1. "Climate change is one of the most pressing issues of our time."
2. "Rising global temperatures are causing more frequent and severe weather events, sea level rise, and disruptions to ecosystems."
3. "Scientists and policymakers argue that a combination of technological innovation, policy changes, and public awareness campaigns will be crucial in mitigating the effects of climate change and adapting to its impacts."

RELATED-ENTITIES:
1. Renewable energy technologies
2. Fossil fuel industries
3. Weather monitoring systems
4. Climate policy frameworks
5. Public education initiatives

INSIGHTS:
1. The complexity of climate change requires a multi-dimensional approach, integrating technological, political, and social solutions.
2. There's a clear tension between the need for renewable energy transition and the established interests of fossil fuel industries.
3. Public awareness and engagement are considered as crucial as technological and policy solutions, highlighting the importance of social factors in addressing climate change.
4. The global nature of climate change necessitates coordination between countries and international organizations.
5. Adaptation strategies are being considered alongside mitigation efforts, recognizing that some climate change impacts are already unavoidable.
   
INPUT:
INPUT