Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/docs/assets/logo-with-bg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
466 changes: 247 additions & 219 deletions docs/docs/docker-compose-setup.md

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions docs/docs/exosphere/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This document provides a comprehensive overview of Exosphere's top-level architecture, focusing on state execution, fanout mechanisms, and the unites keyword for stage unification.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Overview

Exosphere is built around a **state-based execution model** where workflows are composed of discrete states that can be executed independently. The architecture consists of several key components:
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/create-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

The `Runtime` class is the core component that manages the execution environment for your Exosphere nodes. It handles node registration, state polling, execution, and communication with the state manager.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Runtime Setup

Before creating a runtime, you need to set up the state manager and configure your environment variables.
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

The Exosphere dashboard provides a comprehensive web interface for monitoring, debugging, and managing your workflows. This guide shows you how to set up and use the dashboard effectively.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Dashboard Overview

The Exosphere dashboard is a modern web application that connects to your state manager backend through secure server-side routes and provides:
Expand Down
90 changes: 90 additions & 0 deletions docs/docs/exosphere/local-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Local Setup Guide

This guide walks you through setting up Exosphere locally for development and testing. You'll learn how to run both the state manager and dashboard components, either using Docker Compose for quick setup or running them individually for more control.

## Overview

Exosphere consists of two main components that work together:

1. **State Manager** - Backend service that handles workflow execution and state management
2. **Dashboard** - Web interface for monitoring and managing workflows

You can set these up using:

- **Docker Compose** (recommended for quick start) - runs both services together
- **Individual Setup** - run each service separately for development

## Prerequisites

Before you begin, ensure you have:

- [Docker](https://docs.docker.com/get-docker/) installed (for Docker Compose approach)
- [Python 3.12+](https://www.python.org/downloads/) (for individual setup)
- [Node.js 18+](https://nodejs.org/) (for dashboard development)
- [MongoDB](https://www.mongodb.com/try/download/community) instance (cloud or local)

## Setup

=== "Docker Compose Setup (Recommended)"

The fastest way to get Exosphere running locally is using Docker Compose. This approach handles all the configuration and networking automatically.

Follow the guide: [Get Exosphere running locally with Docker](../docker-compose-setup.md)

=== "Individual Component Setup"

If you prefer more control over each component or want to develop locally, you can set up the state manager and dashboard separately.

To get Exosphere state manager running, follow [State Manager Setup](./state-manager-setup.md)

For complete monitoring dashboard setup details, see our [Dashboard Guide](./dashboard.md).


## Testing Your Setup

Once both services are running, test the complete setup:

1. **Verify state manager**:
```bash
curl http://localhost:8000/health
```

2. **Verify dashboard**:
```bash
curl http://localhost:3000
```

3. **Check API documentation**:
- Open http://localhost:8000/docs in your browser
- Test the `/health` endpoint

## Next Steps

With your local Exosphere instance running, you're ready to:

1. **[Register your first node](./register-node.md)** - Create custom processing logic
2. **[Create and run workflows](./create-graph.md)** - Build your first workflow
3. **[Monitor execution](./dashboard.md)** - Use the dashboard to track progress

## Troubleshooting

### Common Issues

**Port conflicts**: If ports 8000 or 3000 are already in use, modify the port mappings in your Docker commands or configuration files.

**MongoDB connection**: Ensure your MongoDB instance is accessible and the connection string is correct.

**Authentication errors**: Verify that `EXOSPHERE_API_KEY` matches `STATE_MANAGER_SECRET`.

**Dashboard can't connect**: Check that the state manager is running and accessible at the configured URI.

### Getting Help

- Check the [Docker Compose Setup Guide](../docker-compose-setup.md) for detailed configuration
- Review the [State Manager Setup Guide](./state-manager-setup.md) for backend troubleshooting
- Consult the [Dashboard Guide](./dashboard.md) for frontend issues
- Join our [Discord community](https://discord.com/invite/zT92CAgvkj) for support

## Production Considerations

For production deployment options, see our [hosted platform](https://exosphere.host) or contact our team for enterprise solutions.
28 changes: 28 additions & 0 deletions docs/docs/exosphere/register-node.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Nodes are the building blocks of Exosphere workflows. Each node defines a specific piece of processing logic with typed inputs, outputs, and secrets. This guide shows you how to create and register custom nodes.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Node Structure

Every node inherits from `BaseNode` and defines three key components:
Expand All @@ -28,6 +30,32 @@ class MyNode(BaseNode):
pass
```


### Node Lifecycle Architecture

```mermaid
stateDiagram-v2
[*] --> Registered
Registered --> Ready
Ready --> Executing
Executing --> Completed
Executing --> Failed
Failed --> Ready
Completed --> Ready

state Executing {
[*] --> ValidateInputs
ValidateInputs --> ProcessData
ProcessData --> ValidateOutputs
ValidateOutputs --> [*]
}

note right of Failed
Automatic retry with
exponential backoff
end note
```

### Inputs

Define the data your node expects to receive:
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/state-manager-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This guide provides step-by-step instructions for setting up the Exosphere state manager in both local and production environments.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Overview

The Exosphere state manager is the core backend service that handles workflow execution, state management, and coordination between nodes. It provides a REST API for managing graph templates, registered nodes, and workflow execution states.
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/trigger-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Once you have created a graph template, you can trigger its execution by sending trigger states to the state manager. This guide shows you how to trigger graphs and monitor their execution.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

The recommended way to trigger graphs is using the Exosphere Python SDK, which provides a clean interface to the State Manager API.

=== "Exosphere Python SDK"
Expand Down
51 changes: 34 additions & 17 deletions docs/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,6 @@ If you're working in a Jupyter notebook or Python REPL, consider these alternati
).start() # Blocks and runs forever
```

## Next Steps

Now that you have the basics, explore:

- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress

## Key Features

Expand All @@ -178,16 +170,41 @@ Now that you have the basics, explore:
- **Error Handling**: Built-in retry mechanisms and error recovery
- **Scalability**: Designed for high-volume batch processing and workflows

## Architecture
## Next Steps

```
+-------------------+ +--------------------+ +-------------------+
| Your Nodes | | Runtime | | State Manager |
| |<-->| |<-->| |
| • Inputs | | • Registration | | • Orchestration |
| • Outputs | | • Execution | | • State Mgmt |
| • Secrets | | • Error Handling | | • Dashboard |
+-------------------+ +--------------------+ +-------------------+
Now that you have the basics, explore:

- **[Local Setup](./exosphere/local-setup.md)** - Set up Exosphere locally for development and testing
- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress


### Data Flow Architecture

```mermaid
sequenceDiagram
participant Client as Client Application
participant Runtime as Runtime
participant Node as Node Executor
participant StateMgr as State Manager
participant MongoDB as State Store

Client->>Runtime: Initialize with nodes
Runtime->>StateMgr: Register runtime & nodes
StateMgr->>MongoDB: Store registration info

loop Workflow Execution
StateMgr->>Runtime: Trigger node execution
Runtime->>Node: Execute node logic
Node->>Runtime: Return outputs
Runtime->>StateMgr: Update execution state
StateMgr->>MongoDB: Persist state changes
StateMgr->>Runtime: Trigger next node (if any)
end

StateMgr->>Client: Return final results
```

## Data Model (v1)
Expand Down
Loading