RFC: Multi-language Runtime Support for Trigger.dev

[zvictor]

RFC: Multi-language Runtime Support for Trigger.dev

Summary

I propose an approach to expand Trigger.dev's runtime support beyond Node.js and Bun to include multiple programming languages and runtimes while maintaining a consistent developer experience and minimizing implementation overhead.

Problem Statement

Currently, Trigger.dev only supports Node.js (with experimental support for Bun), which limits its usability for teams working with diverse technology stacks. Tasks are defined using the Trigger.dev SDK, which is designed specifically for JavaScript/TypeScript environments. To expand adoption and utility, Trigger.dev needs to support additional languages and runtimes natively, without requiring tasks to be written in or proxied through Node.js.

Desired Outcome

Developers should be able to:

1. Write Trigger.dev tasks in their preferred programming language
2. Configure and deploy these tasks using a familiar workflow
3. Trigger and manage tasks across different languages using a consistent interface
4. Benefit from all core Trigger.dev features (retries, queues, observability) regardless of language

The platform should:

1. Execute tasks in their native runtime environment
2. Provide first-class support for various languages without proxying through Node.js
3. Maintain consistency in how tasks are defined, discovered, and executed
4. Scale this approach to support new languages with minimal engineering effort

Goals

1. Developer Experience: Provide a natural, idiomatic experience for each supported language
2. First-Class Support: Tasks should be displayed in the webapp and run by the task runner the same way, independently of the language
3. Type Safety: Maintain type safety when possible, especially when triggering tasks
4. Feature Parity: Ensure all core Trigger.dev features work across languages
5. Extensibility: Create a system that can easily support new languages

Non-Goals

2. Creating full-featured SDKs for each language
3. Supporting language-specific features that don't map to Trigger.dev concepts
4. Building a new protocol layer or adapter for communication with Trigger.dev
5. Redesigning the existing task triggering mechanism

Proposed Solution

I have been thinking about this problem a bit and the best interface I could think of that keeps things simple/intuitive and still allows for good separation of code is to allow for runtime to be set per tasks directory definition, in the dirs option (1), and then set a new pattern for tasks declaration (2):

1. Configuration-Level Support

// trigger.config.ts
export default defineConfig({
  project: "<project ref>",
  runtime: "bun", // 1. the default runtime can still be changed
  dirs: [
    "./trigger" // 2. if not explicitly set, the default runtime is used (#1)
    {
      path: "./python-task",
      runtime: "python",
      options: { ... } // 3. set of exclusive settings of each runtime, e.g. `requirementsFile = './requirements.txt'` for python
    }
  ],
});

2. Task Definition Alternatives

For how tasks are defined within each language, I'm considering several alternatives that could be used individually or in combination. I'd like community feedback on these approaches:

Alternative A: File Naming Convention

Tasks are defined in files following a pattern (*.task.* or task_*.*), with a standard exported function.

# hello_world.task.py
def run(payload, context):
    print(payload["message"])
    return {"status": "success"}

Pros:

• Simple to implement and understand
• Works in any language

Cons:

• Cannot include type definitions
• Limited metadata for task configuration
• No easy way to specify retry policies, queues, etc.

Alternative B: Structured Comments/Docstrings

Task configuration is defined in a structured comment format that is parsable by the task discoverer.

# hello_world.py
"""
@trigger.task
id: python-hello-world
payload:
  message: str
  status: str
retry:
  maxAttempts: 3
queue:
  concurrencyLimit: 1
"""
def run(payload, context):
    print(payload["message"])
    return {"status": "success"}

Pros:

• No external dependencies required
• Works in any language with comments/docstrings
• Can include rich task configuration
• Supports type hints for payload and return values

Cons:

• Parsing comments is more complex and brittle
• No IDE support for validating comment syntax

Alternative C: Language-Specific Decorators/Annotations

For languages where it's natural, minimal decorators or annotations could be provided.

from trigger_dev import task

@task(
  id="python-hello-world",
  payload={ "message": str, "status": str },
  retry={"maxAttempts": 3},
  queue={"concurrencyLimit": 1}
)
def run(payload, context):
    print(payload["message"])
    return {"status": "success"}

Pros:

• Most idiomatic experience in each language
• Best IDE support and developer experience
• Strong typing support
• Clear connection between task configuration and implementation

Cons:

• Requires maintaining minimal special helpers for each language
• More complex to implement across multiple languages

I welcome community feedback on which approach(es) would provide the best balance of simplicity, flexibility, and developer experience.

3. Runtime Implementation

Each supported runtime will need:

1. A task discoverer that finds and registers tasks based on conventions

   • This can be implemented as a build extension using Trigger.dev's existing build extension system
   • The extension would scan directories based on configuration (1)

2. A task runner that executes tasks in the appropriate container environment, keeping full and native support for the language

   • The runner needs to be aligned to the protocol used by the default runner, as designing and implementing a new protocol would be cumbersome (nobody wants to recreate gRPC after all)
   • This means that initially, other languages might not support every feature available to Node.js tasks, but they would still be functional and useful for many use cases.

Notes

• This work will replace the limited Python support in feat(build): Add support for Python scripts via pythonExtension #1686 that currently spawns Python processes from Node.js
• Implementation will build upon the new run engine being developed in Run Engine 2.0 (alpha) #1575
• I believe the configuration-level approach is the most straightforward path forward and seek community input mostly on the best conventions for defining tasks in different languages.
• This approach enables incremental adoption of new languages and runtimes, starting with the most requested ones, while establishing a pattern that can scale to support many more in the future.

[zvictor]

Other Alternatives Considered for Task Definition

1. Full SDKs for Each Language

Pros:

• Most idiomatic experience in each language
• Full type safety and code completion

Cons:

• High maintenance burden
• Version synchronization challenges
• Steeper learning curve for each language

2. Pure Configuration Approach (External Files)

Pros:

• Simplest to implement
• Language-agnostic

Cons:

• Poor developer experience
• Lacks type safety
• Disconnects task logic from its configuration

3. Custom IDL for Task Definitions

Pros:

• Strong type safety across languages
• Formal contract between tasks

Cons:

• Complex to implement and use
• Adds another technology to learn
• Limited adoption of existing IDLs for this use case

[zvictor]

Please vote on your most desired language/runtime at https://discord.com/channels/1066956501299777596/1066956844553207828/1344005190315020348