Skip to content

v4: New lifecycle hooks #1817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Mar 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
f59bad9
Revamping the lifecycle hooks, starting with init
ericallam Mar 21, 2025
0215a8c
vibes
ericallam Mar 21, 2025
2c44e59
init.ts at the root of the trigger dir is now automatically loaded
ericallam Mar 21, 2025
17e1623
Improve init lifecycle hook types and fix tabler icons on spans
ericallam Mar 21, 2025
1777ff4
move onStart to the new lifecycle hook system
ericallam Mar 21, 2025
be676fd
onFailure
ericallam Mar 22, 2025
06f6a04
onStart
ericallam Mar 22, 2025
e816ba4
onComplete
ericallam Mar 22, 2025
aaf2ed8
onWait and onResume
ericallam Mar 22, 2025
e77e8d4
handleError
ericallam Mar 22, 2025
597b7ba
adding imports
ericallam Mar 22, 2025
e34e520
more hooks
ericallam Mar 23, 2025
dc6e659
new locals API
ericallam Mar 24, 2025
6001dfa
Add middleware types
ericallam Mar 24, 2025
2d16d66
Add middleware hooks
ericallam Mar 24, 2025
2e6d0d8
share the hook registration code
ericallam Mar 24, 2025
1167889
use new onStart
ericallam Mar 24, 2025
b974d82
use new onFailure
ericallam Mar 24, 2025
307309f
implement onComplete
ericallam Mar 24, 2025
4a8d693
a couple tweaks
ericallam Mar 24, 2025
ef871b0
starting test executor
ericallam Mar 24, 2025
2262a42
more tests and fixes
ericallam Mar 24, 2025
91225da
test on failure
ericallam Mar 24, 2025
87824a8
dry up some stuff
ericallam Mar 24, 2025
f84ddcd
test oncomplete
ericallam Mar 24, 2025
6ab74c4
implement and test handleError (now catchError)
ericallam Mar 24, 2025
7f804f3
middleware working and tests passing
ericallam Mar 25, 2025
5edbb77
use tryCatch in TaskExecutor
ericallam Mar 25, 2025
6ba2b04
more tests
ericallam Mar 25, 2025
d0a5c16
Add cleanup hook
ericallam Mar 25, 2025
30705fd
implement cleanup
ericallam Mar 25, 2025
23ae77a
handle max duration timeout errors better
ericallam Mar 25, 2025
ba520fc
Make sure and register all the config hooks
ericallam Mar 25, 2025
df9cad6
Get it all working
ericallam Mar 25, 2025
3ca5985
Hooks now all use the new types, and adding some spans
ericallam Mar 25, 2025
268c9f6
implement onWait/onResume
ericallam Mar 25, 2025
887c5ca
Add changeset
ericallam Mar 25, 2025
e745fcb
Remove console.log
ericallam Mar 25, 2025
1def8d0
Use allSettled so onWait/onResume errors don't break anything
ericallam Mar 25, 2025
6d5d10a
Support other init file names
ericallam Mar 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/weak-jobs-hide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---

v4: New lifecycle hooks
4 changes: 2 additions & 2 deletions .cursor/rules/executing-commands.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ But often, when running tests, it's better to `cd` into the directory and then r

```
cd apps/webapp
pnpm run test
pnpm run test --run
```

This way you can run for a single file easily:

```
cd internal-packages/run-engine
pnpm run test ./src/engine/tests/ttl.test.ts
pnpm run test ./src/engine/tests/ttl.test.ts --run
```
25 changes: 25 additions & 0 deletions apps/webapp/app/components/runs/v3/RunIcon.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ export function RunIcon({ name, className, spanName }: TaskIconProps) {
}

if (!name) return <Squares2X2Icon className={cn(className, "text-text-dimmed")} />;
if (tablerIcons.has(name)) {
return <TablerIcon name={name} className={className} />;
}

switch (name) {
case "task":
Expand Down Expand Up @@ -73,6 +76,28 @@ export function RunIcon({ name, className, spanName }: TaskIconProps) {
return <InformationCircleIcon className={cn(className, "text-rose-500")} />;
case "fatal":
return <HandRaisedIcon className={cn(className, "text-rose-800")} />;
case "task-middleware":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-fn-run":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-init":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onStart":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onSuccess":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onFailure":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onComplete":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onWait":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-onResume":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-catchError":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
case "task-hook-cleanup":
return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
}

return <InformationCircleIcon className={cn(className, "text-text-dimmed")} />;
Expand Down
5 changes: 3 additions & 2 deletions internal-packages/run-engine/src/engine/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ export function runStatusFromError(error: TaskRunError): TaskRunStatus {
//e.g. a bug
switch (error.code) {
case "RECURSIVE_WAIT_DEADLOCK":
case "TASK_INPUT_ERROR":
case "TASK_OUTPUT_ERROR":
case "TASK_MIDDLEWARE_ERROR":
return "COMPLETED_WITH_ERRORS";
case "TASK_RUN_CANCELLED":
return "CANCELED";
Expand Down Expand Up @@ -41,8 +44,6 @@ export function runStatusFromError(error: TaskRunError): TaskRunStatus {
case "TASK_RUN_STALLED_EXECUTING_WITH_WAITPOINTS":
case "TASK_HAS_N0_EXECUTION_SNAPSHOT":
case "GRACEFUL_EXIT_TIMEOUT":
case "TASK_INPUT_ERROR":
case "TASK_OUTPUT_ERROR":
case "POD_EVICTED":
case "POD_UNKNOWN_ERROR":
case "TASK_EXECUTION_ABORTED":
Expand Down
20 changes: 20 additions & 0 deletions packages/cli-v3/src/build/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ export type BundleResult = {
runControllerEntryPoint: string | undefined;
indexWorkerEntryPoint: string | undefined;
indexControllerEntryPoint: string | undefined;
initEntryPoint: string | undefined;
stop: (() => Promise<void>) | undefined;
};

Expand Down Expand Up @@ -229,11 +230,26 @@ export async function getBundleResultFromBuild(
let runControllerEntryPoint: string | undefined;
let indexWorkerEntryPoint: string | undefined;
let indexControllerEntryPoint: string | undefined;
let initEntryPoint: string | undefined;

const configEntryPoint = resolvedConfig.configFile
? relative(resolvedConfig.workingDir, resolvedConfig.configFile)
: "trigger.config.ts";

// Check if the entry point is an init.ts file at the root of a trigger directory
function isInitEntryPoint(entryPoint: string): boolean {
const normalizedEntryPoint = entryPoint.replace(/\\/g, "/"); // Normalize path separators
const initFileNames = ["init.ts", "init.mts", "init.cts", "init.js", "init.mjs", "init.cjs"];

// Check if it's directly in one of the trigger directories
return resolvedConfig.dirs.some((dir) => {
const normalizedDir = dir.replace(/\\/g, "/");
return initFileNames.some(
(fileName) => normalizedEntryPoint === `${normalizedDir}/${fileName}`
);
});
}

for (const [outputPath, outputMeta] of Object.entries(result.metafile.outputs)) {
if (outputPath.endsWith(".mjs")) {
const $outputPath = resolve(workingDir, outputPath);
Expand All @@ -254,6 +270,8 @@ export async function getBundleResultFromBuild(
indexControllerEntryPoint = $outputPath;
} else if (isIndexWorkerForTarget(outputMeta.entryPoint, target)) {
indexWorkerEntryPoint = $outputPath;
} else if (isInitEntryPoint(outputMeta.entryPoint)) {
initEntryPoint = $outputPath;
} else {
if (
!outputMeta.entryPoint.startsWith("..") &&
Expand All @@ -280,6 +298,7 @@ export async function getBundleResultFromBuild(
runControllerEntryPoint,
indexWorkerEntryPoint,
indexControllerEntryPoint,
initEntryPoint,
contentHash: hasher.digest("hex"),
};
}
Expand Down Expand Up @@ -357,6 +376,7 @@ export async function createBuildManifestFromBundle({
runControllerEntryPoint: bundle.runControllerEntryPoint ?? getRunControllerForTarget(target),
runWorkerEntryPoint: bundle.runWorkerEntryPoint ?? getRunWorkerForTarget(target),
loaderEntryPoint: bundle.loaderEntryPoint,
initEntryPoint: bundle.initEntryPoint,
configPath: bundle.configPath,
customConditions: resolvedConfig.build.conditions ?? [],
deploy: {
Expand Down
1 change: 1 addition & 0 deletions packages/cli-v3/src/entryPoints/dev-index-worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ await sendMessageInCatalog(
controllerEntryPoint: buildManifest.runControllerEntryPoint,
loaderEntryPoint: buildManifest.loaderEntryPoint,
customConditions: buildManifest.customConditions,
initEntryPoint: buildManifest.initEntryPoint,
},
importErrors,
},
Expand Down
111 changes: 68 additions & 43 deletions packages/cli-v3/src/entryPoints/dev-run-worker.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,30 @@
import type { Tracer } from "@opentelemetry/api";
import type { Logger } from "@opentelemetry/api-logs";
import {
AnyOnCatchErrorHookFunction,
AnyOnFailureHookFunction,
AnyOnInitHookFunction,
AnyOnStartHookFunction,
AnyOnSuccessHookFunction,
apiClientManager,
clock,
ExecutorToWorkerMessageCatalog,
type HandleErrorFunction,
lifecycleHooks,
localsAPI,
logger,
LogLevel,
resourceCatalog,
runMetadata,
runtime,
resourceCatalog,
runTimelineMetrics,
TaskRunErrorCodes,
TaskRunExecution,
timeout,
TriggerConfig,
waitUntil,
WorkerManifest,
WorkerToExecutorMessageCatalog,
runTimelineMetrics,
} from "@trigger.dev/core/v3";
import { TriggerTracer } from "@trigger.dev/core/v3/tracer";
import {
Expand All @@ -29,15 +36,17 @@ import {
logLevels,
ManagedRuntimeManager,
OtelTaskLogger,
StandardLifecycleHooksManager,
StandardLocalsManager,
StandardMetadataManager,
StandardResourceCatalog,
StandardRunTimelineMetricsManager,
StandardWaitUntilManager,
TaskExecutor,
TracingDiagnosticLogLevel,
TracingSDK,
usage,
UsageTimeoutManager,
StandardRunTimelineMetricsManager,
} from "@trigger.dev/core/v3/workers";
import { ZodIpcConnection } from "@trigger.dev/core/v3/zodIpc";
import { readFile } from "node:fs/promises";
Expand Down Expand Up @@ -89,10 +98,16 @@ process.on("uncaughtException", function (error, origin) {

const heartbeatIntervalMs = getEnvVar("HEARTBEAT_INTERVAL_MS");

const standardLocalsManager = new StandardLocalsManager();
localsAPI.setGlobalLocalsManager(standardLocalsManager);

const standardRunTimelineMetricsManager = new StandardRunTimelineMetricsManager();
runTimelineMetrics.setGlobalManager(standardRunTimelineMetricsManager);
standardRunTimelineMetricsManager.seedMetricsFromEnvironment();

const standardLifecycleHooksManager = new StandardLifecycleHooksManager();
lifecycleHooks.setGlobalLifecycleHooksManager(standardLifecycleHooksManager);

const devUsageManager = new DevUsageManager();
usage.setGlobalUsageManager(devUsageManager);
timeout.setGlobalManager(new UsageTimeoutManager(devUsageManager));
Expand Down Expand Up @@ -170,12 +185,46 @@ async function bootstrap() {

logger.setGlobalTaskLogger(otelTaskLogger);

if (config.init) {
lifecycleHooks.registerGlobalInitHook({
id: "config",
fn: config.init as AnyOnInitHookFunction,
});
}

if (config.onStart) {
lifecycleHooks.registerGlobalStartHook({
id: "config",
fn: config.onStart as AnyOnStartHookFunction,
});
}

if (config.onSuccess) {
lifecycleHooks.registerGlobalSuccessHook({
id: "config",
fn: config.onSuccess as AnyOnSuccessHookFunction,
});
}

if (config.onFailure) {
lifecycleHooks.registerGlobalFailureHook({
id: "config",
fn: config.onFailure as AnyOnFailureHookFunction,
});
}

if (handleError) {
lifecycleHooks.registerGlobalCatchErrorHook({
id: "config",
fn: handleError as AnyOnCatchErrorHookFunction,
});
}

return {
tracer,
tracingSDK,
consoleInterceptor,
config,
handleErrorFn: handleError,
workerManifest,
};
}
Expand Down Expand Up @@ -217,7 +266,7 @@ const zodIpc = new ZodIpcConnection({
}

try {
const { tracer, tracingSDK, consoleInterceptor, config, handleErrorFn, workerManifest } =
const { tracer, tracingSDK, consoleInterceptor, config, workerManifest } =
await bootstrap();

_tracingSDK = tracingSDK;
Expand Down Expand Up @@ -257,6 +306,18 @@ const zodIpc = new ZodIpcConnection({
async () => {
const beforeImport = performance.now();
resourceCatalog.setCurrentFileContext(taskManifest.entryPoint, taskManifest.filePath);

// Load init file if it exists
if (workerManifest.initEntryPoint) {
try {
await import(normalizeImportPath(workerManifest.initEntryPoint));
log(`Loaded init file from ${workerManifest.initEntryPoint}`);
} catch (err) {
logError(`Failed to load init file`, err);
throw err;
}
}

await import(normalizeImportPath(taskManifest.entryPoint));
resourceCatalog.clearCurrentFileContext();
const durationMs = performance.now() - beforeImport;
Expand Down Expand Up @@ -321,8 +382,7 @@ const zodIpc = new ZodIpcConnection({
tracer,
tracingSDK,
consoleInterceptor,
config,
handleErrorFn,
retries: config.retries,
});

try {
Expand All @@ -340,42 +400,7 @@ const zodIpc = new ZodIpcConnection({
? timeout.abortAfterTimeout(execution.run.maxDuration)
: undefined;

signal?.addEventListener("abort", async (e) => {
if (_isRunning) {
_isRunning = false;
_execution = undefined;

const usageSample = usage.stop(measurement);

await sender.send("TASK_RUN_COMPLETED", {
execution,
result: {
ok: false,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.MAX_DURATION_EXCEEDED,
message:
signal.reason instanceof Error
? signal.reason.message
: String(signal.reason),
},
usage: {
durationMs: usageSample.cpuTime,
},
metadata: runMetadataManager.stopAndReturnLastFlush(),
},
});
}
});

const { result } = await executor.execute(
execution,
metadata,
traceContext,
measurement,
signal
);
const { result } = await executor.execute(execution, metadata, traceContext, signal);

const usageSample = usage.stop(measurement);

Expand Down
1 change: 1 addition & 0 deletions packages/cli-v3/src/entryPoints/managed-index-worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ await sendMessageInCatalog(
controllerEntryPoint: buildManifest.runControllerEntryPoint,
loaderEntryPoint: buildManifest.loaderEntryPoint,
customConditions: buildManifest.customConditions,
initEntryPoint: buildManifest.initEntryPoint,
},
importErrors,
},
Expand Down
Loading