-
Notifications
You must be signed in to change notification settings - Fork 3.2k
improvement(workflow-execution): perf improvements to passing workflow state + decrypted env vars #2119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Greptile OverviewGreptile SummaryOptimized workflow execution performance by eliminating redundant database calls and parallelizing environment variable decryption. Key improvements:
The changes maintain identical functionality while reducing execution time through better resource utilization. Confidence Score: 5/5
Important Files ChangedFile Analysis
Sequence DiagramsequenceDiagram
participant Client
participant Route as execute/route.ts
participant Utils as environment/utils.ts
participant Core as execution-core.ts
participant DB as Database
Client->>Route: POST /api/workflows/[id]/execute
Route->>DB: Load workflow state (once)
DB-->>Route: Workflow data (blocks, edges, loops, parallels)
Note over Route: Cache workflow data in<br/>cachedWorkflowData
Route->>Route: Create effectiveWorkflowStateOverride<br/>(user override OR cached data)
alt Non-SSE Execution
Route->>Core: executeWorkflowCore(workflowStateOverride)
Core->>Utils: getPersonalAndWorkspaceEnv()
Utils->>DB: Fetch encrypted env vars
DB-->>Utils: Encrypted variables
Note over Utils: Decrypt ALL vars in parallel<br/>using Promise.all
Utils-->>Core: {encrypted, decrypted} env vars
Note over Core: Use cached workflow state<br/>(no DB reload)
Note over Core: Use pre-decrypted vars<br/>(no redundant decryption)
Core->>Core: Process block states (sync reduce)
Core-->>Route: Execution result
Route-->>Client: JSON response
else SSE Execution
Route->>Route: resolveOutputIds(cachedWorkflowData.blocks)
Note over Route: Use cached data<br/>(no DB reload)
Route->>Core: executeWorkflowCore(workflowStateOverride)
Core->>Utils: getPersonalAndWorkspaceEnv()
Utils->>DB: Fetch encrypted env vars
DB-->>Utils: Encrypted variables
Note over Utils: Decrypt ALL vars in parallel<br/>using Promise.all
Utils-->>Core: {encrypted, decrypted} env vars
Note over Core: Use cached workflow state<br/>(no DB reload)
Note over Core: Use pre-decrypted vars<br/>(no redundant decryption)
Core->>Core: Process block states (sync reduce)
Core-->>Route: Execution result (streaming)
Route-->>Client: SSE stream
end
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3 files reviewed, no comments
…pylon, intercom, mailchimp, loading optimizations (#2132) * fix(memory-util): fixed unbounded array of gmail/outlook pollers causing high memory util, added missing db indexes/removed unused ones, auto-disable schedules/webhooks after 10 consecutive failures (#2115) * fix(memory-util): fixed unbounded array of gmail/outlook pollers causing high memory util, added missing db indexes/removed unused ones, auto-disable schedules/webhooks after 10 consecutive failures * ack PR comments * ack * improvement(teams-plan): seats increase simplification + not triggering checkout session (#2117) * improvement(teams-plan): seats increase simplification + not triggering checkout session * cleanup via helper * feat(tools): added sentry, incidentio, and posthog tools (#2116) * feat(tools): added sentry, incidentio, and posthog tools * update docs * fixed docs to use native fumadocs for llms.txt and copy markdown, fixed tool issues * cleanup * enhance error extractor, fixed posthog tools * docs enhancements, cleanup * added more incident io ops, remove zustand/shallow in favor of zustand/react/shallow * fix type errors * remove unnecessary comments * added vllm to docs * feat(i18n): update translations (#2120) * feat(i18n): update translations * fix build --------- Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> * improvement(workflow-execution): perf improvements to passing workflow state + decrypted env vars (#2119) * improvement(execution): load workflow state once instead of 2-3 times * decrypt only in get helper * remove comments * remove comments * feat(models): host google gemini models (#2122) * feat(models): host google gemini models * remove unused primary key * feat(i18n): update translations (#2123) Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> * feat(tools): added zendesk, pylon, intercom, & mailchimp (#2126) * feat(tools): added zendesk, pylon, intercom, & mailchimp * finish zendesk and pylon * updated docs * feat(i18n): update translations (#2129) * feat(i18n): update translations * fixed build --------- Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com> * fix(permissions): add client-side permissions validation to prevent unauthorized actions, upgraded custom tool modal (#2130) * fix(permissions): add client-side permissions validation to prevent unauthorized actions, upgraded custom tool modal * fix failing test * fix test * cleanup * fix(custom-tools): add composite index on custom tool names & workspace id (#2131) --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: waleedlatif1 <waleedlatif1@users.noreply.github.com>
…w state + decrypted env vars (simstudioai#2119) * improvement(execution): load workflow state once instead of 2-3 times * decrypt only in get helper * remove comments * remove comments
Summary
Type of Change
Testing
Tested manually running
Checklist