Skip to content

Both run engines will only lock to versions they can handle #1922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 13, 2025

Conversation

nicktrn
Copy link
Collaborator

@nicktrn nicktrn commented Apr 13, 2025

Previously, either run engine would lock to the current deployment, regardless of version. This meant that v3 runs could end up with a v4 deployment if there were any non-locked runs in the v3 queue. Those runs would fail. Same the other way around on downgrades. This has been fixed by locking to the last deployment of a compatible version if the current deployment is incompatible.

Summary by CodeRabbit

  • New Features
    • Enhanced worker deployment selection to ensure that the most current and eligible deployment details are returned based on promotion status.
    • Improved handling of different deployment types, allowing for more efficient retrieval of associated worker details and tasks.
    • Added metadata properties for better context in worker and deployment configurations.

These updates streamline the process for obtaining deployment information, ensuring users receive more accurate and timely results.

Copy link

changeset-bot bot commented Apr 13, 2025

⚠️ No Changeset found

Latest commit: e1b0aa2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

coderabbitai bot commented Apr 13, 2025

Walkthrough

The changes update two functions responsible for selecting worker deployments by incorporating conditional logic based on deployment types. In one function, a new type property is selected from the deployment, and the function now checks for a promotion before deciding whether to return the current deployment (if its type is "V1") or query for the latest "V1" deployment. In the other function, a check for a "MANAGED" deployment type enables an immediate return of associated worker details, otherwise triggering a query for the latest "MANAGED" deployment.

Changes

File(s) Change Summary
apps/webapp/…/workerDeployment.server.ts
internal-packages/…/worker.ts
Enhanced deployment selection logic with type-based conditions. In findCurrentWorkerDeployment, added a type field and modified control flow to return the deployment if "V1", or query for the latest "V1" deployment. In getWorkerFromCurrentlyPromotedDeployment, added an early return for "MANAGED" deployments and a fallback query for the latest when necessary.
internal-packages/…/setup.ts Updated setupBackgroundWorker function to include engine: "V2" and type: "MANAGED" in the worker and deployment creation data, respectively.

Sequence Diagram(s)

sequenceDiagram
    participant Caller as Caller
    participant FWD as findCurrentWorkerDeployment()
    participant DB as Prisma DB

    Caller->>FWD: Invoke function
    FWD->>Promotion: Check for promotion
    alt Promotion not found
        FWD-->>Caller: Return undefined
    else Promotion exists
        FWD->>Deployment: Check deployment.type
        alt type === "V1"
            FWD-->>Caller: Return current deployment
        else type !== "V1"
            FWD->>DB: Query latest "V1" deployment
            DB-->>FWD: Return latest deployment (or undefined)
            FWD-->>Caller: Return deployment or undefined
        end
    end
Loading
sequenceDiagram
    participant Caller as Caller
    participant GW as getWorkerFromCurrentlyPromotedDeployment()
    participant DB as Prisma DB

    Caller->>GW: Invoke function
    GW->>Deployment: Check deployment.type
    alt type === "MANAGED"
        GW-->>Caller: Return worker, tasks, queues, deployment
    else type !== "MANAGED"
        GW->>DB: Query latest "MANAGED" deployment
        DB-->>GW: Return latest details (or null)
        GW-->>Caller: Return details or null
    end
Loading

Poem

I'm a code rabbit hopping in the night,
Checking deployments with all my might.
"V1" and "MANAGED" now guide the way,
Querying for details without delay.
With every new line, my heart leaps high,
Celebrating changes as time goes by!
🐇💻

Tip

⚡💬 Agentic Chat (Pro Plan, General Availability)
  • We're introducing multi-step agentic chat in review comments and issue comments, within and outside of PR's. This feature enhances review and issue discussions with the CodeRabbit agentic chat by enabling advanced interactions, including the ability to create pull requests directly from comments and add commits to existing pull requests.

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 704c638 and e1b0aa2.

📒 Files selected for processing (1)
  • internal-packages/run-engine/src/engine/tests/setup.ts (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
  • GitHub Check: units / 🧪 Unit Tests
  • GitHub Check: typecheck / typecheck
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (2)
internal-packages/run-engine/src/engine/tests/setup.ts (2)

100-100: Added engine version specification aligns with PR objectives

The addition of engine: "V2" to the worker creation data is a key part of implementing the version compatibility solution described in the PR. This explicit version specification will allow the system to determine whether a worker is compatible with a particular deployment, addressing the issue where incompatible versions could lead to run failures.


238-238: Added deployment type enables version compatibility checks

Adding type: "MANAGED" to the worker deployment creation data provides essential metadata that will be used in conditional logic to determine deployment compatibility. This change supports the PR objective of ensuring run engines only lock to versions they can handle by enabling the system to differentiate between deployment types and apply appropriate version compatibility checks.

✨ Finishing Touches
  • 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai plan to trigger planning for file edits and PR creation.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
internal-packages/run-engine/src/engine/db/worker.ts (1)

292-299: Consider a more explicit name than "MANAGED" for run engine V2.

Using "MANAGED" to denote version 2 may be less transparent than naming it something like "RUN_ENGINE_V2". This is a minor naming preference, but it could improve code clarity if your organization often confuses these terms.

apps/webapp/app/v3/models/workerDeployment.server.ts (1)

92-95: Consider handling the missing promotion scenario more visibly.

Returning undefined is fine, but you could add logging or a warning to spot unexpected absence of promotions in production.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1d920d5 and 704c638.

📒 Files selected for processing (2)
  • apps/webapp/app/v3/models/workerDeployment.server.ts (2 hunks)
  • internal-packages/run-engine/src/engine/db/worker.ts (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
  • GitHub Check: units / 🧪 Unit Tests
  • GitHub Check: typecheck / typecheck
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (5)
internal-packages/run-engine/src/engine/db/worker.ts (2)

302-323: Validate fallback for environments without "MANAGED" deployments.

This code fetches the most recent "MANAGED" deployment if the current environment isn't recognized as "MANAGED". Ensure that relevant tests verify this scenario so you avoid unexpected null returns.


326-329: LGTM for returning the latest "MANAGED" deployment.

The returned structure is consistent with the code above and includes worker, tasks, queues, and deployment details.

apps/webapp/app/v3/models/workerDeployment.server.ts (3)

74-74: Good addition of the type field.

Including this field allows specialized handling of different deployment versions without guesswork.


96-99: Logic verified for V1 deployment check.

No issues here. This early return for "V1" deployments aligns with the intended version-based flow control.


101-135: Request tests covering fallback to the latest "V1" deployment.

Please confirm that you have test coverage for cases without a current "V1" promotion to ensure the fallback logic functions as intended.

@nicktrn nicktrn merged commit faf5d01 into main Apr 13, 2025
12 checks passed
@nicktrn nicktrn deleted the fix/incompatible-version-locks branch April 13, 2025 20:51
matt-aitken pushed a commit that referenced this pull request Apr 16, 2025
* run engine v1 will only lock to v1 deployments

* run engine v2 will only lock to managed v2 deployments

* test: create background worker and deployment with correct engine version
matt-aitken added a commit that referenced this pull request Apr 17, 2025
* WIP on secret env vars

* Editing individual env var values is working

* Sort the env vars by the key

* Deleting values

* Allowing setting secret env vars

* Added medium switch style

* Many style changes to the env var form

* “Copy text” -> “Copy”

* Draw a divider between hidden buttons

* Env var tweaks

* Don’t show Dev:you anymore

* Grouping the same env var keys together

* Styles improved

* Improved styling of edit panel

* Fix bun detection, dev flushing, and init command (#1914)

* update nypm to support text-based bun lockfiles

* add nypm changeset

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add changesets

* chore: remove pre.json after exiting pre mode

* init command to install v4-beta packages

* Revert "chore: remove pre.json after exiting pre mode"

This reverts commit f5694fd.

* make init default to cli version for all packages

* Release 4.0.0-v4-beta.1 (#1916)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Both run engines will only lock to versions they can handle (#1922)

* run engine v1 will only lock to v1 deployments

* run engine v2 will only lock to managed v2 deployments

* test: create background worker and deployment with correct engine version

* Add links to and from deployments (#1921)

* link from deployments tasks to filtered runs view

* jump to deployment

* don't add version links for dev (yet)

* Fix current worker deployment getter (#1924)

* only return last v1 deployment in the shared queue consumer

* be explicit about only returning managed deployments

* Add a docs page for the human-in-the-loop example project (#1919)

* Add a docs page for the human-in-the-loop example project

* Order guides, example projects and example tasks alphabetically in the docs list

* Managed run controller revamp (#1927)

* update nypm to support text-based bun lockfiles

* fix retry spans

* only download debug logs if admin

* add nypm changeset

* pull out env override logic

* use runner env gather helper

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add logger

* add execution heartbeat service

* add snapshot poller service

* fix poller

* add changesets

* create socket in constructor

* enable strictPropertyInitialization

* deprecate dequeue from version

* start is not async

* dependency injection in prep for tests

* add warm start count to all controller logs

* add restore count

* pull out run execution logic

* temp disable pre

* add a controller log when starting an execution

* refactor execution and squash some bugs

* cleanup completed docker containers by default

* execution fixes and logging improvements

* don't throw afet abort cleanup

* poller should use private interval

* rename heartbeat service file

* rename HeartbeatService to IntervalService

* restore old heartbeat service but deprecate it

* use the new interval service everywhere

* Revert "temp disable pre"

This reverts commit e03f417.

* add changeset

* replace all run engine find uniques with find first

* Release 4.0.0-v4-beta.2 (#1928)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Remove batch ID carryover for non-batch waits (#1930)

* add failing test case

* do not carry over previous batch id when blocking with waitpoint

* delete irrelevant test

* Delete project (#1913)

* Delete project

- Don’t schedule tasks if the project is deleted
- Delete queues from the master queues
- Add the old delete project UI back in

* Mark the project as deleted last

* Fix for overriding local variable

* Added a todo for deleting env queues

* Remove todo

* Improve usage flushing (#1931)

* add flush to global usage api

* enable controller debug logs

* initialize usage manager after env overrides

* add previous run id to more debug logs

* add changeset

* For secret env vars, don’t return the value

* Added a new env var repository function for getting secrets with redactions

* Test task for env vars

* Delete heartbeat file, merge mess up

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Saadi Myftija <saadi.myftija@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants