Improve usage flushing #1931
Improve usage flushing #1931
Conversation
🦋 Changeset detectedLatest commit: f9b4d25 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
WalkthroughThis change introduces a new Changes
Sequence Diagram(s)sequenceDiagram
participant Worker
participant UsageAPI
participant UsageManager
Worker->>UsageAPI: flush()
UsageAPI->>UsageManager: flush()
UsageManager-->>UsageAPI: Promise<void>
UsageAPI-->>Worker: Promise<void>
Possibly related PRs
Suggested reviewers
Poem
Tip ⚡💬 Agentic Chat (Pro Plan, General Availability)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
packages/core/src/v3/usage/devUsageManager.ts (1)
51-52: Empty implementation meets interface requirements but could use documentation.The empty implementation of
flush()satisfies the interface requirement. Consider adding a comment to explain why this is intentionally a no-op method for the development environment.- async flush(): Promise<void> {} + /** + * No-op implementation for development environment. + * In production, this would flush usage data to storage. + */ + async flush(): Promise<void> {}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.changeset/sour-mirrors-accept.md(1 hunks)packages/cli-v3/src/entryPoints/managed-run-controller.ts(1 hunks)packages/cli-v3/src/entryPoints/managed-run-worker.ts(4 hunks)packages/cli-v3/src/entryPoints/managed/controller.ts(5 hunks)packages/core/src/v3/usage/api.ts(1 hunks)packages/core/src/v3/usage/devUsageManager.ts(1 hunks)packages/core/src/v3/usage/noopUsageManager.ts(1 hunks)packages/core/src/v3/usage/types.ts(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
packages/cli-v3/src/entryPoints/managed-run-worker.ts (2)
packages/core/src/v3/workers/index.ts (4)
getEnvVar(4-4)DevUsageManager(19-19)ProdUsageManager(20-20)UsageTimeoutManager(21-21)packages/cli-v3/src/executions/taskRunProcess.ts (1)
timeoutInMs(212-216)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: units / 🧪 Unit Tests
🔇 Additional comments (10)
packages/cli-v3/src/entryPoints/managed-run-controller.ts (1)
5-7: LGTM: Enhanced logging improves observability.Adding debug-level logging is a good addition that will help with troubleshooting issues related to the usage flushing improvements in this PR.
packages/core/src/v3/usage/types.ts (1)
16-16: Appropriate interface extension for usage flushing.The new
flush()method is a well-defined addition to theUsageManagerinterface that supports the PR's goal of improving usage flushing capabilities. This change enforces the contract across all implementations.packages/cli-v3/src/entryPoints/managed/controller.ts (2)
256-257: Good practice: Capturing run ID to ensure consistent logging references.Saving the current run ID before any asynchronous operations ensures log consistency even if the state changes during execution.
310-363: LGTM: Updated debug log calls to use consistent run ID references.All debug logs in this method now consistently use the captured
previousRunIdinstead of the potentially changingthis.runFriendlyId. This ensures that log entries for a specific operation can be properly traced and correlated.packages/core/src/v3/usage/noopUsageManager.ts (1)
8-10: Good addition of flush method to complete the interfaceThe implementation of the
flushmethod as a no-op is consistent with the class's purpose and other methods.packages/core/src/v3/usage/api.ts (1)
47-49: Implementation of flush in UsageAPI looks goodThe
flushmethod delegates properly to the underlying usage manager instance, maintaining the API's consistent design pattern..changeset/sour-mirrors-accept.md (1)
1-6: Appropriate changeset for versioningThe changeset correctly identifies the packages affected by the usage flushing improvements and marks them for a patch version update, which is appropriate for this type of non-breaking enhancement.
packages/cli-v3/src/entryPoints/managed-run-worker.ts (3)
249-254: Good refactoring of usage manager initializationMoving the usage manager initialization into the task execution handler ensures that it's properly initialized with fresh environment variables for each task run.
506-506: Improved usage flushing implementationThe updated
flushUsagemethod now correctly uses the global usage API to flush usage data, which is more consistent with the overall architecture.
548-566: Well-structured initialization functionThe new
initializeUsageManagerfunction properly encapsulates the creation and configuration of usage managers, making the code more maintainable and easier to understand.The function properly handles optional parameters with sensible defaults for
heartbeatIntervalMs.
* add flush to global usage api * enable controller debug logs * initialize usage manager after env overrides * add previous run id to more debug logs * add changeset
* WIP on secret env vars * Editing individual env var values is working * Sort the env vars by the key * Deleting values * Allowing setting secret env vars * Added medium switch style * Many style changes to the env var form * “Copy text” -> “Copy” * Draw a divider between hidden buttons * Env var tweaks * Don’t show Dev:you anymore * Grouping the same env var keys together * Styles improved * Improved styling of edit panel * Fix bun detection, dev flushing, and init command (#1914) * update nypm to support text-based bun lockfiles * add nypm changeset * handle dev flushing failures gracefully * fix path normalization for init.ts * add changesets * chore: remove pre.json after exiting pre mode * init command to install v4-beta packages * Revert "chore: remove pre.json after exiting pre mode" This reverts commit f5694fd. * make init default to cli version for all packages * Release 4.0.0-v4-beta.1 (#1916) * chore: Update version for release (v4-beta) * Release 4.0.0-v4-beta.1 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com> * Both run engines will only lock to versions they can handle (#1922) * run engine v1 will only lock to v1 deployments * run engine v2 will only lock to managed v2 deployments * test: create background worker and deployment with correct engine version * Add links to and from deployments (#1921) * link from deployments tasks to filtered runs view * jump to deployment * don't add version links for dev (yet) * Fix current worker deployment getter (#1924) * only return last v1 deployment in the shared queue consumer * be explicit about only returning managed deployments * Add a docs page for the human-in-the-loop example project (#1919) * Add a docs page for the human-in-the-loop example project * Order guides, example projects and example tasks alphabetically in the docs list * Managed run controller revamp (#1927) * update nypm to support text-based bun lockfiles * fix retry spans * only download debug logs if admin * add nypm changeset * pull out env override logic * use runner env gather helper * handle dev flushing failures gracefully * fix path normalization for init.ts * add logger * add execution heartbeat service * add snapshot poller service * fix poller * add changesets * create socket in constructor * enable strictPropertyInitialization * deprecate dequeue from version * start is not async * dependency injection in prep for tests * add warm start count to all controller logs * add restore count * pull out run execution logic * temp disable pre * add a controller log when starting an execution * refactor execution and squash some bugs * cleanup completed docker containers by default * execution fixes and logging improvements * don't throw afet abort cleanup * poller should use private interval * rename heartbeat service file * rename HeartbeatService to IntervalService * restore old heartbeat service but deprecate it * use the new interval service everywhere * Revert "temp disable pre" This reverts commit e03f417. * add changeset * replace all run engine find uniques with find first * Release 4.0.0-v4-beta.2 (#1928) * chore: Update version for release (v4-beta) * Release 4.0.0-v4-beta.2 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com> * Remove batch ID carryover for non-batch waits (#1930) * add failing test case * do not carry over previous batch id when blocking with waitpoint * delete irrelevant test * Delete project (#1913) * Delete project - Don’t schedule tasks if the project is deleted - Delete queues from the master queues - Add the old delete project UI back in * Mark the project as deleted last * Fix for overriding local variable * Added a todo for deleting env queues * Remove todo * Improve usage flushing (#1931) * add flush to global usage api * enable controller debug logs * initialize usage manager after env overrides * add previous run id to more debug logs * add changeset * For secret env vars, don’t return the value * Added a new env var repository function for getting secrets with redactions * Test task for env vars * Delete heartbeat file, merge mess up --------- Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Saadi Myftija <saadi.myftija@gmail.com>
Some runs could previously run into flush errors, also adds better debug logs
Summary by CodeRabbit