bug: IntelliJ plugin filled up disk with agent files #439
Comments
|
Hello @gabtorre 👋 I am sorry you suffer from this issue. Thanks for the detailed report 💟 I understand that the plugin works correctly for you now (after the manual cleanup). We've seen a similar issue recently: #315 Are you able to point the most common message in the logs? Is it: Hopefully, the newest version is not responsible for the wordy logging. We will revisit this issue and verify it. |
|
Hey @mkondratek, In this case, rather than log files, there were thousands of |
|
Thank you for reporting! This is a serious issues that we should prioritize fixing asap. This is not a recent regression, this behavior has been latent in the plugin since September. The root problem is that we never delete the temporary file that's created here Adding logic to delete the file after the process finishes running should be an easy fix so I think it's safe to estimate that this will get resolved before the February 15th launch. |
Addresses #439 Previously, the JetBrains plugin created a new temporary file every time we started a new agent process and only deleted this file when the IDE shut down. This buggy behavior has existed since Septemeber and we haven't received user complaints about it, likely because the plugin was conservative about restarting the agent process. Recent changes introduced retry logic where we could start a new agent process in a loop, and never delete the underlying binary. Some users have reported that this bug caused the plugin to take up unreasonable amounts of disk space. I'm sorry for users who were impacted by this behavior. This PR addresses the problem with a new callback hook that deletes the agent binary after the agent process finishes running. This change means that we never have more than one agent binary sitting on the disk per IntelliJ project. This should prevent the catastrophic situation some users have reported. I manually tested this change by adding a println to when the file gets deleted, started IntelliJ, triggered the "Restart Cody Agent" action and confirmed that the file got deleted.
Addresses #439 Previously, the JetBrains plugin created a new temporary file every time we started a new agent process and only deleted this file when the IDE shut down. This buggy behavior has existed since Septemeber and we haven't received user complaints about it, likely because the plugin was conservative about restarting the agent process. Recent changes introduced retry logic where we could start a new agent process in a loop, and never delete the underlying binary. Some users have reported that this bug caused the plugin to take up unreasonable amounts of disk space. I'm sorry for users who were impacted by this behavior. This PR addresses the problem with a new callback hook that deletes the agent binary after the agent process finishes running. This change means that we never have more than one agent binary sitting on the disk per IntelliJ project. This should prevent the catastrophic situation some users have reported. I manually tested this change by adding a println to when the file gets deleted, started IntelliJ, triggered the "Restart Cody Agent" action and confirmed that the file got deleted.
Addresses #439 Previously, the JetBrains plugin created a new temporary file every time we started a new agent process and only deleted this file when the IDE shut down. This buggy behavior has existed since Septemeber and we haven't received user complaints about it, likely because the plugin was conservative about restarting the agent process. Recent changes introduced retry logic where we could start a new agent process in a loop, and never delete the underlying binary. Some users have reported that this bug caused the plugin to take up unreasonable amounts of disk space. I'm sorry for users who were impacted by this behavior. This PR addresses the problem with a new callback hook that deletes the agent binary after the agent process finishes running. This change means that we never have more than one agent binary sitting on the disk per IntelliJ project. This should prevent the catastrophic situation some users have reported. ## Test plan I manually tested this change by adding a println to when the file gets deleted, started IntelliJ, triggered the "Restart Cody Agent" action and confirmed that the file got deleted. <!-- All pull requests REQUIRE a test plan: https://sourcegraph.com/docs/dev/background-information/testing_principles Why does it matter? These test plans are there to demonstrate that are following industry standards which are important or critical for our customers. They might be read by customers or an auditor. There are meant be simple and easy to read. Simply explain what you did to ensure your changes are correct! Here are a non exhaustive list of test plan examples to help you: - Making changes on a given feature or component: - "Covered by existing tests" or "CI" for the shortest possible plan if there is zero ambiguity - "Added new tests" - "Manually tested" (if non trivial, share some output, logs, or screenshot) - Updating docs: - "previewed locally" - share a screenshot if you want to be thorough - Updating deps, that would typically fail immediately in CI if incorrect - "CI" - "locally tested" -->
|
This PR #566 should fix the root issue where the plugin is restarting a new (crashing) process in a loop. It's 10pm here right now so we will wait until Monday morning to make this fix available in a pre-release (aka. "nightly channel"). We will be cutting a new release (previously referred to as "stable channel") no later than Tuesday. |
Fixes #439 ## Changes * Removed agent recovery loop, recovery is now on demand * Added a bit of defensive code in case of unexpected exceptions * Added few timeouts in places where we shouldn't wait forever * Improved robustness of the shutdown * Added additional safeguard for old agent binary file deletion ## Test plan I added Thread that was destroying agent process in a fixed time after startup. I tested durations from 100ms to 10s. Crashes after 200ms (which simulated complete agent failure) were not spamming the logs with too much info, and were not leaving trash on the disk. Delay around 4-5s was enough for the chat to recover and give really short answer. It was able to recover infinite amount of times, which means we could survive even repeated crashes or hangs of the agent (if that would ever be the case).
Installation Information
Cody plugin version 5.2.20247
IntelliJ IDEA 2023.3.2 (Ultimate Edition)
Build #IU-233.13135.103, built on December 20, 2023
Runtime version: 17.0.9+7-b1087.9 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 14.3
Describe the bug
The plugin seemed stuck in a loop, leading to excessive disk usage. Additionally, it created large temporary copies of the agent file during this process, with many of these logs:
To mitigate the issue, the user cleared all the files associated with the plugin, restarted the macOS, and then reenabled the plugin. After these actions, the plugin behaves correctly, with only two copies of the agent created.
It seems like the plugin may not clean up after itself properly, as indicated by the excessive number of temporary copies created during the loop.
Expected behavior
The plugin cleans up temporary files appropriately to prevent excessive disk usage.
Additional context (logs, images, etc)
Additional context: https://sourcegraph.slack.com/archives/C02842Z6YAK/p1706567669821759
The text was updated successfully, but these errors were encountered: