Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use standard thread library #580

Merged
merged 18 commits into from
Feb 14, 2025
Merged

Use standard thread library #580

merged 18 commits into from
Feb 14, 2025

Conversation

jan-janssen
Copy link
Member

@jan-janssen jan-janssen commented Feb 14, 2025

Summary by CodeRabbit

  • Refactor
    • Simplified the internal threading mechanism by adopting the standard thread implementation for task execution.
  • Chores
    • Removed the deprecated custom thread functionality and streamlined internal thread management.
  • Tests
    • Updated unit tests to align with the new threading approach, ensuring consistent behavior during task execution.
    • Deleted outdated tests related to the removed custom thread class.
    • Enhanced error handling in tests to ensure proper exception propagation when futures are resolved.

Copy link
Contributor

coderabbitai bot commented Feb 14, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

The changes replace the custom RaisingThread with Python’s standard Thread class across multiple modules. Type annotations, method signatures, and conditional checks have been updated accordingly. Additionally, the custom threading implementation has been removed from both production code and tests, with corresponding import adjustments made.

Changes

File(s) Change Summary
executorlib/base/executor.py, executorlib/cache/executor.py, executorlib/interactive/executor.py, executorlib/interactive/shared.py Replaced custom RaisingThread with standard Thread; updated type annotations, method signatures, and thread instantiation.
executorlib/standalone/__init__.py, executorlib/standalone/thread.py Removed RaisingThread from the public API; deleted the custom RaisingThread implementation file.
tests/test_cache_executor_serial.py, tests/test_dependencies_executor.py, tests/test_shared_thread.py Updated tests to use Thread instead of RaisingThread; added necessary imports and removed tests specific to the custom thread behavior.

Possibly related PRs

  • Debug error handling #577: The changes in the main PR, which involve replacing the RaisingThread class with the standard Thread class in the executorlib/base/executor.py, are related to the modifications in the retrieved PR that also replace RaisingThread with Thread in the executorlib/interactive/shared.py, indicating a direct connection in the threading implementation across both PRs.

Poem

I'm a rabbit hopping through code so spry,
With threads now standard, I cheer and sigh.
No custom quirks, just Python's beat,
Each line of change is oh-so-sweet.
Leaping through commits with joyful art,
My carrot dreams dance in every part.
Hoppy coding ahead with a twitch of my heart!


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7353ef5 and d4afda7.

📒 Files selected for processing (7)
  • executorlib/base/executor.py (4 hunks)
  • executorlib/cache/shared.py (1 hunks)
  • executorlib/interactive/shared.py (7 hunks)
  • tests/test_cache_executor_serial.py (5 hunks)
  • tests/test_dependencies_executor.py (6 hunks)
  • tests/test_local_executor.py (4 hunks)
  • tests/test_shell_executor.py (4 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (3)
executorlib/interactive/shared.py (3)

92-92: 🛠️ Refactor suggestion

Update type hint to use Thread instead of RaisingThread.

The type hint in the method signature still references the removed RaisingThread class.

-def _set_process(self, process: list[RaisingThread]):  # type: ignore
+def _set_process(self, process: list[Thread]):
🧰 Tools
🪛 Ruff (0.8.2)

92-92: Undefined name RaisingThread

(F821)


96-97: 🛠️ Refactor suggestion

Update docstring to use Thread instead of RaisingThread.

The docstring still references the removed RaisingThread class.

-    Args:
-        process (List[RaisingThread]): The process for the executor.
+    Args:
+        process (List[Thread]): The process for the executor.

564-565: 🛠️ Refactor suggestion

Update docstring to use Thread instead of RaisingThread.

The docstring still references the removed RaisingThread class.

-        RaisingThread, dict: thread for communicating with the python process which is executing the function and
+        Thread, dict: thread for communicating with the python process which is executing the function and
🧹 Nitpick comments (2)
executorlib/base/executor.py (1)

147-155: Update docstring to reflect Thread usage.

The docstring still references RaisingThread instead of Thread.

-            process (RaisingThread): The process for the executor.
+            process (Thread): The process for the executor.
tests/test_cache_executor_serial.py (1)

84-93: Add test coverage for exception handling.

With the switch to standard Thread, we should add test cases to verify that exceptions in worker threads are properly handled and don't silently fail.

Consider adding a test case like this:

def test_thread_exception_handling(self):
    with FileExecutor() as exe:
        def failing_function():
            raise ValueError("Test exception")
        future = exe.submit(failing_function)
        with self.assertRaises(ValueError):
            future.result()

Also applies to: 125-134, 166-175

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8dc0584 and 42dc067.

📒 Files selected for processing (9)
  • executorlib/base/executor.py (3 hunks)
  • executorlib/cache/executor.py (2 hunks)
  • executorlib/interactive/executor.py (2 hunks)
  • executorlib/interactive/shared.py (4 hunks)
  • executorlib/standalone/__init__.py (0 hunks)
  • executorlib/standalone/thread.py (0 hunks)
  • tests/test_cache_executor_serial.py (4 hunks)
  • tests/test_dependencies_executor.py (4 hunks)
  • tests/test_shared_thread.py (0 hunks)
💤 Files with no reviewable changes (3)
  • tests/test_shared_thread.py
  • executorlib/standalone/thread.py
  • executorlib/standalone/init.py
🔇 Additional comments (7)
executorlib/interactive/executor.py (1)

43-54: LGTM! Thread instantiation looks correct.

The thread instantiation maintains the same functionality while using the standard Thread class.

executorlib/cache/executor.py (1)

66-80: LGTM! Thread instantiation looks correct.

The thread instantiation maintains the same functionality while using the standard Thread class.

tests/test_dependencies_executor.py (3)

6-6: LGTM! Import statement updated correctly.

The import statement has been updated to use the standard Thread class from the threading module.


93-101: LGTM! Thread instantiation updated correctly.

The thread instantiation has been updated to use the standard Thread class, maintaining the same target function and keyword arguments.


149-157: LGTM! Remaining thread instantiations updated correctly.

The remaining thread instantiations have been updated to use the standard Thread class, maintaining the same functionality.

Also applies to: 207-215

executorlib/interactive/shared.py (2)

8-8: LGTM! Import statement updated correctly.

The import statement has been updated to use the standard Thread class from the threading module.


152-156: LGTM! Thread instantiations updated correctly.

All thread instantiations have been updated to use the standard Thread class, maintaining the same functionality.

Also applies to: 208-211, 592-595

@@ -29,7 +29,7 @@ def __init__(self, max_cores: Optional[int] = None):
cloudpickle_register(ind=3)
self._max_cores = max_cores
self._future_queue: Optional[queue.Queue] = queue.Queue()
self._process: Optional[Union[RaisingThread, list[RaisingThread]]] = None
self._process: Optional[Union[Thread, list[Thread]]] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify exception handling behavior with standard Thread.

The switch from RaisingThread to standard Thread might affect how exceptions are propagated from worker threads to the main thread. The standard Thread class doesn't automatically propagate exceptions to the main thread.


🏁 Script executed:

#!/bin/bash
# Search for exception handling patterns in the codebase
rg -A 5 "try:|except:|finally:" --type py

Length of output: 11463


🏁 Script executed:

#!/bin/bash
# Search for any remaining references to RaisingThread in the repository
rg "RaisingThread"

Length of output: 477


Attention: Inconsistent Thread Type References May Affect Exception Propagation

The update in executorlib/base/executor.py correctly changes the _process annotation to use the standard Thread. However, several parts of the codebase still reference RaisingThread (e.g., docstrings and method signatures in executorlib/base/executor.py and executorlib/interactive/shared.py). This inconsistency could lead to confusion regarding exception propagation—since the standard Thread does not automatically propagate exceptions to the main thread as RaisingThread might have.

  • Action Items:
    • Ensure that exception handling with the standard Thread is explicitly managed if propagating errors is required.
    • Update all references (docstrings, type annotations, and method signatures) from RaisingThread to Thread (or alternatively, reintroduce RaisingThread functionality where needed) for consistency.

@jan-janssen jan-janssen marked this pull request as draft February 14, 2025 11:00
@jan-janssen jan-janssen linked an issue Feb 14, 2025 that may be closed by this pull request
Copy link

codecov bot commented Feb 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.91%. Comparing base (c037c78) to head (d4afda7).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #580      +/-   ##
==========================================
- Coverage   95.96%   95.91%   -0.06%     
==========================================
  Files          26       25       -1     
  Lines        1191     1174      -17     
==========================================
- Hits         1143     1126      -17     
  Misses         48       48              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jan-janssen jan-janssen marked this pull request as ready for review February 14, 2025 15:51
@jan-janssen jan-janssen merged commit ff49de3 into main Feb 14, 2025
29 of 30 checks passed
@jan-janssen jan-janssen deleted the standard_thread branch February 14, 2025 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[testing] Clean up Exception handling in threads
1 participant