Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Job Management, Optimize Queries, and Introduce Thread-Safe Features #512

Merged
merged 6 commits into from
Nov 26, 2024

Conversation

brand-it
Copy link
Owner

New Features

  1. Thread Safety Enhancements

    • Implemented concurrently method in ApplicationWorker to limit concurrent jobs (e.g., UploadWorker is limited to 2 concurrent jobs).
    • Added Backgrounder for managing jobs, providing a centralized job management approach.
  2. Serialization and Scope Addition

    • Introduced serialize :segment_map in DiskTitle to store structured data as JSON.
    • Added a scope :sort_by_segment_map to sort DiskTitle by its segment_map.
  3. Enhanced Listeners

    • Created MkvDiskLoadListener and integrated it into services like CreateDisksService and workers such as LoadDiskWorker.
  4. Service Layer Enhancements

    • Introduced a listener option in CreateDisksService for flexible subscription handling.

Improvements

  1. Database Query Optimizations

    • Used .includes in multiple components and controllers to reduce N+1 query issues:
      • Added preloads for related models like Video, DiskTitles, and Episodes.
      • Modified query scopes for MoviesController, SeasonsController, and TvsController to use .includes for related associations.
    • Optimized uploaded_recently_video_blobs and uploadable_video_blobs in UploadProcessComponent.
  2. Job Handling

    • Replaced Job.sort_by_created_at.active.find_by calls with Backgrounder.managers.find for better job lookup performance.
  3. Cleaner Code

    • Replaced redundant methods like find_or_initalize_disk with find_or_initialize_disk.
    • Corrected parameter names (e.g., from job: to listener: in service calls).
  4. Improved UX

    • Added compact and human-readable labels using .compact_blank for DiskTitle#to_label.
  5. Consistent Duration Display

    • Standardized the use of distance_of_time_in_words with explicit options for better readability in views.

Bug Fixes

  1. Typographical Errors

    • Fixed typos like segement_map to segment_map.
  2. Corrected Scopes and Relations

    • Fixed incorrect method calls (e.g., episode.video_blobs.any? corrected to episode.uploaded_video_blobs.any?).
    • Corrected scope usage in components and services to ensure proper filtering and ordering.
  3. Initialization and Caching Issues

    • Resolved issues with uninitialized variables by ensuring proper lazy-loading (@episode_numbers in VideoBlob).
    • Fixed caching in Job#name_constant to avoid redundant constantization.
  4. Concurrency Bugs

    • Removed thread-related class variables (e.g., @@threads) in ApplicationWorker and replaced them with centralized job handling in Backgrounder.
  5. Job Argument Handling

    • Ensured arguments in Job#worker are consistently symbolized to avoid errors during initialization.

@brand-it brand-it enabled auto-merge (squash) November 26, 2024 17:45
@brand-it brand-it merged commit 9ca02bc into main Nov 26, 2024
2 checks passed
@brand-it brand-it deleted the add-workers-table branch November 26, 2024 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant