Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running GoodJob in production with systemd throws an error due to a wrong communication with WatchDog #1172

Closed
tagrudev opened this issue Dec 10, 2023 · 4 comments

Comments

@tagrudev
Copy link
Contributor

Hello, I am trying to run good_job in production using the systemd service example that was added in #1029, unfortunately there's a problem with the communication between good_job and watchdog.

My setup

ruby 3.2.2
good_job 3.21.2 
rails 7.1.1
ubuntu 22.04

My goodjob service running as a user service

Description=GoodJob Background Job Processor
After=postgresql.service network.target

[Service]
Type=notify
WatchdogSec=5s

WorkingDirectory=/my/working/directory

Environment=RAILS_ENV=production
ExecStart=/bin/bash -lc 'exec /usr/local/rbenv/shims/bundle exec good_job start'

RestartSec=1s
Restart=always

StandardOutput=journal
StandardError=journal
SyslogIdentifier=goodjob

[Install]
WantedBy=default.target

This results in

Screenshot 2023-12-10 at 22 46 09

Goodjob is running, but I think there's a problem with the communication with Watchdog.
I think the problem comes from https://github.com/bensheldon/good_job/pull/1029/files#diff-121a5f6ab045b7079f271a0b111a46097545d1d13550bc5b1394d3e7adb94bfbR178

Looking at the other methods it's missing return unless exception is that the problem? adding this locally fixes the issue, but I am not sure if the watchdog communication is ok.

@tagrudev
Copy link
Contributor Author

I guess

ActiveSupport::Notifications.instrument("systemd_watchdog_error.good_job", { error: thread_error })
GoodJob._on_thread_error(thread_error) if thread_error

This part should be

if thread_error
  ActiveSupport::Notifications.instrument("systemd_watchdog_error.good_job", { error: thread_error })
  GoodJob._on_thread_error(thread_error)
end

@bensheldon
Copy link
Owner

@tagrudev Nice find! Can you make a PR for that?

@tagrudev
Copy link
Contributor Author

Hey @bensheldon - yes, already did #1173

@tagrudev
Copy link
Contributor Author

I am closing this issue since the PR is now merged.

@github-project-automation github-project-automation bot moved this from Inbox to Done in GoodJob Backlog v2 Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

2 participants