Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop support for forking workers, use spawn by default #19556

Merged
merged 1 commit into from
Nov 26, 2019

Conversation

jrafanie
Copy link
Member

Make systemd optional on systemd enabled systems

Api/web service worker needs ui-classic as it can try to read an existing UI session, which
can contain serialized classes from ui-classic.

Previously, we tried removing fork here: #16130
It was reverted here: #16154

Some of the followups needed to fix the original problems including passing down
ems_id to per ems workers, resolved in:
#16199 and
#18648

At this point, things should just work.

# TODO: The api process now looks at the existing UI session as of: https://github.com/ManageIQ/manageiq-api/pull/543
# ui-classic should not be but is serialializing its classes into session, so we need to have access to them for deserialization
# sandboxes;FC:-ActiveSupport::HashWithIndifferentAccess{I"dashboard;FC;q{I"perf_options;FS:0ApplicationController::Performance::Options$typ0:daily_date0:hourly_date0: days0:
%w[manageiq_default ui_dependencies graphql_api]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skateman See above... because we serialize UI classic classes into session and the web service / api worker can read this existing UI session, it also need to know about UI classic classes ☹️

For now, I'm adding ui dependencies to the web service worker but we shouldn't have to pull in all of ui classic for this.

@jrafanie jrafanie force-pushed the spawn_instead_of_fork branch 2 times, most recently from 2baf4a6 to bd32890 Compare November 25, 2019 19:19
if ENV['MIQ_SPAWN_WORKERS'] || !Process.respond_to?(:fork)
start_runner_via_spawn
elsif systemd_worker?
if ENV['MIQ_SYSTEMD_WORKERS'] && systemd_worker?
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agrare @carbonin feel free to suggest a better name. Before spawn was an option, now, it's the default so I needed a way to opt-in to using systemd workers instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 that's good name

def self.build_command_line(guid, ems_id = nil)
raise ArgumentError, "No guid provided" unless guid

require 'awesome_spawn'
cmd = "#{Gem.ruby} #{runner_script}"
cmd = "nice #{nice_increment} #{cmd}" if ENV["APPLIANCE"]
cmd = "nice -n #{nice_increment} #{cmd}" if ENV["APPLIANCE"]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure when nice went to -n format but I think it's been a long time, since we went to fork, so this is why this is changed here.

#
# Because we fork workers from the server, if we don't disconnect the pool,
# any call to reconnect! on a connection will cause the worker's connection
# to have the server's application_name.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, this is still needed in general because the first connection to the db will happen before we can set the application name, so it defaults to the rails application name. The comment is no longer relevant though.

@jrafanie
Copy link
Member Author

@agrare I did some "light" refresh / event worker testing on this. I think you solved the ems_id problem we originally had in #16130

Is there anything else that should be tested with this?

@jrafanie jrafanie changed the title [WIP] Drop support for forking workers, use spawn by default Drop support for forking workers, use spawn by default Nov 25, 2019
@miq-bot miq-bot removed the wip label Nov 25, 2019
Make systemd optional on systemd enabled systems

Api/web service worker needs ui-classic as it can try to read an existing UI session, which
can contain serialized classes from ui-classic.

Previously, we tried removing fork here: ManageIQ#16130
It was reverted here: ManageIQ#16154

Some of the followups needed to fix the original problems including passing down
ems_id to per ems workers, resolved in:
ManageIQ#16199 and
ManageIQ#18648

At this point, things should just work.
@miq-bot
Copy link
Member

miq-bot commented Nov 25, 2019

Checked commit jrafanie@a03c912 with ruby 2.5.5, rubocop 0.69.0, haml-lint 0.20.0, and yamllint 1.10.0
6 files checked, 0 offenses detected
Everything looks fine. 👍

@agrare
Copy link
Member

agrare commented Nov 25, 2019

I did some "light" refresh / event worker testing on this. I think you solved the ems_id problem we originally had in #16130

Is there anything else that should be tested with this?

@jrafanie if refresh and event catcher workers both work then all of the per_ems provider workers should work.

Copy link
Member

@carbonin carbonin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Down with fork, long live spawn 🔥 🍴

@carbonin carbonin self-assigned this Nov 26, 2019
@carbonin carbonin merged commit c22d88e into ManageIQ:master Nov 26, 2019
@carbonin carbonin added this to the Sprint 126 Ending Dec 9, 2019 milestone Nov 26, 2019
@jrafanie jrafanie deleted the spawn_instead_of_fork branch December 11, 2019 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants