-
Notifications
You must be signed in to change notification settings - Fork 125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add conversion_host_id to miq_request_task #281
Add conversion_host_id to miq_request_task #281
Conversation
It looks like this information is currently being stored in the |
@carbonin That's even more complex. Today, the information stored in options[:transformation_host_id] is the id of a host (class Host). The PR stores the conversion host (class ConversionHost) id. A conversion host has a @agrare what are the possible values for |
@fdupont-redhat if It is a little more complicated than just |
3713049
to
659be65
Compare
@miq-bot add-label gaprindashvili/no, schema, transformation |
@fdupont-redhat 'agrare, carbonin' is an invalid reviewer, ignoring... |
def up | ||
add_column :miq_request_tasks, :conversion_host_id, :bigint | ||
|
||
ServiceTemplateTransformationPlanTask.all.reject { |task| task.options[:transformation_host_id].nil? }.each do |task| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few things here ...
First, I don't think this query will work the way you're intending it to. STI is disabled so I think you would be better off using the base class and specifying the type
in a where
clause.
Also, instead of using reject
here, can we make this a guard clause in the each
block? That will simplify this and prevent us from iterating through the list of tasks twice.
So something like this:
MiqRequestTask.where(:type => "ServiceTemplateTransformationPlanTask").each do |task|
next unless task.options[:transformation_host_id]
...
end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the tip. Updated.
end | ||
|
||
def down | ||
ServiceTemplateTransformationPlanTask.all.select { |task| task.conversion_host.present? }.each do |task| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, use the base class and you don't need to loop through the tasks twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.
|
||
class Host < ActiveRecord::Base | ||
self.inheritance_column = :_type_disabled | ||
has_many :tags, :class_name => "AddConversionHostIdToMiqRequestTasks::Tag" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this relation needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remnant of tests. Removed.
|
||
class ConversionHost < ActiveRecord::Base | ||
self.inheritance_column = :_type_disabled | ||
has_many :service_template_transformation_plan_tasks, :class_name => "AddConversionHostIdToMiqRequestTasks::MiqRequestTask" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This relation isn't being used in the migration, why define it here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct. Removed.
db/migrate/20181001131632_add_conversion_host_id_to_miq_request_tasks.rb
Show resolved
Hide resolved
belongs_to :conversion_host, :class_name => "AddConversionHostIdToMiqRequestTasks::ConversionHost" | ||
end | ||
|
||
class ServiceTemplateTransformationPlanTask < MiqRequestTask |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this can be removed ... See my other comments about STI and using the base class for querying.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
add_column :miq_request_tasks, :conversion_host_id, :bigint | ||
|
||
ServiceTemplateTransformationPlanTask.all.reject { |task| task.options[:transformation_host_id].nil? }.each do |task| | ||
host = Host.find_by(:id => task.options[:transformation_host_id]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this can use find
rather than find_by
. Also this entire block would be way easier to read if task.options[:transformation_host_id]
was stored in a local variable (ideally before the guard clause).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
|
||
ServiceTemplateTransformationPlanTask.all.reject { |task| task.options[:transformation_host_id].nil? }.each do |task| | ||
host = Host.find_by(:id => task.options[:transformation_host_id]) | ||
task.conversion_host = ConversionHost.where(:id => task.options[:transformation_host_id]).first_or_create do |ch| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
find_or_create_by!
would be better over .where(:id => ...).first_or_create
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, if options[:transformation_host_id]
is a reference to the Host
record id
then it can't also be a reference to the ConversionHost
id
.
This where
just looks wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yeah good catch, @fdupont-redhat did you mean to ConversionHost.where(:id => host.conversion_host.try(:id))
or something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed for find_or_create_by!
.
The where was wrong. Now it checks the resource_id
attribute against the host_id
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should check for the type also then, just checking for the id and not the type
add_column :miq_request_tasks, :conversion_host_id, :bigint | ||
|
||
ServiceTemplateTransformationPlanTask.all.reject { |task| task.options[:transformation_host_id].nil? }.each do |task| | ||
host = Host.find_by(:id => task.options[:transformation_host_id]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should probably throw in a next if host.nil?
since you're using host.name
, host.type
, ... below just in case the host was deleted and the task options weren't updated which given they aren't kept in sync by refresh is a distinct possibility.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, if that's a possibility @agrare then we should also keep the find_by
(find
raises if the record doesn't exist).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added test on host.nil?
task.conversion_host = ConversionHost.where(:id => task.options[:transformation_host_id]).first_or_create do |ch| | ||
ch.name = host.name | ||
ch.resource_type = host.type | ||
ch.resource_id = host.id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be able to just do ch.resource = host
here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When using find_or_create_by!
the resource_id
attribute is already set, so now I simply add the resource_type
.
ch.name = host.name | ||
ch.resource_type = host.type | ||
ch.resource_id = host.id | ||
ch.address = host.ipaddress |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're going to delegate ipaddress to the resource per #242 (comment) then we should leave this blank in the conversion_host record. That way if the host ip address changes it will be automatically picked up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch.
it "creates conversion host" do | ||
task = task_stub.create! | ||
host = host_stub.create! | ||
conversion_host = conversion_host_stub.create!( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can just do .create!(:resource => host)
no need to split up the id/type
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are you creating a ConversionHost
in a spec that should be testing that a conversion host is created?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question @carbonin. Probably, because I'm dumb 😄
Removed and spec updated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@agrare It fails if I use :resource => host
. The error message is:
ActiveModel::UnknownAttributeError:
unknown attribute 'resource' for AddConversionHostIdToMiqRequestTasks::ConversionHost.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You probably need to add the relation to the stubbed model
:resource_id => host.id, | ||
:resource_type => host.type | ||
) | ||
task.options = { :transformation_host_id => conversion_host.id } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can pass :options => {:trans....}
into the create!()
and then you don't need the extra save
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
spec/migrations/20181001131632_add_conversion_host_id_to_miq_request_tasks_spec.rb
Outdated
Show resolved
Hide resolved
:resource_type => host.type | ||
) | ||
conversion_host.save | ||
task.conversion_host = conversion_host |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Likewise just pass :conversion_host
in to the .create!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
migrate | ||
|
||
task.reload | ||
expect(task.attributes).not_to include('conversion_host_id') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to check that ActiveRecord deleted the column, if that is broken we have bigger issues
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed.
:resource_id => host.id, | ||
:resource_type => host.type | ||
) | ||
task.options = { :transformation_host_id => conversion_host.id } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, shouldn't this be a Host
reference, not a conversion_host
reference?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
spec/migrations/20181001131632_add_conversion_host_id_to_miq_request_tasks_spec.rb
Show resolved
Hide resolved
eb6b3cb
to
7d45a4e
Compare
|
||
MiqRequestTask.where(:type => 'ServiceTemplateTransformationPlanTask').each do |task| | ||
host_id = task.options[:transformation_host_id] | ||
if host_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know it is a style thing but I usually prefer next if host_id.nil?
over if host_id
and then everything else indented. Reminds me of the java only one return style which brings back bad memories 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Me too. But code climate complains about cognitive complexity when using next if
. Hadn't met that error before and I use quite some next if
or next unless
in Automate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't worry about code climate, ha and rubocop complained for not using next to skip
Can't win :)
7d45a4e
to
04ca79a
Compare
…d from task.options
spec/migrations/20181001131632_add_conversion_host_id_to_miq_request_tasks_spec.rb
Show resolved
Hide resolved
db/migrate/20181001131632_add_conversion_host_id_to_miq_request_tasks.rb
Show resolved
Hide resolved
task.options[:transformation_host_id] = task.conversion_host.resource.id | ||
task.save! | ||
task.conversion_host.id | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should .uniq
also since there will be many tasks with the same conversion host. You can do end.uniq.compact
or ConversionHost.destroy(conversion_host_ids.uniq.compact)
your call
@@ -35,11 +34,12 @@ def up | |||
|
|||
def down | |||
conversion_host_ids = MiqRequestTask.where(:type => 'ServiceTemplateTransformationPlanTask').map do |task| | |||
return if task.conversion_host.nil? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be next if
end | ||
task.save! | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the #up
method looks good to me now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@carbonin can you do another once-over?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking much better, just a few comments on the specs.
let(:conversion_host_stub) { migration_stub(:ConversionHost) } | ||
|
||
migration_context :up do | ||
it "when host doesn't exist" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit, but it would be great if this wrote out what we were testing for.
For example, it "doesn't set the conversion host when the host object doesn't exist"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
expect(task.conversion_host).to be_nil | ||
end | ||
|
||
it "when host exist" do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, it "references the existing Host in the ConversionHost record"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
task.reload | ||
|
||
expect(task.options).to eq(:dummy_key => 'dummy_value') | ||
expect(AddConversionHostIdToMiqRequestTasks::ConversionHost.find_by(:resource_id => conversion_host_id)).to be_nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be conversion_host_stub.find_by...
right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
task.reload | ||
|
||
expect(task.options).to eq(:dummy_key => 'dummy_value') | ||
expect(AddConversionHostIdToMiqRequestTasks::ConversionHost.find_by(:resource => host)).not_to be_nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here, I think this should use the stub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
|
||
expect(task.options).to eq(:dummy_key => 'dummy_value') | ||
expect(AddConversionHostIdToMiqRequestTasks::ConversionHost.find_by(:resource => host)).not_to be_nil | ||
expect(task.conversion_host).to eq(AddConversionHostIdToMiqRequestTasks::ConversionHost.find_by(:resource => host)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Checked commits fabiendupont/manageiq-schema@659be65~...7c3eb63 with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aside from the code comments, the BZ is marked with a Target release of 5.10 (based on hammer branch) and we've already passed the schema freeze for hammer. I hope this PR isn't expected to be backported to hammer.
|
||
class ConversionHost < ActiveRecord::Base | ||
self.inheritance_column = :_type_disabled | ||
belongs_to :resource, :polymorphic => true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you can use polymorphic relations with the stub models. If you do, you'll get something like: :resource_type => MigrationClass::StubModel
which won't be found in production.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah you're right I thought in production it would use the type from the record but appears not
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, who wants to take on changing this? Or should we write a follow up migration to alter all the broken resources to the "production ready" name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can, should be able to just edit this migration?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're going to update the .where
to .in_my_region.find_each
anyway
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Jason would say we need a new migration since developers may have already run this one...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay #284 data migration to fix the resource_type
def up | ||
add_column :miq_request_tasks, :conversion_host_id, :bigint | ||
|
||
MiqRequestTask.where(:type => 'ServiceTemplateTransformationPlanTask').each do |task| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually we prefer find_each
to load records in batches.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, this should probably be scoped with in_my_region
to avoid running the migration on more records than necessary.
db/migrate/20181001131632_add_conversion_host_id_to_miq_request_tasks.rb
Show resolved
Hide resolved
next unless host_id | ||
host = Host.find_by(:id => host_id) | ||
if host.present? | ||
task.conversion_host = ConversionHost.find_or_create_by!(:resource => host) do |ch| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ConversionHost.find_or_initialize_by(:resource_type => "Host", :resource_id => host.id).update_attributes!(.....)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see why we should use find_or_initialize_by
+ update_attributes
over what's here, but yeah it does look like this will break because the host class will be incorrect.
def down | ||
conversion_host_ids = MiqRequestTask.where(:type => 'ServiceTemplateTransformationPlanTask').map do |task| | ||
next if task.conversion_host.nil? | ||
task.options[:transformation_host_id] = task.conversion_host.resource.id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe task.conversion_host.resource_id
to avoid loading resource
(which I think would fail anyway)?
:type => 'ServiceTemplateTransformationPlanTask', | ||
:options => { :dummy_key => 'dummy_value', :transformation_host_id => conversion_host_id } | ||
) | ||
host.destroy! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should be able to generate a record id (in the current region) without creating and destroying a record
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how you could do that without querying the sequences. I think it would have been nice to create the host in a before block, but other than that I think this is fine.
end | ||
|
||
def down | ||
conversion_host_ids = MiqRequestTask.where(:type => 'ServiceTemplateTransformationPlanTask').map do |task| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also be scoped with in_my_region
task.reload | ||
|
||
expect(task.options).to eq(:dummy_key => 'dummy_value') | ||
expect(conversion_host_stub.find_by(:resource => host)).not_to be_nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't this eq(something)
?
Fix an invalid polymorphic resource_type from ManageIQ#281 which resulted in the type being "AddConversionHostIdToMiqRequestTasks::Host" instead of "Host".
Fix an invalid polymorphic resource_type from ManageIQ#281 which resulted in the type being "AddConversionHostIdToMiqRequestTasks::Host" instead of "Host".
Fix an invalid polymorphic resource_type from ManageIQ#281 which resulted in the type being "AddConversionHostIdToMiqRequestTasks::Host" instead of "Host".
Fix an invalid polymorphic resource_type from ManageIQ#281 which resulted in the type being "AddConversionHostIdToMiqRequestTasks::Host" instead of "Host".
…d_to_transformation_task Add conversion_host_id to miq_request_task (cherry picked from commit eb7e077) https://bugzilla.redhat.com/show_bug.cgi?id=1634029
Backport approved by Engineering/QE. Hammer backport details:
|
The
ServiceTemplateTransformationPlanTask
class relies on a conversion host to actually run the migration. The conversion host is modeled inConversionHost
class. From a database point of view, theServiceTemplateTransformationPlanTask
class is stored inmiq_request_tasks
table, because it's a sub-sub-class ofMiqRequestTask
. This PR adds aconversion_host_id
column tomiq_request_tasks
to set the relationship.Associated RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1634029