Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miq shortcut seeding #14915

Merged
merged 2 commits into from
May 3, 2017
Merged

Miq shortcut seeding #14915

merged 2 commits into from
May 3, 2017

Conversation

kbrock
Copy link
Member

@kbrock kbrock commented Apr 26, 2017

  • fix MiqShortcut fixture (miq_shortcut.csv) to remove duplicate name.
  • Now populate both cloud / snapshots and cloud / volumes into the database
  • Warn when there are problems with the fixture.
  • remove N+1 from MiqShortcut#seed

UPDATE: #14914 took care of the duplicate shortcut fixture

initial seed population

ms bytes objects query query ms rows comments
282.0 8,270,837 83,793 217 147.2 54 before-3
219.9 6,427,604 63,869 163 107.7 after-4
22% 22% 24% 25% 27% 100% diff

seed update

ms bytes objects query query ms rows comments
102.3 2,752,629 29,810 55 50.6 108 before-update-3
29.7 917,063 10,043 1 3.0 54 after-update-4
71.0% 67% 66% 98% 94% 50% diff

Copy link
Member

@NickLaMuro NickLaMuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix seems fine, but noticed a few things that may or may not need to be addressed. Possibly someone with more familiarity around this might be able to comment.

Also, not sure why this is data that is in the DB in the first place, but that is not my concern... 🍵 🐸

@@ -20,8 +26,7 @@ def self.seed
end
end

all.each do |rec|
next if names.include?(rec.name)
db_data.each do |name, rec|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this block is unnecessary as you are now doing the delete above? Or am I reading this wrong?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only delete the records from the local list.
The records left in db_data are the records that did not have a corresponding seed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotcha.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I see what you are doing now, thanks for the clarification. Looks good. 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, that delete was causing duplicate records in the database. Took an approach more similar to what was there before.

if db_records.size != db_data.size
names = db_records.group_by(&:name).select { |n, v| v.size > 1 }.map(&:first)
_log.warn("Duplicate seeds for names: #{names.join(",")}")
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this warning is only going to go off when re-running the MiqShortcut.seed after the data has been entered, correct? So this might not catch things right away. Should we maybe be running this logic on the seed_data keys instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, sounds good to me

@kbrock
Copy link
Member Author

kbrock commented Apr 27, 2017

fixed to no longer keep adding extra records when the seed data is bad.

ms bytes objects query query ms rows comments
42.7 924,383 10,147 1 3.0 54 no name-1

@chessbyte chessbyte assigned jrafanie and gtanzillo and unassigned jrafanie Apr 27, 2017
@miq-bot
Copy link
Member

miq-bot commented Apr 27, 2017

This pull request is not mergeable. Please rebase and repush.

@kbrock
Copy link
Member Author

kbrock commented Apr 28, 2017

/cc @isimluk we'll get the seeds down to no time flat before you know it ;)

@miq-bot
Copy link
Member

miq-bot commented Apr 28, 2017

Checked commits kbrock/manageiq@20a2816~...1ec3644 with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
1 file checked, 0 offenses detected
Everything looks fine. 🍰

s[:sequence] = index
rec = find_by(:name => s[:name])
rec = db_data[s[:name]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about db_data.delete(s[:name]) instead?

Then we can remove next if from the last loop (making it quicker/no-op in most cases).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed on pm with @kbrock. Not a blocker for this pr.

Copy link
Member

@isimluk isimluk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@isimluk
Copy link
Member

isimluk commented May 3, 2017

@kbrock I am afraid we may need a bz at this point, if you want this to be fine/yes.

@isimluk isimluk assigned isimluk and unassigned gtanzillo May 3, 2017
@isimluk isimluk added this to the Sprint 60 Ending May 8, 2017 milestone May 3, 2017
@isimluk isimluk merged commit 2a4935d into ManageIQ:master May 3, 2017
@kbrock kbrock added fine/no and removed fine/yes labels May 3, 2017
@kbrock
Copy link
Member Author

kbrock commented May 3, 2017

meh, good call @isimluk - just saying no.
Low risk, ok gain.

@kbrock kbrock deleted the miq_shortcut branch May 3, 2017 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants