-
Notifications
You must be signed in to change notification settings - Fork 898
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tool to remove grouping from report results #17589
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
#!/usr/bin/env ruby | ||
require 'trollop' | ||
|
||
opts = Trollop.options do | ||
banner "Remove extras[:grouping] from MiqReportResult#report column.\n\nUsage: ruby #{$PROGRAM_NAME} [options]\n\nOptions:\n\t" | ||
opt :dry_run, "For testing, rollback any changes when the script exits.", :short => :none, :default => false | ||
opt :batch_size, "Limit memory usage by process this number of report results at a time.", :default => 50 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does 50 batch size make sense? Considering the size of these, I wanted extra queries instead of possibly bloating the memory of the process. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think so. The deserialization is probably going to hit us harder than the queries time themselves. I don't see this as being an issue with |
||
opt :count, "Stop checking after this number of report results.", :default => 0 | ||
end | ||
|
||
puts "Using options: #{opts.inspect}\n\n" | ||
|
||
if defined?(Rails) | ||
puts "Warning: Rails is already loaded! Please do not invoke using rails runner. Exiting with help text.\n\n" | ||
Trollop.educate | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @NickLaMuro changed the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Works for me. I didn't realize what you had was standard for the |
||
end | ||
|
||
# Load rails after trollop options are set. No one wants to wait for -h. | ||
require File.expand_path('../config/environment', __dir__) | ||
|
||
# Wrap all changes in a transaction and roll them back if dry-run or an error occurs. | ||
ActiveRecord::Base.connection.begin_transaction(:joinable => false) | ||
|
||
if opts[:dry_run] | ||
puts "Running in dry-run, all changes will be rolled back when complete." | ||
|
||
at_exit do | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just because I am not sure: I assume that (Nick could probably just go test this in a ruby script himself...) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah, good point. I'll test that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No need, I already did: $ ruby -e "at_exit { puts 'foo' }; exit 42"
foo
$ echo $?
42 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did several tests of dry run so I'm pretty confident it works. It's a little messy how I'm doing it right now but I can't think of a better way to handle the begin/commit/rollback for dry-run/error/exit |
||
ActiveRecord::Base.connection.rollback_transaction | ||
end | ||
end | ||
|
||
start = Time.now.utc | ||
total = 0 | ||
fixed = 0 | ||
|
||
MiqReportResult.find_each(:batch_size => opts[:batch_size]).with_index do |rr, i| | ||
begin | ||
break if opts[:count].positive? && i == opts[:count] | ||
total += 1 | ||
|
||
next if rr.report.nil? || rr.report.extras.nil? | ||
|
||
if rr.report.extras.key?(:grouping) | ||
rr.report.extras.except!(:grouping) | ||
rr.save! | ||
if rr.reload.report.extras.key?(:grouping) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When would this happen? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This was here before I changed line 43 from "doesn't need fixing" |
||
puts "MiqReportResult: #{rr.id} could NOT be fixed" | ||
else | ||
puts "MiqReportResult: #{rr.id} fixed" | ||
fixed += 1 | ||
end | ||
else | ||
puts "MiqReportResult: #{rr.id} doesn't need fixing" | ||
end | ||
rescue => err | ||
puts "\nWarning: Rolling back all changes since an error occurred on MiqReportResult with id: #{rr.try(:id)}: #{err.message}" | ||
ActiveRecord::Base.connection.rollback_transaction unless opts[:dry_run] | ||
exit 1 | ||
end | ||
end | ||
|
||
ActiveRecord::Base.connection.commit_transaction unless opts[:dry_run] | ||
puts "\nProcessed #{total} rows. #{fixed} were fixed. #{Time.now.utc - start} seconds" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
require 'optparse'
for lyfe!!1!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm waiting for @kbrock to rename trollor to optimal or whatever we decided. 🕐
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or trollop
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I always favor stdlib things when possible, so that was what my comment was all about.