Change migration progress log to be more verbose

[janezpodhostnik]

Changed the progress log:

• it now logs once every 10% or once every minute if the progress is slower
• it can be called concurrently
• it prints an eta that is estimated from the current progress and remaining work
• log.Logger is now the first parameter
• to add progress you call the progress logger with the difference, not the current total progress

[codecov-commenter]

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (fbfecc1) 56.25% compared to head (d36df28) 56.30%.
Report is 450 commits behind head on master.

Files	Patch %	Lines
module/util/log.go	83.33%	7 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5122      +/-   ##
==========================================
+ Coverage   56.25%   56.30%   +0.04%     
==========================================
  Files         974      976       +2     
  Lines       90742    91804    +1062     
==========================================
+ Hits        51047    51687     +640     
- Misses      35889    36286     +397     
- Partials     3806     3831      +25     

Flag	Coverage Δ
unittests	56.30% <83.87%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fxamacker]

Looks good and logging is definitely useful during migration.

I left some comments about locking/concurrency overhead. Some are more worthwhile to optimize than others.

Maybe instead of calling logging function with every operation, we can call logging with batch of operations to reduce locking overhead that is incurred even when logging isn't sampled under the hood.

[fxamacker]

Suggested change

	// LogProgress takes a total and return function such that when called adds the given
	// number to the progress and logs the progress every 10% or every 60 seconds whichever
	// comes first.
	// LogProgress takes a LogProgressConfig and return function such that when called adds the given
	// number to the progress and logs the progress every specified seconds or % completed
	// (whichever comes first) as specified in LogProgressConfig.

[fxamacker]

nit: maybe rename sample to sampled.

Suggested change

	sample := s.basicSampler.Sample(lvl)
	sampled := s.basicSampler.Sample(lvl)

[fxamacker]

Maybe optimize by calling logGrouping() every n payloads (not every payload) because logGrouping() has locking overhead.

[fxamacker]

Maybe optimize by calling logAccount() every n account migrations because logAccount() has locking overhead.

[fxamacker]

Same as above.

[fxamacker]

It seems this function is called for reading and writing every node during checkpointing. Maybe check performance impact with logging and locking overhead here and optimize if needed.

[janezpodhostnik]

The comments about locking will be addressed here #5140