Estimate time providing unrealistic estimates

[rjkennedy98]

In certain cases, because of changes in URIs being processed or because of changes in server performance, CORB job is providing an inaccurate ETC.
exempli gratia:
INFO: completed 1050801/1500000, 263 tps(avg), 19 tps(cur), ETC 00:28:25, 8 active threads.

[hansenmc]

Would you expect that the current tps be applied in the calculation, to provide more weight than the overall average?

For example, calculate the ETC from both the average current tps rates with the rates you provided:
(avg ETC 28 minutes + current ETC 390 minutes)/2 = more realistic ETC of 209 minutes?

[bbandlamudi]

We were thinking we add another tps to the progress message i.e., average of last 10 or 5 (can make it configurable) tps and use this for calculation. We can manage a fixed size list of last few tps and clear this list when paused. If the average current rate is 0 tps, we can just say ETC as N/A.

We see the same problem at FFM. I normally do my own estimation based on how the performance is at present (or last few). We can make it easier for our users by doing this for them.

Let us know how this sounds.

[bbandlamudi]

Updated ETC logic to take last 10 (configurable through NUM-TPS-FOR-ETC option) tps estimates taken at 30sec intervals instead of total average tps from the start. The ETC will print "0:00-1 (paused)" if the job is paused and use the new tps values after the job is resumed.