[COST-3571] generate daily reports #184
Conversation
Codecov Report
@@ Coverage Diff @@
## main #184 +/- ##
==========================================
+ Coverage 88.01% 88.72% +0.71%
==========================================
Files 11 11
Lines 2328 2386 +58
==========================================
+ Hits 2049 2117 +68
+ Misses 200 189 -11
- Partials 79 80 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
|
maskarb
changed the title
| t = t.Add(1 * time.Hour) | ||
| } | ||
|
|
||
| if r.initialDataCollection && t.Sub(startTime).Hours() == 96 { |
There was a problem hiding this comment.
This might still split into multiple files for the 4 day period? Also anything specific about the 96 mark or just feels right?
There was a problem hiding this comment.
The idea here was to fit the number of reports into the default max of 30. Assuming a cluster has 90 days of data, this would generate 90*24/96 ~= 23 reports.
If the reports are too big and require splitting, then yea, this logic goes out the window. But it is a best effort to fit a full 90 days into 30 reports.
There was a problem hiding this comment.
Thinking about this some more, when we split a payload, that is still counted as a single report. Here is the logic for that:
https://github.com/project-koku/koku-metrics-operator/blob/main/packaging/packaging.go#L428-L435
We get the timestamp from the tar.gz and insert into a set, then count the length of the set.
All payloads from a split report use this basename:
https://github.com/project-koku/koku-metrics-operator/blob/main/packaging/packaging.go#L517
which is just TIMESTAMP-cost-mgmt...
* only gather data for full days on initial CR creation * files are packaged after 96 hours of data are collected * when upgrading the operator, move and package the old files and regenerate data for the latest day * add a method for copying files instead of defaulting to moving files * report files generated before midnight are copied and packaged. the files remain in the data dir so we can append the rest of the days data to the file * after midnight, the files are moved out of the data dir and packaged
COST-3571