Bucket Logging: BG worker to upload bucket logs #7816
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@dannyzaken , @romayalon, @liranmauda |
liranmauda
reviewed
Feb 29, 2024
There was a problem hiding this comment.
Few comment:
- When we use dbg, we don't need to specify the file name.
for example: instead of writingdbg.log0('Bucket_log_uploader: system_store did not finish initial load');we should write:dbg.log0('system_store did not finish initial load'); - please add javadoc for typescript on every function.
aspandey
force-pushed
the
bucket_logging_bg_worker
branch
7 times, most recently
from
dac9516 to
feea1ec
Compare
liranmauda
reviewed
Mar 13, 2024
Comment on lines
280
to
284
| required: ['name', 'log_bucket', 'log_prefix'], | ||
| properties: { | ||
| name: { | ||
| $ref: 'common_api#/definitions/bucket_name', | ||
| }, |
There was a problem hiding this comment.
Let's take it out into a different PR.
liranmauda
reviewed
Mar 13, 2024
Comment on lines
136
to
137
| let log_objects = null; | ||
| let log_file_path = null; |
There was a problem hiding this comment.
decleration of variable does not require to be initiated with null
aspandey
force-pushed
the
bucket_logging_bg_worker
branch
3 times, most recently
from
b903cff to
4295ad4
Compare
Bucket logs will be collected and placed on /log/noobaa_bucket_logs/ directory. The BG worker, BucketLogUploader, will keep scanning this directory and upload the object on a log bucket. This log bucket name will be fetched from the log object name. Signed-off-by: Ashish Pandey <aspandey@redhat.com>
aspandey
force-pushed
the
bucket_logging_bg_worker
branch
from
4295ad4 to
cb0857e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bucket logs will be collected and placed on /log/noobaa_bucket_logs/ directory. The BG worker, BucketLogUploader, will keep scanning this directory and upload the object on a log bucket.
This log bucket name will be fetched from the log object name.
Testing Instructions:
Once the core pod is UP and running , exec into it and create /log/noobaa_bucket_logs/ directory.
Create some files with bucket name which we have created using nb command.
if you have created bucket like this -
nb api bucket_api create_bucket '{
"name": "log.bucket"
}'
Then create files as follows -
echo "Bucket logging " >> /log/noobaa_bucket_logs/log.bucket-1.log
After some time - config.BUCKET_LOG_UPLOADER_DELAY = 5 * 60 * 100; -
This log will be uploaded to log.bucket.