Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docker-base]: Rate limit priority INFO and lower in syslog #5666

Merged
merged 2 commits into from
Oct 20, 2020

Conversation

theasianpianist
Copy link
Contributor

@theasianpianist theasianpianist commented Oct 19, 2020

Signed-off-by: Lawrence Lee lawlee@microsoft.com

Temporary fix for #5570

- Why I did it
There is currently a bug where messages from swss with priority lower than the current log level are still being counted against the syslog rate limiting threshhold. This leads to rate-limiting in syslog when the rate-limiting conditions have not been met, which causes several sonic-mgmt tests to fail since they are dependent on LogAnalyzer. It also omits potentially useful information from the syslog. Only rate-limiting messages of level INFO and lower allows these tests to pass successfully.

- How I did it
Add $SystemLogRateLimitSeverity option to rsyslog.conf in the base Dockerfiles

- How to verify it
Build a new orchagent image and install it. Run the ACL tests, which should now pass. No rate limiting should occur in the syslogs.

- Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Copy link
Contributor

@jleveque jleveque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also please update the related comments to read:

# Set a rate limit on messages from the container of INFO level (level 6) or lower

@jleveque
Copy link
Contributor

You may also want to make the same change here: files/image_config/rsyslog/rsyslog-container.conf.j2

* Update comments to reflect new behavior
* Also change files/image_config/rsyslog/rsyslog-container.conf.j2

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
@theasianpianist theasianpianist merged commit 207587d into sonic-net:master Oct 20, 2020
@theasianpianist theasianpianist deleted the log-rate-limit branch October 20, 2020 18:52
santhosh-kt pushed a commit to santhosh-kt/sonic-buildimage that referenced this pull request Feb 25, 2021
…t#5666)

There is currently a bug where messages from swss with priority lower than the current log level are still being counted against the syslog rate limiting threshhold. This leads to rate-limiting in syslog when the rate-limiting conditions have not been met, which causes several sonic-mgmt tests to fail since they are dependent on LogAnalyzer. It also omits potentially useful information from the syslog. Only rate-limiting messages of level INFO and lower allows these tests to pass successfully.

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

orchagent continuously prints out "set: setting attribute 0x10000004 status: SAI_STATUS_SUCCESS"
3 participants