Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[image_config] add rasdaemon.timer #14300

Merged
merged 1 commit into from
Apr 17, 2023
Merged

[image_config] add rasdaemon.timer #14300

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions files/build_templates/sonic_debian_extension.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -437,6 +437,11 @@ sudo cp $IMAGE_CONFIGS/corefile_uploader/core_uploader.py $FILESYSTEM_ROOT/usr/b
sudo cp $IMAGE_CONFIGS/corefile_uploader/core_analyzer.rc.json $FILESYSTEM_ROOT_ETC_SONIC/
sudo chmod og-rw $FILESYSTEM_ROOT_ETC_SONIC/core_analyzer.rc.json

# Rasdaemon service configuration. Use timer to start rasdaemon with a delay for better fast/warm boot performance
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a downside here as well? Can the device potentially lose the hardware/memory errors?

I think the asnwer is yes. In that case, I think we should do this only where necessarily needed. Is there a merit of doing this for anything except warmboot?

For other cases (cold/fast/load-mg/config-reload) : The memory/hardware errors are more likely to be hit during bootup time and indicate a possibly bad hw. We want such errors to be logged while the system is booting up.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vaibhavhd
load-mg/config-reload - rasdaemon is not touched.
warm/fast - we must also delay it for fast reboot. It is the same kexec operation. Same operation from CPU/mem perspective.
Please note, that we don't have a way to delay per different boot types, e.g pmon, snmp, lldp are delayed regardless of the boot type.

rasdaemon was added as a replacement for mcelog. MCE exceptions are recorded into a kernel ring buffer, so they aren't lost if it reads it later (or at least this is how mcelog worked). rasdaemon states it reads not just /dev/mce but several sources EDAC, MCE, PCI, ... I am not aware how can I test whether events are lost or not as I don't know a way to generate this exceptions. At least an MCE exceptions should not be missed in my understanding.

sudo cp $IMAGE_CONFIGS/rasdaemon/rasdaemon.timer $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM
sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT systemctl disable rasdaemon.service
sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT systemctl enable rasdaemon.timer

sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y install libffi-dev libssl-dev

{% if include_bootchart == 'y' %}
Expand Down
10 changes: 10 additions & 0 deletions files/image_config/rasdaemon/rasdaemon.timer
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[Unit]
Description=Delays rasdaemon until SONiC has started

[Timer]
OnUnitActiveSec=0 sec
OnBootSec=1min 30 sec
Unit=rasdaemon.service

[Install]
WantedBy=multi-user.target