Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to get disk metrics for /sys/kernel/debug/tracing: [Errno 13] Permission denied: '/sys/kernel/debug/tracing' #5675

Closed
SaloSentinelOne opened this issue Feb 9, 2020 · 3 comments

Comments

@SaloSentinelOne
Copy link

SaloSentinelOne commented Feb 9, 2020

for part in psutil.disk_partitions(all=True):

>>> import psutil
>>> 
>>> psutil.disk_partitions(all=True)
[
sdiskpart(device='sysfs', mountpoint='/sys', fstype='sysfs', opts='rw,nosuid,nodev,noexec,relatime'), 
sdiskpart(device='proc', mountpoint='/proc', fstype='proc', opts='rw,nosuid,nodev,noexec,relatime'),
sdiskpart(device='udev', mountpoint='/dev', fstype='devtmpfs', opts='rw,nosuid,relatime,size=65254592k,nr_inodes=16313648,mode=755'),
sdiskpart(device='devpts', mountpoint='/dev/pts', fstype='devpts', opts='rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000'),
sdiskpart(device='tmpfs', mountpoint='/run', fstype='tmpfs', opts='rw,nosuid,noexec,relatime,size=13053332k,mode=755'),
sdiskpart(device='/dev/nvme0n1p2', mountpoint='/', fstype='ext4', opts='rw,relatime,data=ordered'),
sdiskpart(device='securityfs', mountpoint='/sys/kernel/security', fstype='securityfs', opts='rw,nosuid,nodev,noexec,relatime'),
sdiskpart(device='tmpfs', mountpoint='/dev/shm', fstype='tmpfs', opts='rw,nosuid,nodev'),
sdiskpart(device='tmpfs', mountpoint='/run/lock', fstype='tmpfs', opts='rw,nosuid,nodev,noexec,relatime,size=5120k'),
sdiskpart(device='tmpfs', mountpoint='/sys/fs/cgroup', fstype='tmpfs', opts='ro,nosuid,nodev,noexec,mode=755'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/systemd', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd'),
sdiskpart(device='pstore', mountpoint='/sys/fs/pstore', fstype='pstore', opts='rw,nosuid,nodev,noexec,relatime'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/cpu,cpuacct', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,cpu,cpuacct'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/pids', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,pids'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/devices', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,devices'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/blkio', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,blkio'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/freezer', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,freezer'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/perf_event', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,perf_event'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/memory', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,memory'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/cpuset', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,cpuset'),
sdiskpart(device='cgroup', mountpoint='/sys/fs/cgroup/net_cls,net_prio', fstype='cgroup', opts='rw,nosuid,nodev,noexec,relatime,net_cls,net_prio'),
sdiskpart(device='systemd-1', mountpoint='/proc/sys/fs/binfmt_misc', fstype='autofs', opts='rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13040'),
sdiskpart(device='mqueue', mountpoint='/dev/mqueue', fstype='mqueue', opts='rw,relatime'),
sdiskpart(device='debugfs', mountpoint='/sys/kernel/debug', fstype='debugfs', opts='rw,relatime'),
sdiskpart(device='hugetlbfs', mountpoint='/dev/hugepages', fstype='hugetlbfs', opts='rw,relatime'),
sdiskpart(device='sunrpc', mountpoint='/run/rpc_pipefs', fstype='rpc_pipefs', opts='rw,relatime'),
sdiskpart(device='/dev/md0', mountpoint='/mnt/dfs', fstype='xfs', opts='rw,noatime,attr2,inode64,noquota'),
sdiskpart(device='binfmt_misc', mountpoint='/proc/sys/fs/binfmt_misc', fstype='binfmt_misc', opts='rw,relatime'),
sdiskpart(device='tracefs', mountpoint='/sys/kernel/debug/tracing', fstype='tracefs', opts='rw,relatime')
]
# uname -a
Linux node 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 GNU/Linux
# pip3 show psutil
Name: psutil
Version: 5.6.7
Summary: Cross-platform lib for process and system monitoring in Python.
Home-page: https://github.com/giampaolo/psutil
Author: Giampaolo Rodola
Author-email: g.rodola@gmail.com
License: BSD
Location: /usr/local/lib/python3.5/dist-packages
Requires: 
@SaloSentinelOne
Copy link
Author

I suggest filtering some of the results psutil returns.
This bug affects our production and we can't have disk alerts or metrics because of it.

@therve
Copy link
Contributor

therve commented Feb 11, 2020

Can you add tracefs to the file_system_blacklist configuration to see if that unblocks you? We can add it by default if it does.

@SaloSentinelOne
Copy link
Author

@therve
It worked, thank you, +1 on the changing defaults suggestion.

DaveWK added a commit to DaveWK/integrations-core that referenced this issue Dec 15, 2022
Attempting to include tracefs volumes causes a spammy WARN-level log message by
default.

Issues where this was reported:
DataDog#5675
DataDog/dd-agent#3875
steveny91 pushed a commit that referenced this issue Jan 19, 2023
* Exclude tracefs by default from disk checks

Attempting to include tracefs volumes causes a spammy WARN-level log message by
default.

Issues where this was reported:
#5675
DataDog/dd-agent#3875

* fix test cases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants