Check Beats overflow handling

[urso]

Indexing/transporting unsigned integers >2^63 has been a problem, due to consumers (libs/language/Elasticsearch) not naturally supporting integers of this size. With #8991 we automatically cap values to 2^63-1 that can overflow. This might not always be the right solution (see comment).

We should check the list of known issues if the masking is ok or not:

• Unexpectedly huge byte values returned for system.memory.swap.used.bytes Unexpectedly huge byte values returned for system.memory.swap.used.bytes #6271
• Metricbeat - non incremental network usage metrics Metricbeat - non incremental network usage metrics #2783
• Integer overflow in Beats metrics Integer overflow in Beats metrics #5433
• system.fsstat.total_files hits long MAX [Metricbeat] system.fsstat.total_files hits long MAX #5386
• memstat.rss reports negative values memstat.rss reports negative values #7505
• Max uint64 used as infinite limit in cgroups Missing cgroup fields on hardened Debian 8 machines leads to errors #5854

Note: Also to figure out if we want to backport #8891 or not.

@andrewkroh @ruflin @jsoriano @kvch Please add issues from github/discuss I did miss :)

[jsoriano]

@urso #6271 is an overflow caused by how we calculate some values, it should be considered a bug on the calculation and there is no need to handle it in a special way.

I'm not sure to understand why #2783 is in this list 🙂
If this is for the overflows caused by very big numbers I think that there is no general solution, if they are very big gauges we could collect them as doubles. If they are continuously increasing values or counters, then I think capping them is fine.

[urso]

I'm not sure to understand why #2783 is in this list 🙂

The list contains known issues. I didn't look into the actual issues. Just add every issue/discussion you are aware of to the list.

We have to see if #8991 is the right solution for some of these(e.g. close the issue), or if we have to fix the actual issue. If we find #8991 is a good solution for some, we might consider to backport it to 6.x. Otherwise we will have to fix the particular issue within the module! E.g. #2783 is very old. If we find #8991 is good here, close it. Otherwise fix it (it's open for quite some time).

[elasticmachine]

Pinging @elastic/infrastructure

[urso]

@jsoriano Any progress on identifying metricsets that need to handle values?

[fearful-symmetry]

#5854 is related. There's a lot of things that are explicitly designed to squash uint64-sized values. Trying to comb through metricbeat for examples of this is going to be...a long effort.

[jsoriano]

@jsoriano Any progress on identifying metricsets that need to handle values?

I don't remember to actively look for cases per metricset :(

@fearful-symmetry thanks for pointing to this issue, I have added it to the list.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.