Add option max-tags-per-database to limit high cardinality data

[daviesalex]

Much as max-series-per-database in #7095, add a max-tags-per-database to limit high cardinality data.

Writes beyond that should be dropped and logged with a rate limit on the log (one summary log per minute or something saying x writes dropped).

This is important because if you accidentally load in a huge amount of high cardinality data, you can easily get into a place that InfluxDB will OOM if you attempt to delete the data, so your only choice is to try to move the files on disk out the way which deletes other data.

[sebito91]

Would add to this that we should have alerts to the log when we start to
get close to the threshold, like 85%, 90%, etc.

Helps us stay proactive in cleanup and maintenance.

On Thursday, August 11, 2016, Alex Davies notifications@github.com wrote:

Much as max-series-per-database in #7095
#7095, add a
max-tags-per-database to limit high cardinality data.

Writes beyond that should be dropped and logged with a rate limit on the
log (one summary log per minute or something saying x writes dropped).

This is important because if you accidentally load in a huge amount of
high cardinality data, you can easily get into a place that InfluxDB will
OOM if you attempt to delete the data, so your only choice is to try to
move the files on disk out the way which deletes other data.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#7146, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADA1nuIdOrtPkcMwEeq8XvT9D6NiTP0gks5qe58egaJpZM4Jim-5
.

Sebastian Borza

PGP: EDC2 BF61 4B91 14F2 AAB4 06C9 3744 7F3F E411 0D3E

[daviesalex]

Of course if we do #7151 well (which looks great, thanks!), this becomes a non-issue; this is more of a sticking plaster until that can be done!

[jwilder]

A similar proposal is in #6679

[summerjava]

This is important because if you accidentally load in a huge amount of high cardinality data, you can easily get into a place that InfluxDB will OOM if you attempt to delete the data,

I am curious, why would InfluxDB OOM when I attempt to delete the data?
Is it the index created on tag?
Can you give a Detailed explanation?