Non optimal default behavior

[nunokisc]

Host operating system:

Linux 3.10.0-962.3.2.lve1.5.25.10.el7.x86_64 #1 SMP Wed May 29 04:37:40 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
or
Linux 2.6.32-896.16.1.lve1.4.49.el6.x86_64 #1 SMP Fri Jan 5 05:15:23 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

mysqld_exporter version:

mysqld_exporter, version 0.11.0 (branch: HEAD, revision: 5d71796)
build user: root@3d3ff666b0e4
build date: 20180629-15:00:35
go version: go1.10.3

MySQL server version

Databases:

• Mysql between 5.6 and 5.7
• MariaDB >10

mysqld_exporter command line flags

Default

What did you do that produced an error?

I work at a company that provides internet services and we start installing monitoring with prometheus and everything went well until we decided to start installing the mysqld exporters and configure them on prometheus. The prometheus began to consume more than 64gb of memory in a few minutes and did not stop growing.
We came to find out that the problem was with the "collect.info_schema.tables" parameter that is enabled by default. We set up 120 machines with approximately 50000 tables in 700db's on each machine it became tricky for prometheus to process all this information.
This issue is only to alert to the fact that it may be dangerous to let this parameter enable by default, because with this parameter enable in the datacenter dimension we have, prometheus simply does not work and with this parameter disable prometheus works as if nothing were.

With this parameter enable we reach 9 million head times series and with the parameter disable we have around 250 thousand head time series. Now only consume about 2gb of memory.

[SuperQ]

Yes, that's a bit over the top. 😄 I agree, this collector should be disabled by default.