Use new partition methods #1427
Conversation
robert-milan
changed the title
From my grep the following projects rely on raintank/schema: fakemetrics, hosted-metrics-api, tsdb-gw, carbon-relay-ng
|
|
Thanks, I'll look at updating those in the future as well. |
- instead of comparing the partitionScheme string on every call, just keep a record of the schema.PartitionByMethod to use. This allows the Partition() method to just return m.PartitionID() directly. - in fakemetrics/kafkamdm use a Partitioner Interface instead of using partition.Kafka directly.
|
Did we forget about PartitionBySeriesWithTagsFnv? |
I left it out because according to the benchmarks in raintank/schema#26 it doesn't look like it performs well. Is there a reason we need both? Does one perform better in certain situations? I can add an option to use the method, or just remove it from schema. |
|
We deliberately included it for cases of existing datasets to not require migrations . The code should have comment somewhere explaining it. Both should be available through MT flags. Don't forget to update all config files using one of the dev scripts |
|
I guess I'm confused on that. Since all of these methods are new and have never been used how would someone already have existing datasets which use a method that hashes based on |
|
It allows to introduce tagging (with mediocre distribution ) without remapping non tagged series |
|
Thanks Dieter, that was the missing piece for me. I will implement in the next few PRs. |
This PR replaces and supersedes #1282.
Now that raintank/schema#26 is merged we can switch over to using
PartionIDinMetrictankandtsdb-gw.Remove the
Partitionerinterface in this PR or a later PR.This PR adds a
keyBySeriesWithTagsoption to partitioning.After this is merged I think the following steps should be followed:
tsdb-gwto use the new partitioning code.raintank/schemaintoMetrictank.tsdb-gwto use the vendored code inMetrictank.EatDotsin the validation methods.See also: #1123, raintank/schema#26, raintank/tsdb-gw#138