-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
timestamp stage with multiple formats #2590
Comments
This is an interesting idea! There would be some overhead of this approach in CPU terms but I think we are balancing that against ease of use. As you describe this can be accomplished now but only in a very verbose and config heavy way. I think this is a good idea, however we are pretty swamped at the moment, if this is something you are able to help contribute a PR for that would be great, else I'm happy to keep the issue open and see if anyone else comes along or we have some time to take a look! |
Yes, I'd be happy to PR this feature. As far as the UX goes for this feature, which one of the following options would you prefer?
|
I'm in favor of 2. and name the array |
Fixes #2590 Co-authored-by: Amin Jamali <ajamali@vmware.com>
Is your feature request related to a problem? Please describe.
Yes, We consume data that can have multiple types of timestamp formats (RFC3339Nano, Unix, or "2006-01-02 03:04:05.000000000 +0000 UTC"). All timestamps arrive under the same json key but different formats. Currently we inspect the input beforehand, find out the format, and create a unique promtail config for a given selector.
Describe the solution you'd like
It would be ideal to define multiple formats under
timestamp
stage. timestamp stage would then process each format and if it errors, it would try the next one on the list. An example timestamp stage would look like the following:Describe alternatives you've considered
Pre-process the input and create unique promtail configs.
The text was updated successfully, but these errors were encountered: