Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core feature] Enhance task logging template utils for start/finish time #3348

Open
2 tasks done
andrewwdye opened this issue Feb 16, 2023 · 1 comment
Open
2 tasks done
Labels
enhancement New feature or request flytepropeller needs-decision Requires decision

Comments

@andrewwdye
Copy link
Contributor

Motivation: Why do you think this is important?

Flyteplugins provides default task logging support for Cloudwatch and Stackdriver; however, the links don't include start/finish times. Depending when the links are followed, the user may need to adjust time filters in the log viewer, leading to initial confusion over missing logs and adding an unnecessary step.

Template params for {{ .podUnixStartTime }} and {{ .podUnixFinishTime }} are provided; however. they are incompatible with at least Cloudwatch and Stackdriver. Cloudwatch expects time queries in unix timestamp using millisecond granularity and Stackdriver expects time queries in RFC3339 or ISO 8601 formats.

Goal: What should the final outcome look like, ideally?

Flyte task log links should contain time specific references based on the task's start time (and end time if available).

Describe alternatives you've considered

Require the user to manually filter to the task timeframe

Propose: Link/Inline OR Additional context

One option is to introduce new template params for PodUnixMsecStartTime and PodTimestampStartTime (and their Finish counterparts), corresponding to unix milliseconds and RFC 3339, respectively.

As part of this change we should update the log link to use the container end time if available. Today it is hardcoded to now().

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@andrewwdye andrewwdye added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Feb 16, 2023
@EngHabu
Copy link
Contributor

EngHabu commented Feb 16, 2023

Gotemplates support many operators that can unblock these scenarios since, specially with timestamps, it's really hard to provide templates that satisfy all needs (one can argue we should at least support OOB some of these default providers)
This https://coveooss.github.io/gotemplate/docs/basic_features/arithmetic/ can be used to multiply by 1000

{{ mul .podUnixStartTime 1000 }}

This https://coveooss.github.io/gotemplate/docs/functions_reference/sprig-date/ can be used to format datetime I believe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request flytepropeller needs-decision Requires decision
Projects
None yet
Development

No branches or pull requests

3 participants