-
Notifications
You must be signed in to change notification settings - Fork 614
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docs] Document logging inside Python tasks running on a remote cluster #4830
Comments
🤖 Hey @neverett! Great to see you back with another insightful question. Hope you're doing well! To set up logging inside normal Python tasks running on a remote cluster in Flyte, you can use the built-in logging module in Python. The logs will be sent to the logging system configured in Flyte, and they will be accessible from the Flyte web interface. Here's an example of how to log from your tasks: import logging
def my_task():
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message") In your specific case, you mentioned setting the You can find more information about this in the flytestdlib/logger/config.go file. For GCP (Google Cloud Platform) users, Flyte can be configured to use GCP Stackdriver for logging. You need to update the plugins:
logs:
kubernetes-enabled: false
stackdriver-enabled: true
gcp-project: your-gcp-project-id # Replace with your actual GCP project ID
stackdriver-logresourcename: k8s_container Remember to replace I hope this helps! If you have any other questions, feel free to ask. Sources
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
Description
Question from Slack:
Response:
In order to do this, you need to use the flytekit logger and set the level to 20. To do this in ImageSpec (or add to Dockerfile):
To log, import logger from flytekit and call in the task, making sure to set up the env variable:
https://github.com/flyteorg/flytekit/blob/f721eef474bb04ec2df581a690fb703e7c2b7956/flytekit/deck/deck.py#L6
https://github.com/flyteorg/flytekit/blob/f721eef474bb04ec2df581a690fb703e7c2b7956/flytekit/deck/deck.py#L149
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?
The text was updated successfully, but these errors were encountered: