Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make log format configurable #4851

Merged
merged 22 commits into from
Nov 27, 2024
Merged

feat: make log format configurable #4851

merged 22 commits into from
Nov 27, 2024

Conversation

Kraego
Copy link
Contributor

@Kraego Kraego commented Nov 26, 2024

Since I had problems with the encoding of the colored json log in the OKD shell (see Image), I've made the logformat configurable.

image

@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. enhancement New feature or request labels Nov 26, 2024
Copy link

codspeed-hq bot commented Nov 26, 2024

CodSpeed Performance Report

Merging #4851 will degrade performances by 27.98%

Comparing Kraego:main (d14f0e9) with main (2de119b)

Summary

⚡ 2 improvements
❌ 1 regressions
✅ 12 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main Kraego:main Change
test_successful_run_with_input_type_any 233.2 ms 143.1 ms +62.89%
test_successful_run_with_input_type_text 232 ms 322.1 ms -27.98%
test_successful_run_with_output_type_debug 231.1 ms 153.8 ms +50.21%

@Kraego
Copy link
Contributor Author

Kraego commented Nov 26, 2024

Would it be better to use LANGFLOW_LOG_ENV? Just found the variable in the code, currently there is no decription in the environments_variables.md for this setting.

imho: Setting the format directly will give the user more flexibility and is less implicit than the log environments but comes with the downside of configuration errors in the logging which will make diagnostics pretty hard.

If the maintainers decide that LANGFLOW_LOG_ENV should be used for my usecase, I will change this PR to just containing the needed additions in the documentation for this env var.

@ogabrielluiz ogabrielluiz changed the title Made logformat configurable feat: make log format configurable Nov 26, 2024
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Nov 26, 2024
@ogabrielluiz
Copy link
Contributor

Hey @Kraego

How are you?

This is a valid problem. If we were to use LANGFLOW_ENV we'd have to build something like the following to check for valid format strings:

import logging

def is_valid_log_format(format_string):
    """Validates a logging format string by attempting to format it with a dummy LogRecord.

    Args:
        format_string (str): The format string to validate.

    Returns:
        bool: True if the format string is valid, False otherwise.
        str: Error message if invalid, empty string if valid.
    """
    record = logging.LogRecord(
        name="dummy",
        level=logging.INFO,
        pathname="dummy_path",
        lineno=0,
        msg="dummy message",
        args=None,
        exc_info=None
    )

    formatter = logging.Formatter(format_string)

    try:
        # Attempt to format the record
        formatter.format(record)
    except (KeyError, ValueError, TypeError) as e:
        # Formatting failed, return False and the error message
        msg = f"Invalid log format string: {format_string}"
        raise ValueError(msg) from e
    return True

What do you think?

@Kraego
Copy link
Contributor Author

Kraego commented Nov 26, 2024

Hi @ogabrielluiz,

thanks I am doing well hope you too ...

As far as I see:

  • Option1: LANGFLOW_ENV has hard defined log formats in the code, so if you think this is sufficient for the user all we have to do is provide a documentation of this settings
  • Option2: If you think providing the full flexibility of complete custom log formats, Yes I think such a check method would be necessary and also a seperate env variable for configuration + LANGFLOW_ENV could/should have a documentation.

Both Options above will resolve my problem.

What option should I implement?

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Nov 26, 2024
@Kraego
Copy link
Contributor Author

Kraego commented Nov 26, 2024

I've added your proposed format string check

Copy link
Contributor

@ogabrielluiz ogabrielluiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Kraego !

LGTM

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Nov 26, 2024
@Kraego Kraego requested a review from ogabrielluiz November 26, 2024 17:58
@Kraego
Copy link
Contributor Author

Kraego commented Nov 26, 2024

@ogabrielluiz now ready for merge

@ogabrielluiz
Copy link
Contributor

Hey @Kraego

Sounds good!

I'll merge it ASAP. You can leave it out of date because it will only merge when no other PR is merging.

@dosubot dosubot bot removed the lgtm This PR has been approved by a maintainer label Nov 26, 2024
@Kraego Kraego requested a review from ogabrielluiz November 27, 2024 07:24
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Nov 27, 2024
@ogabrielluiz ogabrielluiz merged commit 1e6ada3 into langflow-ai:main Nov 27, 2024
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants