Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize CallsiteParameterAdder #606

Merged
merged 5 commits into from
Mar 26, 2024
Merged

Conversation

methane
Copy link
Contributor

@methane methane commented Mar 26, 2024

Summary

CallsiteParameterAdder is slower than logging.Logger.findCaller() because findCaller() doesn't use inspect at all.
This commit doesn't stop using inspect.getframeinfo(), but skips heavy code_context calculation.
This commit changes CallsiteParameterAdder to use frame object directly like findCaller().

Bench:

import structlog
import timeit
import os

null = open(os.devnull, "w", encoding="utf-8")

structlog.configure(
    processors=[
        structlog.processors.CallsiteParameterAdder([
            structlog.processors.CallsiteParameter.FILENAME,
            structlog.processors.CallsiteParameter.LINENO,
            ]),
        structlog.processors.JSONRenderer()
    ],
    logger_factory=structlog.PrintLoggerFactory(null),
    cache_logger_on_first_use=False
)
log = structlog.get_logger()

N = 100_000
x = timeit.timeit(lambda: log.info("hello"), number=N)
print(f"{x*1000_000/N}us")

Result:

before:
18.325596249997034us

getframeinfo(frame, context=0):
12.43612457998097us

use frame directly:
6.971802920015762us

Pull Request Check List

  • Do not open pull requests from your main branch – use a separate branch!
    • There's a ton of footguns waiting if you don't heed this warning. You can still go back to your project, create a branch from your main branch, push it, and open the pull request from the new branch.
    • This is not a pre-requisite for your your pull request to be accepted, but you have been warned.
  • Added tests for changed code.
    • The CI fails with less than 100% coverage.
  • New APIs are added to our typing tests in api.py.
  • Updated documentation for changed code.
    • New functions/classes have to be added to docs/api.rst by hand.
    • Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
      • The next version is the second number in the current release + 1. The first number represents the current year. So if the current version on PyPI is 23.1.0, the next version is gonna be 23.2.0. If the next version is the first in the new year, it'll be 24.1.0.
  • Documentation in .rst and .md files is written using semantic newlines.
  • Changes (and possible deprecations) are documented in the changelog.
  • Consider granting push permissions to the PR branch, so maintainers can fix minor issues themselves without pestering you.

methane and others added 5 commits March 26, 2024 20:05
CallsiteParameterAdder is slower than logging.Logger.findCaller()
because findCaller() doesn't use inspect at all.
This commit doesn't stop using inspect.getframeinfo(), but skips
heavy code_context calculation.
Copy link
Owner

@hynek hynek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!

@hynek hynek merged commit 6959f1d into hynek:main Mar 26, 2024
16 checks passed
@methane methane deleted the optimize-callsite branch March 26, 2024 15:33
@mdrachuk
Copy link

mdrachuk commented May 7, 2024

@hynek Can 24.2.0 be released with this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants