Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Context Prop #325

Closed
wants to merge 73 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
5f84c2b
stopgap PR to start the discussion on context propagation.
toumorokoshi Nov 7, 2019
6bea6ec
Merge remote-tracking branch 'origin/master' into context-prop
Dec 10, 2019
bbb583d
adding Baggage API
Dec 10, 2019
c3f408b
no-op implementations of baggage injector/extractor
Dec 10, 2019
8ec6106
cleaning up baggage prop interface
Dec 11, 2019
9788f69
make extract/inject static
Dec 11, 2019
6a46a25
rename DistributedContext to CorrelationContext
Dec 11, 2019
5ef40df
adding ContextKeys for trace and distributedcontext
Dec 11, 2019
0442f29
moving api/propagators -> api/propagation. adding more to context api
Dec 11, 2019
622d787
add from_context/with_span_context helpers
Dec 12, 2019
317e937
splitting propagator into extractor/injector
Dec 12, 2019
64cfe9b
checkpoint checkin, this PR will not leave draft mode
Dec 13, 2019
22a5c64
implement context scoping
Dec 13, 2019
375b78e
minor cleanup
Dec 13, 2019
5ca5328
clean up example code
Dec 16, 2019
b475933
pass context where needed
Dec 16, 2019
c4829c4
removing baggage module
Dec 16, 2019
c7610fa
small lint improvements
Dec 16, 2019
7489eb5
more lint fixes
Dec 16, 2019
a4c4150
fixing a few more lint issues
Dec 17, 2019
0946846
fixing inject/extract signatures and tracecontext tests
Dec 17, 2019
033e27e
fixing tests
Dec 17, 2019
cc813bb
rename current -> snapshot
Dec 17, 2019
7391372
fix remaining sdk tests
Dec 17, 2019
fa6d437
small context cleanup, return obj, remove unused __getattr__
Dec 17, 2019
5fa8e02
store both current span and extracted span context
Dec 17, 2019
164ef68
test cleanup
Dec 17, 2019
b37c3b0
fix ot shim tests
Dec 17, 2019
4ddac90
fixing http_requests tests
Dec 18, 2019
f500132
refactoring common getter/setter for propagation
Dec 18, 2019
93e88de
add convenience methods to set/get span from context
Dec 18, 2019
72c0dbd
fix wsgi and flask tests
Dec 18, 2019
f3c8076
add parameter to extract to support custom getters
Dec 18, 2019
d82e4c5
changing to copy for now, still need better mechanism here
Dec 18, 2019
8927455
enable wsgi/request in example
Dec 18, 2019
a4b7d0c
fix tracecontext tests
Dec 18, 2019
3b8bf5d
move distributedcontext to correlationcontext
Dec 18, 2019
9efb6e4
lint fixes
Dec 18, 2019
5272bed
add default injector/extractor. move tracecontext propagator to sdk
Dec 18, 2019
ea0905c
use default propagator to avoid installing sdk
Dec 18, 2019
4c2c4de
moving context implementation to sdk
Dec 18, 2019
1447d7f
tests fixed after context move
Dec 19, 2019
1cd03c5
lint fixes
Dec 19, 2019
7c9597c
Revert "lint fixes"
Dec 19, 2019
4ca46d9
Revert "tests fixed after context move"
Dec 19, 2019
48c2a7d
Revert "moving context implementation to sdk"
Dec 19, 2019
c7130a1
lint fix
Dec 19, 2019
5169723
fix tracecontext tests
Dec 19, 2019
673224c
mypy cleanup
Dec 19, 2019
4f008a4
Merge 'origin/master' into context-prop
Dec 19, 2019
66d67b8
fix context prop example
Dec 20, 2019
4442a04
mypy cleanup. adding some ignores for now, will review later
Dec 20, 2019
17bb4b1
adding context merge method
Dec 20, 2019
b82dc78
adding documentation
Jan 8, 2020
b850f99
Adding tests for concurrency, fixing context
Jan 9, 2020
33a5b78
Clean up tests and context
Jan 13, 2020
be91061
rename from_context to span_context_from_context
Jan 13, 2020
f84c4d3
Fix example
Jan 13, 2020
b1ba228
Context cleanup
Jan 14, 2020
c3906ea
Revert behaviour of no context to INVALID_SPAN_CONTEXT
Jan 15, 2020
8d0b142
More refactors of Context
Jan 15, 2020
6965c33
Removing context correlation propagators
Jan 20, 2020
c129e82
Lint changes
Jan 20, 2020
0a6c385
Adding test and fixing restore behaviour
Jan 20, 2020
57ad2d2
adding restore test
Jan 20, 2020
cfdfc62
Pass context to set_value
Jan 20, 2020
257627c
Rename HTTPInjector HTTPExtractor to Injector Extractor
Jan 20, 2020
b210df8
Moving propagation api into propagation/__init__.py
Jan 20, 2020
73dbfab
Fix example
Jan 20, 2020
b9be7bd
Context refactor
Jan 21, 2020
d70a47c
Adding set_in_carrier parameter, updating docs
Jan 21, 2020
1e3ee56
rename dctx_api
Jan 21, 2020
47f521f
Gutting Context class
Jan 23, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 123 additions & 16 deletions opentelemetry-api/src/opentelemetry/context/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,34 +138,141 @@ async def main():
asyncio.run(main())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation above has to be updated as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the restore issue. Will update the docs once the API changes are closer to done

"""

import threading
import typing
from contextlib import contextmanager

from .base_context import Context
from .base_context import Context, Slot

try:
from .async_context import (
AsyncRuntimeContext,
ContextVarSlot,
)

_context_class = AsyncRuntimeContext # pylint: disable=invalid-name
_slot_class = ContextVarSlot # pylint: disable=invalid-name
except ImportError:
from .thread_local_context import (
ThreadLocalRuntimeContext,
ThreadLocalSlot,
)

_context_class = ThreadLocalRuntimeContext # pylint: disable=invalid-name
_slot_class = ThreadLocalSlot # pylint: disable=invalid-name

_slots = {} # type: typing.Dict[str, 'Slot']
_lock = threading.Lock()


def _register_slot(name: str, default: "object" = None) -> Slot:
"""Register a context slot with an optional default value.

:type name: str
:param name: The name of the context slot.

:type default: object
:param name: The default value of the slot, can be a value or lambda.

:returns: The registered slot.
"""
with _lock:
if name not in _slots:
_slots[name] = _slot_class(name, default) # type: Slot
return _slots[name]


def set_value(
name: str, val: "object", context: typing.Optional[Context] = None,
) -> Context:
"""
To record the local state of a cross-cutting concern, the
Context API provides a function which takes a context, a
key, and a value as input, and returns an updated context
which contains the new value.

Args:
name: name of the entry to set
value: value of the entry to set
context: a context to copy, if None, the current context is used
"""
# Function inside the module that performs the action on the current context
# or in the passsed one based on the context object
if context:
ret = Context()
ret.snapshot = dict((n, v) for n, v in context.snapshot.items())
ret.snapshot[name] = val
return ret

# update value on current context:
slot = _register_slot(name)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid taking the lock when updating slots that are registered already?
Maybe a conditional here to call _register_slot() only on new ones?

slot.set(val)
return current()


def value(name: str, context: Context = None) -> typing.Optional["object"]:
"""
To access the local state of an concern, the Context API
provides a function which takes a context and a key as input,
and returns a value.

Args:
name: name of the entry to retrieve
context: a context from which to retrieve the value, if None, the current context is used
"""
if context:
return context.value(name)

# get context from current context
if name in _slots:
return _slots[name].get()
return None


def current() -> Context:
return _CONTEXT.current()
"""
To access the context associated with program execution,
the Context API provides a function which takes no arguments
and returns a Context.
"""
ret = Context()
for key, slot in _slots.items():
ret.snapshot[key] = slot.get()

return ret

def new_context() -> Context:
try:
from .async_context import ( # pylint: disable=import-outside-toplevel
AsyncRuntimeContext,
)

context = AsyncRuntimeContext() # type: Context
except ImportError:
from .thread_local_context import ( # pylint: disable=import-outside-toplevel
ThreadLocalRuntimeContext,
)
def set_current(context: Context) -> None:
"""
To associate a context with program execution, the Context
API provides a function which takes a Context.
"""
_slots.clear() # remove current data

for key, val in context.snapshot.items():
slot = _register_slot(key)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it could be optimized by taking the lock once and having a lock-less function that registers the slot. For instance

slot = _slot_class(key)
slot.set(val)
_slots[key] = slot

slot.set(val)

context = ThreadLocalRuntimeContext() # type: Context
return context

@contextmanager
def use(**kwargs: typing.Dict[str, object]) -> typing.Iterator[None]:
snapshot = current()
for key in kwargs:
set_value(key, kwargs[key])
yield
set_current(snapshot)


def new_context() -> Context:
return _context_class()


def merge_context_correlation(source: Context, dest: Context) -> Context:
return dest.merge(source)
ret = Context()

for key in dest.snapshot:
ret.snapshot[key] = dest.snapshot[key]

_CONTEXT = new_context()
for key in source.snapshot:
ret.snapshot[key] = source.snapshot[key]
return ret
73 changes: 49 additions & 24 deletions opentelemetry-api/src/opentelemetry/context/async_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,54 @@
except ImportError:
pass
else:
import typing # pylint: disable=unused-import
# import contextvars
import typing
from . import base_context

class AsyncRuntimeContext(base_context.BaseContext):
class Slot(base_context.BaseContext.Slot):
def __init__(self, name: str, default: object):
# pylint: disable=super-init-not-called
self.name = name
self.contextvar = ContextVar(name) # type: ContextVar[object]
self.default = base_context.wrap_callable(
default
) # type: typing.Callable[..., object]

def clear(self) -> None:
self.contextvar.set(self.default())

def get(self) -> object:
try:
return self.contextvar.get()
except LookupError:
value = self.default()
self.set(value)
return value

def set(self, value: object) -> None:
self.contextvar.set(value)
class ContextVarSlot(base_context.Slot):
def __init__(self, name: str, default: object):
# pylint: disable=super-init-not-called
self.name = name
self.contextvar = ContextVar(name) # type: ContextVar[object]
self.default = base_context.wrap_callable(
default
) # type: typing.Callable[..., object]

def clear(self) -> None:
self.contextvar.set(self.default())

def get(self) -> object:
try:
return self.contextvar.get()
except LookupError:
value = self.default()
self.set(value)
return value

def set(self, value: object) -> None:
self.contextvar.set(value)

class AsyncRuntimeContext(base_context.Context):
def with_current_context(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if this method has to be different for both implementations.

Wouldn't an implementation like the following work?

def with_current_context(
self, func: typing.Callable[..., "object"]
) -> typing.Callable[..., "object"]:
"""Capture the current context and apply it to the provided func.
"""
caller_context = self.snapshot()
def call_with_current_context(
*args: "object", **kwargs: "object"
) -> "object":
try:
backup_context = self.snapshot()
self.apply(caller_context)
return func(*args, **kwargs)
finally:
self.apply(backup_context)
return call_with_current_context

self, func: typing.Callable[..., "object"]
) -> typing.Callable[..., "object"]:
"""Capture the current context and apply it to the provided func.
"""

# TODO: implement this
# ctx = contextvars.copy_context()
# ctx.run()
# caller_context = self.current()

# def call_with_current_context(
# *args: "object", **kwargs: "object"
# ) -> "object":
# try:
# backup_context = self.current()
# self.set_current(caller_context)
# # return ctx.run(func(*args, **kwargs))
# return func(*args, **kwargs)
# finally:
# self.set_current(backup_context)

# return call_with_current_context
Loading