Handle case where there is a 'time' data key.

[danielballan]

In databroker, we make a best effort to fit the streaming "document model" into standard PyData types.

In v1, we use DataFrames (.table(...)) or generators (.data(...)). In v2, we use xarray Datasets. They have different trade-offs. DataFrames do not handle image (or in general N-dimensional) data well. Datasets do.

Both provide a concept of special columns that are used for alignment. DataFrames call it "index"; Datasets call it "coordinates". They are related conceptually and in terms of their implementation (xarray uses pandas.Index internally). But they have some semantic differences. DataFrames' indexes live in a separate namespace from columns, so there is no worry about name collisions. Datasets put their columns ("variables") and their indexes ("coordinates") in one namespace, and so the same name cannot be used for both a variable and a coordinate.

For databroker, this means that if the user uses "time" as a data key in an Event document, it will collide with the Event time like this:

{
    "time": ...,  # all Events have an overall time
    "data": {
        "time": ...,  # ophyd-defined name for a data key might also be "time"
    },
    ...
}

And in v2 we currently do not handle that gracefully. @tacaswell has proposed that we leave the user-defined name alone and add underscopes to the front of our coordinate time, trying first _time and then __time and then ___time (etc.) until we find a name that doesn't collide. Hopefully _time will usually be enough, but this retry-based approach is guaranteed to avoid a collision even if some user decides to use both time and _time for example.

[danielballan]

[Copied from Slack] This is the place to start:

databroker/databroker/mongo_normalized.py

Lines 87 to 113 in a2a5d66

	# Build the time coordinate.
	time_shape = (max_seq_num - 1,)
	time_chunks = normalize_chunks(
	("auto",) * len(time_shape),
	shape=time_shape,
	limit=CHUNK_SIZE_LIMIT,
	dtype=FLOAT_DTYPE.to_numpy_dtype(),
	)
	time_variable = ArrayStructure(
	macro=ArrayMacroStructure(
	shape=time_shape,
	chunks=time_chunks,
	dims=["time"],
	),
	micro=FLOAT_DTYPE,
	)
	time_data_array = DataArrayStructure(
	macro=DataArrayMacroStructure(
	variable=time_variable, coords={}, coord_names=[], name="time"
	),
	micro=None,
	)
	if unicode_columns is None:
	unicode_columns = {}
	dim_counter = itertools.count()
	data_vars = {}
	metadata = {"data_vars": {}, "coords": {"time": {"attrs": {}}}}

[gwbischof]

In databroker-intake the event.data.time overrides event.time