YATSM schema

[ceholden]

Write document defining schema or spec for timeseries result storage used in YATSM. Preliminarily:

YATSM

Version: 1.0

1. Purpose

This specification describes the vocabulary and schema for describing timeseries within Yet Another Timeseries Model (YATSM).

2. Definitions

Term	Definition
segment	A period of time without disturbances. A segment represents a period of stable conditions, including stable land cover, such as permanent developed or forested cover, and stable land cover dynamics, such as a prolonged period of regrowing forest or the gradual succession of vegetation species.
break	An abrupt change in some characteristic of a segment, including changes in the magnitude, timing, or variability of observed data. breaks interrupt a segment and necessitate the estimation of another segment.
ephemeral	A break in a segment that does not persist. segments separated by ephemeral breaks are often functionally identical and may be joined together if ephemeral change processes are not of interest. Examples of ephemeral changes include precipitation driven early green-ups of vegetation in arid environments, non-fatal insect or weather driven defoliation events, or flooding that does not permanently change or alter the land cover or land cover characteristics.
ensemble	A history of segments and breaks for a given unit of area.
event	A change in land surface condition that does not constitute an abrupt change or segment break. Examples include a grassland fire, flood event, or hail storm over crops that does not change land cover and is not large enough to become an ephemeral change, but is something that would be worthwhile recording. Events may be classified based on the departures from the expected signal.
...	fill in more here

3. Schema

Timeseries models store their ensemble results as a collection of segments. Each segment has the following properties documented here in JSON format but stored within YATSM as numpy structured arrays:

dtype=[
    ('start', 'i4'),  # ordinal date of first observation in segment
    ('end', 'i4'),  # ordinal date of last observation in segment
    ('break', 'i4'),  # ordinal date of break that ends segment
    ('coef', 'float32', (n_features, n_models)),  # coefficients for all features in X design matrix for all fitted Y data
    ('px', 'u2'),  # column index coordinate of segment
    ('py', 'u2')  # row index coordinate of segment
]

Additional attributes, including the magnitude of a break or phenological attributes, may be stored as attributes of each segment.