Notes from discussion regarding training input

[CBroz1]

We plan to ...

1. Extend ndx-pose - check backwards compatibility
2. Assume external filepaths for images/videos
3. Check in with Mathis team regarding this development

Pose Training input:
  Training parameters: json unspecified
  Skeletons:
    1+Skeleton:
      id: string
      1+Node Labels: string
      1+Edges: optional
  Training Frames:
    1+Training Frame:
      Annotator: optional string
      Source Video: optional
        Path: relative filepath
        Frame Index: unsigned int
      Source Frame: optional filepath
      Instances:
        1+Links to Skeleton: id
        1+Node Positions:
          x: unsigned int
          y: unsigned int
          z: optional unsigned int

[CBroz1]

I posted to the DLC gitter to ask for input

[roomrys]

Proposed refactoring of pose training structure:

1. Removed training parameters
2. Add attribute for node visibility
3. Add Identity group
4. Add link to instance identity

Pose Training input:
  Identities: optional
       1+Identity: string
            name: string
            description: string
  Skeletons:
       1+Skeleton:
             id: string
             1+Node Labels: string
             1+Edges: optional
  Training Frames:
    1+Training Frame:
        Annotator: optional string
        Source Video: optional
             Path: relative filepath
             Frame Index: unsigned int
        Source Frame: optional filepath
        Instances:
             1+Links to identity: name
             1+Links to Skeleton: id
             1+Node Positions:
                  x: float
                  y: float
                  z: optional float
                  visible: bool

[CodyCBakerPhD]

@roomrys mostly LGTM; the only thing I'd suggest is for Source Video and Source Frame, these should be ImageSeries (which allows the user flexibility to choose between external vs. internal storage) and RGBImage data types, respectively.

With our default approach of writing the image frames directly to h5py.Dataset, both of those should work as-is for now but leaves it open for long-term flexibility for other user-chosen methods of data storage.

[roomrys]

Forked repo here with these specifications. Currently adding tests in SLEAP for writing/reading the proposed format. Subject to change and open to recommendations/concerns!

[CodyCBakerPhD]

https://github.com/talmolab/ndx-pose-training/blob/main/spec/ndx-pose.extensions.yaml is looking good - should we open a draft PR to ndx-pose to continue the discussion about any specifics there?