Skip to content

Optimize hparams to work with data providers natively #3425

@wchargin

Description

@wchargin

As of #3419, the hparams plugin uses the data provider APIs to read
data, but some of its internals still operate on the multiplexer data
structures (TensorEvent, TensorProto, etc.). These should be
migrated to just use the data provider APIs throughout, so that we don’t
have to convert back and forth. Callers that call read_scalars
repeatedly should also be migrated to call it just once with a
RunTagFilter representing the desired cross product, because this has
two orders of magnitude less processing overhead and (for networked data
providers) is one RPC instead of O(|metrics| × |sessions|) many.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions