Plots per step

[daavoo]

Loving the Images per step slider!

I think it would be awesome if we could extend the logic for non-linear Plots (i.e. confusion matrix).

Like:

import random

from dvclive import Live

with Live(report=None, save_dvc_exp=True) as live:
    y_true = [0, 0, 1, 1]
    for i in range(10):
        y_pred = [int(random.random() > 0.5) for _ in range(4)]
        live.log_sklearn_plot(
            "confusion_matrix", y_true, y_pred, 
            name=f"confusion_matrix/{i}.json")
        live.next_step()

[dberenbaum]

@daavoo Do you have any use cases where a confusion matrix is saved per step? Not to push back (the idea sounds good), just curious how important you see it since I see it's marked p1.

[daavoo]

@daavoo Do you have any use cases where a confusion matrix is saved per step? Not to push back (the idea sounds good), just curious how important you see it since I see it's marked p1.

Out of my context if it is p1 . I consider it an useful thing / as relevant as the image support (one of the GIFs I put in #1640 is a confusion matrix), but it is not a deal breaker

[dberenbaum]

If that's the behavior you want, maybe it's better to save the confusion matrix as an image?

[dberenbaum]

I would say let's discuss before doing research in vega since we don't want to keep investing in vega.

[daavoo]

If that's the behavior you want, maybe it's better to save the confusion matrix as an image?
I would say let's discuss before doing research in vega since we don't want to keep investing in vega.

Yes, makes sense, sorry for the lack of clarity I was answering about the usefulness of the confusion matrix per steps.

[dberenbaum]

Yup, agree with everything you wrote, and sorry if I'm being a downer. I just want to clarify before anyone spends too much time here on the wrong idea.

[julieg18]

It was made p1 for now in planning meeting. I'm starting out with researching how easy it is to implement.

[julieg18]

Did some research, it should be doable but implementing in the frontend will be a a bit more difficult since template plots have a lot more complexity happening than our image plots. As discussed in planning, going to deprioritize this for the time being.

[shcheklein]

I guess let's get to plotly first, then we can come back to things like this folks.