Spark TTFB Stats

[pyropy]

• Adds chart for displaying time to first byte stats per provider

• Adds chart for displaying time to first byte stats for overall network

Related to CheckerNetwork/roadmap#208

[cloudflare-workers-and-pages]

Deploying spark-dashboard with    Cloudflare Pages

Latest commit:	1f33cee
Status:	✅  Deploy successful!
Preview URL:	https://6da0db63.spark-dashboard.pages.dev
Branch Preview URL:	https://add-spark-ttfb-stats.spark-dashboard.pages.dev

View logs

[juliangruber]

Blocked by CheckerNetwork/spark-stats#281

[juliangruber]

Converted to draft as this is not ready to be merged yet

[pyropy]

Marking ready for review as CheckerNetwork/spark-stats#281 is merged.

I am not sure if we would need to backfill historical retrieval timing data first because without the data charts look broken.

[NikolasHaimerl]

Thanks for the PR @pyropy , shouldn't the graphs show successful HTTP requests as well?
They are running on the life application here.

[pyropy]

Thanks for the PR @pyropy , shouldn't the graphs show successful HTTP requests as well? They are running on the life application here.

Charts shown here have been created before we have merged new charts which show HTTP and Graphsync retrieval success score. The charts I have added (related to time to first byte stats) do not differentiate between the graphsync and HTTP retrievals.

I have merged main into this feature branch and new charts showing HTTP retrieval stats should be there now.

[bajtos]

I am not sure if we would need to backfill historical retrieval timing data first because without the data charts look broken.

We can wait a day or two until the data is populated.

However, if it's less than a half day of work, and you have the space for it, then feel free to backfill the data.

I think you would have to re-run spark-evaluate for historical rounds. That feels like too much hassle to me. WDYT?

More comments

• Please remove "p50" and "median" from the labels. The data we are showing is not p50/median; it is just an imprecise approximation.
• I see how it can be useful to dashboard consumers to understand what "ttfb" means. Let's provide a more descriptive text explaining that we are collecting p50 values for each retrieval task, and then calculating p50 from those p50's to get the single number presented in charts.
• Can we add a ttfb column to "Spark Miner RSR Table" table at the bottom of the network-wide dashboard? The column should be sortable. If this is not trivial, then please create a follow-up issue to the https://github.com/space-meridian/roadmap.

[bajtos]

Almost there!

[NikolasHaimerl]

LGTM. One thing we could think of is when viewing the ttfb for a particular miner we gain little information on what that data means other than how it is computed (Median of all ttfb values). I think the relationship between a single miners ttfb to all miners ttfb is what really matters here.
We could for example also plot the overall ttfb curve on the same chart that show each individual miner's ttfb. Also, we could create a boxplot chart of all accumulated ttfbs and show were in the distribution a particular miner currently resides in.
@bajtos WDYT?

[bajtos]

One thing we could think of is when viewing the ttfb for a particular miner we gain little information on what that data means other than how it is computed (Median of all ttfb values). I think the relationship between a single miners ttfb to all miners ttfb is what really matters here. We could for example also plot the overall ttfb curve on the same chart that show each individual miner's ttfb. Also, we could create a boxplot chart of all accumulated ttfbs and show were in the distribution a particular miner currently resides in. @bajtos WDYT?

I love both ideas!

However, I prefer to land this PR ASAP to conclude the PoC and defer additional improvements until we get feedback from people outside our team.

How about adding your suggestions as stretch goals to M5.1?

[pyropy]

LGTM. One thing we could think of is when viewing the ttfb for a particular miner we gain little information on what that data means other than how it is computed (Median of all ttfb values). I think the relationship between a single miners ttfb to all miners ttfb is what really matters here. We could for example also plot the overall ttfb curve on the same chart that show each individual miner's ttfb. Also, we could create a boxplot chart of all accumulated ttfbs and show were in the distribution a particular miner currently resides in. @bajtos WDYT?

I think those are great ideas! I'm going to create a issue out of your comment and we can address that in a separate PR.

[bajtos]

👏🏻