Performance Testing Setup

[Jannis]

As we are working on performance improvements and new architectural decisions, we need a robust testing approach.

As a graph node developer, I want to...

• deploy multiple configurations / versions of graph-node that I want to compare
• to index the same subgraphs
• capturing subgraph-level indexing performance metrics

Relevant metrics:

• sync time across the same block range
• processing time breakdown by stage
• impact on graph-node (memory consumption etc.)

Questions:

• How are we going to measure pipelined execution? Will our existing stopwatch metrics still make sense?

[dotansimha]

@Jannis what's the scope of this issue? we were thinking about adding some performance testing for the GraphQL API at some point (we are usually using K6 for that)

[evaporei]

@dotansimha this issue is only on the indexing side for now. I'm curious what you folks where thinking on the query side 👀

[dotansimha]

@dotansimha this issue is only on the indexing side for now. I'm curious what you folks where thinking on the query side 👀

On the query side, we can create integration tests with some Subgraphs, and then execute K6 testing to get insights on how long it takes to query and execute calls to the DB. We can use run it as part of the CI, and probably detect regressions?

(created: #3273)

[azf20]

Initial setup is implemented (on the indexing side), closing