Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add fieldFetched metric to @explain #1637

Closed
islamaliev opened this issue Jul 14, 2023 · 0 comments
Closed

Add fieldFetched metric to @explain #1637

islamaliev opened this issue Jul 14, 2023 · 0 comments
Assignees
@islamaliev
Labels
area/planner Related to the planner system feature New feature or request

Comments

@islamaliev
Copy link
Contributor

islamaliev commented Jul 14, 2023 •
edited
Loading

As each document in defradb consist of several fields with each of them being stored separately. Hence fetching 1 doc that has only 1 field and fetching 1 doc that has dozens of fields has a big difference.
That's why it would be a valuable information to add the number of fields fetched to the @Explain query.
@islamaliev islamaliev changed the title Add propFetched metric to @explain Add fieldFetched metric to @explain Jul 14, 2023
@islamaliev islamaliev added feature New feature or request area/planner Related to the planner system labels Jul 14, 2023
@islamaliev islamaliev self-assigned this Jul 26, 2023
@islamaliev islamaliev mentioned this issue Jul 26, 2023
Merged
4 tasks
islamaliev added a commit that referenced this issue Jul 27, 2023
@islamaliev
feat: Make fetcher calculate docFetches and fieldFetches (#1713)
afee323 
## Relevant issue(s)

Resolves #1636 and #1637

## Description

With this change the calculation of stats for fetching documents is
moved to fetcher.

This was necessary because each call to `fetcher.FetchNext...` could
potentially perform multiple doc fetches.
But scanNode (previously responsible for this) would just increment by 1.
Moreover, scanNode would incorrectly count end of fetching (returned nil
doc) as 1 as well.

Also this fix introduces another (probably more important) metric
`fieldFetches` which better indicates the execution cost because 
it much closer correlates with number of IO operations, as eace 
document's field is stored in a separate KV pair.
shahzadlone pushed a commit to shahzadlone/defradb that referenced this issue Feb 23, 2024
@islamaliev
feat: Make fetcher calculate docFetches and fieldFetches (sourcenetwo…
a517a2c 
…rk#1713)

## Relevant issue(s)

Resolves sourcenetwork#1636 and sourcenetwork#1637

## Description

With this change the calculation of stats for fetching documents is
moved to fetcher.

This was necessary because each call to `fetcher.FetchNext...` could
potentially perform multiple doc fetches.
But scanNode (previously responsible for this) would just increment by 1.
Moreover, scanNode would incorrectly count end of fetching (returned nil
doc) as 1 as well.

Also this fix introduces another (probably more important) metric
`fieldFetches` which better indicates the execution cost because 
it much closer correlates with number of IO operations, as eace 
document's field is stored in a separate KV pair.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@islamaliev islamaliev

Labels
area/planner Related to the planner system feature New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@islamaliev