Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(v2): refactor metastore #3652

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
bee43e4
hide raft command
kolesnikovae Oct 28, 2024
4ebebdb
move compactor api definition
kolesnikovae Oct 28, 2024
80f2406
add missing files
kolesnikovae Oct 28, 2024
2dab2c5
decompose metastore api
kolesnikovae Oct 28, 2024
5486155
refactor metastore grpc api
kolesnikovae Oct 28, 2024
9bed1a2
refactor metastore state wip
kolesnikovae Oct 28, 2024
4cfe66a
refactor metastore fsm
kolesnikovae Oct 30, 2024
3da7c93
refactor raft handlers
kolesnikovae Oct 30, 2024
248963f
refactor compaction planner
kolesnikovae Oct 31, 2024
8132057
document some ideas
kolesnikovae Oct 31, 2024
befc6f0
refactor compaction planner further
kolesnikovae Oct 31, 2024
3141057
refactor compaction block queue
kolesnikovae Oct 31, 2024
ae5e255
refactor compaction planner
kolesnikovae Oct 31, 2024
e171ca7
introduce tombstone cleaner
kolesnikovae Nov 1, 2024
7cc105d
improve job queue
kolesnikovae Nov 2, 2024
fc41ac2
redesign compaction planning
kolesnikovae Nov 3, 2024
2ff0c96
remove namespace from metrics
kolesnikovae Nov 3, 2024
942b905
compaction queue draft
kolesnikovae Nov 3, 2024
27fdbe6
dedup compaction job types
kolesnikovae Nov 4, 2024
197456d
compaction planner draft
kolesnikovae Nov 4, 2024
b44fa84
job planner draft
kolesnikovae Nov 4, 2024
0ae46ca
job planner draft
kolesnikovae Nov 4, 2024
76d29d4
compaction package
kolesnikovae Nov 4, 2024
4525e1e
compaction scheduling
kolesnikovae Nov 5, 2024
6230b1b
compaction abstraction
kolesnikovae Nov 5, 2024
933b881
schedule planning
kolesnikovae Nov 5, 2024
a14a9ef
compaction doc
kolesnikovae Nov 6, 2024
fca4077
compaction doc
kolesnikovae Nov 6, 2024
dce4571
compaction doc
kolesnikovae Nov 6, 2024
60025f0
schedule state update handling
kolesnikovae Nov 6, 2024
10df184
schedule implementation
kolesnikovae Nov 6, 2024
f43e48e
improve compaction worker failure handling
kolesnikovae Nov 7, 2024
144c997
index service
kolesnikovae Nov 7, 2024
5bd565f
explicit tombstone check
kolesnikovae Nov 7, 2024
b8689c3
cleanup
kolesnikovae Nov 8, 2024
71cca9e
cleanup, fixes, docs
kolesnikovae Nov 8, 2024
f398235
api improvements
kolesnikovae Nov 9, 2024
bf8bdc3
add scheduler queue tests
kolesnikovae Nov 9, 2024
aa8cf16
merge schedule update and assigner
kolesnikovae Nov 9, 2024
2e3df61
better job naming
kolesnikovae Nov 9, 2024
6fd1b25
add plan tests
kolesnikovae Nov 10, 2024
6dcd6f5
time-based block batch flush
kolesnikovae Nov 10, 2024
ee98dd8
add plan change test
kolesnikovae Nov 10, 2024
ad76970
add schedule test
kolesnikovae Nov 10, 2024
b087848
add sophisticated schedule tests
kolesnikovae Nov 11, 2024
a6b5da9
better tombstone handling
kolesnikovae Nov 11, 2024
97575d5
better tombstone handling
kolesnikovae Nov 11, 2024
32d6c8d
add note about CompactionJobState
kolesnikovae Nov 11, 2024
4667b9d
add more compaction tests
kolesnikovae Nov 11, 2024
10ce913
add more scheduler tests
kolesnikovae Nov 11, 2024
2abece3
update docs
kolesnikovae Nov 11, 2024
6ae9dee
store scratches
kolesnikovae Nov 11, 2024
185b8a0
add block queue store
kolesnikovae Nov 11, 2024
8ba2005
add block queue store
kolesnikovae Nov 11, 2024
d64bb39
improve tombstone api
kolesnikovae Nov 12, 2024
03e0c6d
tombstones orchestration
kolesnikovae Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions .mockery.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,19 @@ packages:
interfaces:
MetastoreServiceClient:
MetastoreServiceServer:
CompactionPlannerServer:
MetadataQueryServiceClient:
MetadataQueryServiceServer:
TenantServiceClient:
TenantServiceServer:
RaftNodeServiceClient:
RaftNodeServiceServer:
github.com/grafana/pyroscope/api/gen/proto/go/querier/v1/querierv1connect:
interfaces:
QuerierServiceClient:
github.com/grafana/pyroscope/pkg/frontend:
interfaces:
Limits:
github.com/grafana/pyroscope/api/gen/proto/go/compactor/v1:
interfaces:
CompactionPlannerServer:
github.com/grafana/pyroscope/pkg/experiment/metastore/discovery:
interfaces:
Discovery:
Expand All @@ -37,3 +41,12 @@ packages:
interfaces:
SegmentWriterClient:
IngesterClient:
github.com/grafana/pyroscope/pkg/experiment/distributor/placement:
interfaces:
Placement:

github.com/grafana/pyroscope/pkg/experiment/metastore/compaction/compactor:
interfaces:
BlockQueueStore:
Tombstones:
JobStore:
96 changes: 0 additions & 96 deletions api/compactor/v1/compactor.proto

This file was deleted.

Loading