New architecture prototype #3388
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add dummy metastore
…3400) * Distribute profiles to shards based on tenant/service_name/labels with no replication * Add retries in case of delivery issues (WIP) * Calculate shard for ingested profiles, send to ingesters in push request * Set replication factor and token count via config * Fix tests * Run make helm/check check/unstaged-changes * Run make reference-help * Simplify shard calculation * Add a metric for distributed bytes * Register metric * Revert undesired change
* start working on segments writer add shards awaiting segment flush in requests upload blocks add some tracing * upload meta in case of metastore error * do not upload metadata to dlq * add some flags * skip some tests. fmt * skip e2e tests * maybe fix microservices_test.go. I have no idea what im doing * change partition selection * rm e2e yaml * fmt
# Conflicts: # .github/workflows/helm-ci.yml
# Conflicts: # examples/golang-pgo/Dockerfile # examples/language-sdk-instrumentation/golang-push/rideshare/Dockerfile # examples/language-sdk-instrumentation/golang-push/rideshare/Dockerfile.load-generator # examples/language-sdk-instrumentation/golang-push/simple/Dockerfile # operations/pyroscope/helm/pyroscope/rendered/micro-services.yaml
kolesnikovae
force-pushed
the
experiment/new-architecture
branch
from
da7f14c to
04e7036
Compare
# Conflicts: # api/gen/proto/go/types/v1/types.pb.go # go.mod # go.sum
kolesnikovae
changed the title
korniltsev
reviewed
Aug 7, 2024
| @@ -279,14 +283,14 @@ type FileWriter struct { | |||
| name string | |||
| } | |||
|
|
|||
| func NewFileWriter(name string) (*FileWriter, error) { | |||
| func NewFileWriter(name string, bufferSize int) (*FileWriter, error) { | |||
There was a problem hiding this comment.
I will revert these changes in tsdb/index/index.go - it is not used anymore
There was a problem hiding this comment.
It's currently used in compactor. I see no need to change it now
korniltsev
reviewed
Aug 7, 2024
| @@ -0,0 +1,71 @@ | |||
| syntax = "proto3"; | |||
|
|
|||
| package metastore.v1; | |||
There was a problem hiding this comment.
nit: maybe we should put it in v2 or v2-experimental package
There was a problem hiding this comment.
This is the first version of metastore API.
As long as the API is not public, I see no practical benefits of separating the API
This reverts commit 2188cde.
simonswine
approved these changes
Aug 12, 2024
There was a problem hiding this comment.
Thank you so much for cleaning that up and make it mergable it main @kolesnikovae .
Also kudos to everyone else working on this experiment 👏
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR is the main development branch for the work on the new architecture (#3192).
The PR will never be merged; it was created only for visibility. Once the prototype is finished, we will publish appropriate documentation based on the experiment results.Update:
The experiment is over. The results are very promising.
The new code is very well isolated and does not affect behaviour of the current version, therefore I'd like to merge the changes: we will continue working on the new components in the main branch.