Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop uploading debug meta files to obj store #3839

Closed
yeya24 opened this issue Feb 26, 2021 · 15 comments · Fixed by #5110
Closed

Stop uploading debug meta files to obj store #3839

yeya24 opened this issue Feb 26, 2021 · 15 comments · Fixed by #5110

Comments

@yeya24
Copy link
Contributor

yeya24 commented Feb 26, 2021

Is your proposal related to a problem?

(Write your answer here.)

Screenshot from 2021-02-25 22-31-03

As agreed in one previous community meeting, we decided to remove the step of uploading debug meta files because seems it is hardly used.

@thanos-io/thanos-maintainers

Describe the solution you'd like

Remove step

thanos/pkg/block/block.go

Lines 106 to 108 in 12c826a

if err := bkt.Upload(ctx, path.Join(DebugMetas, fmt.Sprintf("%s.json", id)), strings.NewReader(metaEncoded.String())); err != nil {
return cleanUp(logger, bkt, id, errors.Wrap(err, "upload debug meta file"))
}
and related tests code.

@hitanshu-mehta
Copy link
Contributor

Can i work on this issue?

@bwplotka
Copy link
Member

Yes please @hitanshu-mehta !

@pracucci
Copy link
Contributor

/cc @pstibrany

@GiedriusS
Copy link
Member

Maybe we should also remove whatever already meta files exist in the bucket already? #3397

@pstibrany
Copy link
Contributor

I have found these files useful when tracking down history of some blocks during our data conversion. I would weakly prefer to keep them, as the overhead seems tiny.

@stale
Copy link

stale bot commented Jun 2, 2021

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

@stale stale bot added the stale label Jun 2, 2021
@kakkoyun
Copy link
Member

kakkoyun commented Jun 3, 2021

Still valid.

@stale stale bot removed the stale label Jun 3, 2021
@stale
Copy link

stale bot commented Aug 2, 2021

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

@stale stale bot added the stale label Aug 2, 2021
@GiedriusS GiedriusS removed the stale label Aug 2, 2021
@yeya24
Copy link
Contributor Author

yeya24 commented Aug 4, 2021

I am wondering if this issue is still valid or not. There was a pr before but looks like we didn't reach an agreement.

@pstibrany
Copy link
Contributor

I think I was the only one for keeping the debug meta files. As I haven't had any use for them in last ~1year and instead suffered from trying to delete lots of them from object store, I am fine with getting rid of them.

(I hope my previous weak preference to keep them isn't what stalled the work here.)

@stale
Copy link

stale bot commented Oct 11, 2021

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

@stale stale bot added the stale label Oct 11, 2021
@stale
Copy link

stale bot commented Oct 30, 2021

Closing for now as promised, let us know if you need this to be reopened! 🤗

@stale stale bot closed this as completed Oct 30, 2021
@GiedriusS GiedriusS reopened this Oct 30, 2021
@stale stale bot removed the stale label Oct 30, 2021
@stale
Copy link

stale bot commented Jan 9, 2022

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

@stale stale bot added the stale label Jan 9, 2022
@yeya24
Copy link
Contributor Author

yeya24 commented Jan 9, 2022

Still valid

@stale stale bot removed the stale label Jan 9, 2022
@r0mdau
Copy link
Contributor

r0mdau commented Jan 29, 2022

I tried something, kudos to your work @hitanshu-mehta . I was trying to build thanos using this issue as example. I thought maybe it can helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants