Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loki resources leak #6978

Closed
tommaso-ascani opened this issue Jul 29, 2024 · 6 comments
Closed

Loki resources leak #6978

tommaso-ascani opened this issue Jul 29, 2024 · 6 comments
Assignees
@DavidePrincipi @tommaso-ascani
Labels
verified All test cases were verified successfully

Comments

@tommaso-ascani
Copy link

tommaso-ascani commented Jul 29, 2024 •
edited by DavidePrincipi
Loading

Steps to reproduce

Found a resources leak during initial tests of new loki cloud log manager forwarder.
The problem cause is still unknown.

See also

Components

  • Core: 2.9.1
  • Loki: 1.2.0 (grafana/loki 2.2.1)
@tommaso-ascani tommaso-ascani self-assigned this Jul 29, 2024
@github-project-automation github-project-automation bot moved this to 🆕 New in NethServer Jul 29, 2024
@tommaso-ascani tommaso-ascani mentioned this issue Jul 29, 2024
Merged
@DavidePrincipi DavidePrincipi moved this from 🆕 New to 🔖 Ready in NethServer Jul 29, 2024
@DavidePrincipi DavidePrincipi changed the title Loki - Resources leak Loki resources leak Jul 29, 2024
DavidePrincipi added a commit to NethServer/ns8-loki that referenced this issue Aug 7, 2024
@tommaso-ascani @renovate @DavidePrincipi
Resources leak bug fix (#15)
3b5ad89 
NethServer/dev#6978

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Davide Principi <davide.principi@nethesis.it>
@DavidePrincipi
Copy link
Member

In testing loki 1.2.1-dev.1

Test case

Test the bug is not reproducible: extended unicode chars do not crash log queries.

@DavidePrincipi DavidePrincipi moved this from 🏗 In progress to 👀 Testing in NethServer Aug 7, 2024
@DavidePrincipi DavidePrincipi added the testing Packages are available from testing repositories label Aug 7, 2024
@DavidePrincipi
Copy link
Member

DavidePrincipi commented Aug 7, 2024 •
edited
Loading

There's an ongoing issue with Loki 2.7+ grafana/loki#7649 (comment)

Let's wait for a while, if Loki developers fix it.

@DavidePrincipi DavidePrincipi removed the testing Packages are available from testing repositories label Aug 8, 2024
@DavidePrincipi DavidePrincipi moved this from 👀 Testing to 🏗 In progress in NethServer Aug 8, 2024
@DavidePrincipi DavidePrincipi mentioned this issue Sep 9, 2024
Merged
@DavidePrincipi DavidePrincipi self-assigned this Sep 9, 2024
DavidePrincipi added a commit to NethServer/ns8-core that referenced this issue Sep 10, 2024
@DavidePrincipi
Merge pull request #703 from NethServer/bug-6978-2
64849aa 
Silence "context canceled" messages in Loki

Refs NethServer/dev#6978
@DavidePrincipi
Copy link
Member

In testing:

  • core 2.9.3-dev.1
  • loki 1.2.1-dev.1

@DavidePrincipi DavidePrincipi added the testing Packages are available from testing repositories label Sep 10, 2024
@DavidePrincipi DavidePrincipi moved this from In progress to Testing in NethServer Sep 10, 2024
@DavidePrincipi DavidePrincipi moved this to In progress in NethServer Sep 11, 2024
@DavidePrincipi DavidePrincipi mentioned this issue Sep 11, 2024
Merged
DavidePrincipi added a commit to NethServer/ns8-core that referenced this issue Sep 11, 2024
@DavidePrincipi
fixup! Silence "context canceled" messages in Loki
8c0bc09 
Refs NethServer/dev#6978
DavidePrincipi added a commit to NethServer/ns8-loki that referenced this issue Sep 11, 2024
@DavidePrincipi
Merge pull request #25 from NethServer/bug-6978-2
d4a6ae5 
Bump Loki 2.9 for NethServer/dev#6978
@DavidePrincipi
Copy link
Member

In testing:

  • core 2.9.3-dev.4
  • loki 1.2.1-dev.2

@DavidePrincipi
Copy link
Member

VERIFIED

  1. Log searches returning lines with UTF8 Rune do not crash logcli any more
  2. After upgrade to 2.9 the retention releases disk space after 30 minutes
  3. Lines from loki-server containing "context canceled" are discarded by Promtail

@DavidePrincipi DavidePrincipi added verified All test cases were verified successfully and removed testing Packages are available from testing repositories labels Sep 11, 2024
@DavidePrincipi DavidePrincipi mentioned this issue Sep 11, 2024
Merged
DavidePrincipi added a commit to NethServer/ns8-loki that referenced this issue Sep 12, 2024
@DavidePrincipi
Limit log query result size (#26)
891a8f9 
Set a max number of records returned by Loki to avoid leaking memory.

Refs NethServer/dev#6978
@DavidePrincipi
Copy link
Member

Released

@github-project-automation github-project-automation bot moved this from In progress to Done in NethServer Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DavidePrincipi DavidePrincipi

@tommaso-ascani tommaso-ascani

Labels
verified All test cases were verified successfully
Projects
NethServer
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@DavidePrincipi @tommaso-ascani