Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Counting runs count takes too much MongoDB CPU #2199

Open
benedeki opened this issue Nov 13, 2023 · 1 comment · May be fixed by #2207
Open

Counting runs count takes too much MongoDB CPU #2199

benedeki opened this issue Nov 13, 2023 · 1 comment · May be fixed by #2207
Labels
bug Something isn't working DB Database affected priority: high Critical to the health of the project Rest-API Rest-API component affected

Comments

@benedeki
Copy link
Collaborator

Describe the bug

When application is counting Runs count for the landing page info, it consumer too much CPU.

To Reproduce

See server logs

Expected behavior

It might not be necessary to run full scan of the table.

Log

2023-10-27T06:31:47.590+0000 I COMMAND  [conn7104853] command menas_galactic.run_v1 command: aggregate { aggregate: "run_v1", pipeline: [ { $match: {} }, { $group: { _id: 1, n: { $sum: 1 } } } ], cursor: {}, $db: "menas_galactic", $clusterTime: { clusterTime: Timestamp(1698388263, 159), signature: { hash: BinData(0, E86133DC3872EAE27881D51B179B0E150F1E874A), keyId: 7256402329547571201 } }, lsid: { id: UUID("5a5b1fa6-3fc7-4f36-9a38-e9ae6c73e7cd") } } planSummary: COLLSCAN keysExamined:0 docsExamined:6077042 cursorExhausted:1 numYields:47554 nreturned:1 reslen:259 locks:{ Global: { acquireCount: { r: 47556 } }, Database: { acquireCount: { r: 47556 } }, Collection: { acquireCount: { r: 47556 } } } storage:{ data: { bytesRead: 15558954293, timeReadingMicros: 17675795 }, timeWaitingMicros: { cache: 2796143 } } protocol:op_msg 42437ms
@benedeki benedeki added bug Something isn't working priority: high Critical to the health of the project Rest-API Rest-API component affected DB Database affected labels Nov 13, 2023
benedeki added a commit that referenced this issue Nov 13, 2023
@benedeki
#2199: Counting runs count takes too much MongoDB CPU
e98ee00 
* replaced DB-intensive `documentCount` for `estimatedDocumentCount`
@benedeki benedeki mentioned this issue Nov 13, 2023
Merged
benedeki added a commit that referenced this issue Dec 8, 2023
@benedeki
#2199: Counting runs count takes too much MongoDB CPU (#2200)
fc5a512 
* replaced DB-intensive `documentCount` for `estimatedDocumentCount`
@benedeki
Copy link
Collaborator Author

benedeki commented Dec 8, 2023

Release notes:
Replaced DB-intensive documentCount for estimatedDocumentCount. That should improve the load on MongoDB

@benedeki benedeki linked a pull request Dec 21, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working DB Database affected priority: high Critical to the health of the project Rest-API Rest-API component affected
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

#2199 Counting runs count takes too much MongoDB CPU AbsaOSS/enceladus
1 participant
@benedeki