Skip to content

Commit 075a967

Browse files
Josipmrdenkgolubic
Josipmrden
and
kgolubic
authored
Peak memory observation in the system (#788)
* Peak memory observation in the system * Update storage info where needed * Update best practics * Add information from PR review --------- Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>
1 parent 0433d0c commit 075a967

File tree

3 files changed

+27
-8
lines changed

3 files changed

+27
-8
lines changed

‎pages/data-migration/best-practices.mdx‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -545,6 +545,21 @@ of the `--storage-gc-cycle-sec` flag. That means that Memgraph's garbage
545545
collector potentially still didn't deallocate unused objects and free the
546546
memory. You can free up memory by running the `FREE MEMORY` query.
547547

548+
## Monitoring memory during import
549+
550+
Often times, the user needs to batch transactions in the `IN_MEMORY_TRANSACTIONAL` storage mode.
551+
This is due to large-update transactions that are creating a lot of `Delta` objects during the
552+
transaction lifetime. `Delta` objects are necessary for reverting the query if an error during
553+
query execution happens. They are approximately of 56 bytes in size, but large-update transactions
554+
can make the number of delta objects be huge - therefore resulting in Memgraph getting out of memory.
555+
556+
We have exposed the `peak_memory_res` (peak resident memory) variable in the `SHOW STORAGE INFO;` command. with which you
557+
can monitor when the peak resident memory rises in the system.
558+
It will help you diagnose bottlenecks and high-memory queries which can be optimized.
559+
That is most common in import queries because users would want to import the whole dataset with one command only.
560+
561+
For more information, check our [storage memory usage](/fundamentals/storage-memory-usage).
562+
548563
## Do you have more questions?
549564

550565
Schedule a 30 min session with one of our engineers to discuss how Memgraph fits

‎pages/database-management/server-stats.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,15 @@ The result will contain the following fields:
2929

3030
| Field | Description |
3131
|------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
32+
| name | Name of the current database. |
3233
| vertex_count | The number of stored nodes (vertices). |
3334
| edge_count | The number of stored relationships (edges). |
3435
| average_degree | The average number of relationships of a single node. |
36+
| vm_max_map_count | The number of memory-mapped areas that the kernel allows a process to have. If it is unknown, returns -1. </br> For more info, check out [virtual memory section of the docs](/fundamentals/storage-memory-usage#virtual-memory). |
3537
| memory_res | The non-swapped physical RAM memory a task has used, reported by the OS (in B, KiB, MiB, GiB or TiB). |
38+
| peak_memory_res | Peak RAM memory usage in the system during the whole run. |
3639
| disk_usage | The amount of disk space used by the data directory (in B, KiB, MiB, GiB or TiB). |
3740
| memory_tracked | The amount of RAM allocated in the system and tracked by Memgraph (in B, KiB, MiB, GiB or TiB).<br/>For more info, check out [memory control](/fundamentals/storage-memory-usage). |
38-
| vm_max_map_count | The number of memory-mapped areas that the kernel allows a process to have. If it is unknown, returns -1. </br> For more info, check out [virtual memory section of the docs](/fundamentals/storage-memory-usage#virtual-memory). |
3941
| allocation_limit | The current allocation limit set for this instance (in B, KiB, MiB, GiB or TiB).<br/>For more info, check out the [memory control](/fundamentals/storage-memory-usage#control-memory-usage). |
4042
| global_isolation_level | The current `global` isolation level.<br/>For more info, check out [isolation levels](/fundamentals/transactions#isolation-levels). |
4143
| session_isolation_level | The current `session` isolation level. |

‎pages/fundamentals/storage-memory-usage.mdx‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -729,13 +729,15 @@ SHOW STORAGE INFO;
729729
| storage info | value |
730730
+--------------------------------+--------------------------------+
731731
| "name" | "memgraph" |
732-
| "vertex_count" | 2677 |
733-
| "edge_count" | 11967 |
734-
| "average_degree" | 8.94061 |
735-
| "memory_usage" | "44.32MiB" |
736-
| "disk_usage" | "133.63KiB" |
737-
| "memory_allocated" | "6.74MiB" |
738-
| "allocation_limit" | "15.28GiB" |
732+
| "vertex_count" | 0 |
733+
| "edge_count" | 0 |
734+
| "average_degree" | 0 |
735+
| "vm_max_map_count" | 453125 |
736+
| "memory_res" | "43.16MiB" |
737+
| "peak_memory_res" | "43.16MiB" |
738+
| "disk_usage" | "104.46KiB" |
739+
| "memory_tracked" | "8.52MiB" |
740+
| "allocation_limit" | "58.55GiB" |
739741
| "global_isolation_level" | "SNAPSHOT_ISOLATION" |
740742
| "session_isolation_level" | "" |
741743
| "next_session_isolation_level" | "" |

0 commit comments

Comments
 (0)