Skip to content

Commit

Permalink
Basic support of ES GC metrics for jvm9 (elastic#7628)
Browse files Browse the repository at this point in the history
GC log format for JVM9 is more detailed than for JVM8.

Differences and possible improvements:
* To get cpu_times.* a corellation between log lines is required.
* Some GC metrics are available in jvm8 are not in jvm9
  (class_unload_time_sec, weak_refs_processing_time_sec, ...)
* heap.used_kb is empty, but it can be calculated as young_gen.used_kb +
  old_gen.size_kb
* GC phase times are logged in miliseconds vs seconds in jvm8
  • Loading branch information
immon authored and ruflin committed Jul 18, 2018
1 parent 28ec0a4 commit e8ae08a
Show file tree
Hide file tree
Showing 2 changed files with 848 additions and 1 deletion.
4 changes: 3 additions & 1 deletion filebeat/module/elasticsearch/gc/ingest/pipeline.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
"(?:%{JVM8HEADER}|%{JVM9HEADER}) Total time for which application threads were stopped: %{BASE10NUM:elasticsearch.gc.threads_total_stop_time_sec} seconds, Stopping threads took: %{BASE10NUM:elasticsearch.gc.stopping_threads_time_sec} seconds",
"(?:%{JVM8HEADER}) \\[GC \\(%{DATA:elasticsearch.gc.phase.name}\\) \\[YG occupancy: %{BASE10NUM:elasticsearch.gc.young_gen.used_kb} K \\(%{BASE10NUM:elasticsearch.gc.young_gen.size_kb} K\\)\\]%{BASE10NUM}: \\[Rescan \\(parallel\\) , %{BASE10NUM:elasticsearch.gc.phase.parallel_rescan_time_sec} secs\\]%{BASE10NUM}: \\[weak refs processing, %{BASE10NUM:elasticsearch.gc.phase.weak_refs_processing_time_sec} secs\\]%{BASE10NUM}: \\[class unloading, %{BASE10NUM:elasticsearch.gc.phase.class_unload_time_sec} secs\\]%{BASE10NUM}: \\[scrub symbol table, %{BASE10NUM:elasticsearch.gc.phase.scrub_symbol_table_time_sec} secs\\]%{BASE10NUM}: \\[scrub string table, %{BASE10NUM:elasticsearch.gc.phase.scrub_string_table_time_sec} secs\\]\\[1 CMS-remark: %{BASE10NUM:elasticsearch.gc.old_gen.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.old_gen.size_kb}K\\)\\] %{BASE10NUM:elasticsearch.gc.heap.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.heap.size_kb}K\\), %{BASE10NUM:elasticsearch.gc.phase.duration_sec} secs\\] %{PROCTIME}",
"(?:%{JVM8HEADER}) \\[GC \\(%{DATA:elasticsearch.gc.phase.name}\\) \\[%{BASE10NUM} CMS-initial-mark: %{BASE10NUM:elasticsearch.gc.old_gen.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.old_gen.size_kb}K\\)\\] %{BASE10NUM:elasticsearch.gc.heap.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.heap.size_kb}K\\), %{BASE10NUM:elasticsearch.gc.phase.duration_sec} secs\\] %{PROCTIME}",
"%{JVM9HEADER} GC\\(%{BASE10NUM}\\) ParNew: %{BASE10NUM}K-\\>%{BASE10NUM:elasticsearch.gc.young_gen.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.young_gen.size_kb}K\\)",
"%{JVM9HEADER} GC\\(%{BASE10NUM}\\) Old: %{BASE10NUM}K-\\>%{BASE10NUM:elasticsearch.gc.old_gen.used_kb}K\\(%{BASE10NUM:elasticsearch.gc.old_gen.size_kb}K\\)",
"(?:%{JVM8HEADER}|%{JVM9HEADER}) %{GREEDYMULTILINE:message}"
],
"pattern_definitions": {
Expand Down Expand Up @@ -54,4 +56,4 @@
}
}
]
}
}
Loading

0 comments on commit e8ae08a

Please sign in to comment.