Skip to content

Flame Graph Report

Daniel Odievich edited this page Nov 5, 2019 · 18 revisions

Overview

Flame Graphs and Flame Chart reports visualize a collection of stack traces (aka call stacks) in an adjacency diagram with an inverted icicle layout. They are a useful way to visualize many call graphs in single screen.

You can also watch this training video:

AppDynamics DEXTER - Flame Graphs

Flame Graphs report is produced when:

Report File Locations

Flame Graph report for Entities has the following file name:

Report\<Controller>\<Application>\<APP|TIER|NODE|BT>\FlameGraph.<ControllerName>.<ApplicationName>.<EntityName>.<Start Date and Time>-<End Date and Time>.svg
Report\<Controller>\<Application>\<APP|TIER|NODE|BT>\FlameChart.<ControllerName>.<ApplicationName>.<EntityName>.<Start Date and Time>-<End Date and Time>.svg

For example:

demo2.appdynamics.com\ECommerce.41\APP\FlameGraph.Application.demo2.appdynamics.com.ECommerce.41.2017123001-2017123003.svg
demo2.appdynamics.com\ECommerce.41\APP\FlameChart.Application.demo2.appdynamics.com.ECommerce.41.2017123001-2017123003.svg

What is Flame Graph

A Flame Graph visualizes a collection of stack traces (aka call stacks), shown as an adjacency diagram with an inverted icicle layout. Flame Graphs are commonly used to visualize CPU profiler output, where stack traces are collected using sampling.

Flame Graphs and Flame Charts (a type of Flame Graph with time dimension on the X axis) provide an excellent tool to understand the frequent execution paths your application is taking, visualize dependencies between various frameworks and identify performance problems.

Brendan Gregg’s website http://www.brendangregg.com/flamegraphs.html and his talk https://queue.acm.org/detail.cfm?id=2927301 are excellent in helping understand the concepts behind Flame Graphs. Couple other essential reads on using Flame Graphs are https://medium.com/netflix-techblog/java-in-flames-e763b3d32166 and https://medium.com/netflix-techblog/node-js-in-flames-ddd073803aa4.

A Flame Graph has the following characteristics:

  • A stack trace is represented as a column of boxes, where each box represents a function (a stack frame).
  • The y-axis shows the stack depth, ordered from root at the bottom to leaf at the top. The function beneath a function is its parent.
  • The x-axis spans the stack trace collection. It does not show the passage of time, so the left-to-right ordering has no special meaning. The left-to-right ordering of stack traces is performed alphabetically on the function names, from the root to the leaf of each stack. This maximizes box merging: when identical function boxes are horizontally adjacent, they are merged.
  • The width of each function box shows the frequency at which that function was present in the stack traces, or part of a stack trace ancestry. Functions with wide boxes were more present in the stack traces than those with narrow boxes, in proportion to their widths.
  • If the width of the box is sufficient, it displays the full function name. If not, either a truncated function name with an ellipsis is shown, or nothing.
  • The background color for each box is mapped to the well known frameworks as defined in Method Call Mapping.
  • The profile visualized may span a single thread, multiple threads, multiple applications, or multiple hosts.

A Flame Chart changes from Flame Graph in the following way:

  • The x-axis adds the passage of time with hour, 10 minute and 1 minute synthetic frames.

Accessing Flame Graph Reports

Flame Graphs and Flame Charts reports for Application, Tiers, Nodes and Business Transactions are linked to from Detected Entity and Entity Metrics reports from FlameGraphLink and FlameGraphChart columns.

Full Size

Flame Graph reports for Snapshots are linked to from the Snapshots report from FlameGraphLink column.

Full Size

Flame Graphs and Flame Chart reports are SVG files that are best viewed in your favorite internet browser.

Interacting with Report

The vertical arrangement shows depth of stack calls, starting at the bottom. The topmost stack is on CPU.

Stack frame color-codes certain frameworks (Java, Sun, Oracle, Google, Amazon, Microsoft) with some colors, leaving default colors range from burnt orange at Stack depth 0 to yellow at maximum stack depth. Colors are defined in as defined in Method Call Mapping.

The horizontal arrangement is alphabetical.

Stack frame width is proportional to its percentage in samples. Wider -> more samples.

Stack frames with tiny width are filtered out.

The timing data for each frame is included. Methods with execution time higher than 50ms are highlighted with Red border, growing in thickness as the duration increases.

Colors are assigned to the types of framework. Legend in upper-right corner shows what they are. Click "Legend" on the top row to show/hide legend.

Hovering mouse over any block gives you its contents as tooltip.

The top line specifies which Entity you are looking at.

This is Flame Graph for Application:

Full Size

This is Flame Graph for Tier:

Full Size

This is Flame Graph for Node:

Full Size

This is Flame Graph for Business Transaction:

Full Size

Visualization of Duration of Calls

The red border around the frame indicates ranges of the average duration of the call in milliseconds. It is calculated by dividing the total time by number of samples:

Duration From Duration To Line thickness
0 100 None
100 250 1
250 500 2
500 1000 3
1000 5000 4
5000 10000 5
10000 Infinity 6

Visualization of Exit Calls

The dashed red or blue border around the frame indicates that the call contains an Exit to a database, queue or web destination. Mouse over the block to see the details of exit calls

Zooming

Clicking a single block trims the report to just that element:

Full Size

You can continue to filter by clicking anywhere.

Full Size

Use Reset Zoom to go back to full view.

Search for and Highlight Frame

Click Search or Ctrl-F to open search dialog, type your query (regex is supported):

Full Size

Results are highlighted in purple. The Matched % is number of samples against total number of samples in all call stacks.

Full Size

You can zoom in:

Full Size

Use Reset Search to remove search highlights.

Clone this wiki locally