Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding debug symbols and metadata for profiling kernels #612

Closed
georgemitenkov opened this issue Apr 23, 2021 · 3 comments · Fixed by #628
Closed

Adding debug symbols and metadata for profiling kernels #612

georgemitenkov opened this issue Apr 23, 2021 · 3 comments · Fixed by #628
Assignees
Labels

Comments

@georgemitenkov
Copy link
Collaborator

Issue: #592

We want to have some debug symbols that would allow us to profile the LLVM IR kernels. Consider adding these.

@georgemitenkov georgemitenkov self-assigned this Apr 23, 2021
@georgemitenkov georgemitenkov changed the title Adding debug symbols and metadata for profiling kernels. Adding debug symbols and metadata for profiling kernels Apr 23, 2021
@pramodk
Copy link
Contributor

pramodk commented Apr 23, 2021

Just a minor note: for #548, we need to add comment/annotation to the loop body:

$ → cat hh.1.s
....
# LLVM-MCA-BEGIN Body
LBB0_2:                                 ## %for.body
                                        ## =>This Inner Loop Header: Depth=1
	movq	%rsp, %rax
	addq	$-16, %rax
	movq	%rax, %rsp
	movq	%rsp, %rax
	addq	$-16, %rax
	movq	%rax, %rsp
	movq	-16(%rbp), %rax
	movslq	-4(%rbp), %rcx
	movq	(%rax), %rdx
	movq	8(%rax), %rsi
	movq	16(%rax), %rax
	vmovsd	(%rdx,%rcx,8), %xmm0            ## xmm0 = mem[0],zero
	vsubsd	(%rax,%rcx,8), %xmm0, %xmm0
	vdivsd	(%rsi,%rcx,8), %xmm0, %xmm0
	vmovsd	%xmm0, (%rax,%rcx,8)
	incl	-4(%rbp)
	movq	-16(%rbp), %rax
	movl	-4(%rbp), %ecx
	cmpl	92(%rax), %ecx
	jl	LBB0_2
# LLVM-MCA-END

In case this is minor/easy to do with few lines. Otherwise don't worry for now.

@georgemitenkov
Copy link
Collaborator Author

Do you mean annotation that it is a loop? In this case, I have actually made a ticket for that yesterday: #607. (Also: #606). In LLVM there is llvm.loop metadata that Clang emits, so that on I we know that the basic blocks are from the loop 👍

Or you mean debug symbols/metadata in general for profiling?

@pramodk
Copy link
Contributor

pramodk commented Apr 23, 2021

May be I shouldn't have mixed the topic in this ticket. Let's get debug symbols/metadata in general for profiling for JIT kernels, as we discussed. I will add separate ticket regarding my comment above (this is very minor/low priority thing)

@georgemitenkov georgemitenkov linked a pull request May 7, 2021 that will close this issue
georgemitenkov added a commit that referenced this issue May 8, 2021
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612 

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
pramodk added a commit that referenced this issue May 10, 2021
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612 

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
olupton added a commit that referenced this issue Aug 20, 2021
Fixes NVHPC compiler warning: warning #612-D: overloaded virtual
function "nmodl::ast::Ast::get_shared_ptr" is only partially
overridden in class "PyAst".
olupton added a commit that referenced this issue Aug 23, 2021
Fixes NVHPC compiler warning: warning #612-D: overloaded virtual
function "nmodl::ast::Ast::get_shared_ptr" is only partially
overridden in class "PyAst".
olupton added a commit that referenced this issue Aug 26, 2021
* Tweak NVHPC warning suppressions. This makes them
  work properly in standalone builds, when NMODL is not
  a submodule of CoreNEURON.
* Add const version of PyAst::get_shared_ptr. Fixes
  NVHPC compiler warning: warning #612-D: overloaded
  virtual function "nmodl::ast::Ast::get_shared_ptr" is
  only partially overridden in class "PyAst".
* Remove unused variable. Fixes compiler warning.
* Tweak code so clang-format-{11,12} agree. Otherwise
  clang-format-12 added whitespace that
  clang-format-11 did not.
* Call flex and bison with relative paths. This avoids
  #line directives and asserts containing the absolute
  path of the build directory, which should in turn
  improve ccache performance in the CI.
* Fix comment and add note about ccache misses.
pramodk added a commit that referenced this issue Mar 8, 2022
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
iomaganaris pushed a commit that referenced this issue May 10, 2022
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
iomaganaris pushed a commit that referenced this issue May 12, 2022
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
iomaganaris pushed a commit that referenced this issue Sep 15, 2022
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
iomaganaris pushed a commit that referenced this issue Sep 15, 2022
Added debug support to LLVM code generation pipeline. Currently,
only basic support was added:

1. Debug information about functions (name)
2. Debug information about module

**What has been changed and added**

1. A new class `DebugBuilder` was created. It is used as a wrapper
around LLVM's `DIBuilder` and holds important information such as
`LLVMContext`, debug file and compile unit. It also wraps `DIBuilder`'s
functionality into a more suitable API.

2. A temporary `Location` struct has been added. It encapsulates the
location of the source AST construct and reflects `ModToken` on LLVM
code generation level. It is only used if the location of the source NMODL
function is known.

3. LLVM visitor know takes an extra `add_debug_information` flag and
handles debug information creation. Fore readability, `IRBuilder` was
renamed to `ir_builder`.

4. JIT runner is now able to listen for GDB, perf (build LLVM with
`-DLLVM_USE_PERF=ON`) and VTune (build LLVM with
`-DLLVM_USE_INTEL_JITEVENTS=ON`) events.

5. Necessary cmake changes were added to optionally support JIT event
listeners (`-DNMODL_HAVE_JIT_EVENT_LISTENERS`).

**How to generate debug information**

Debug information is attached to every function, procedure or artificially
created kernel (and corresponding wrappers). Debug information is enable
by default, so to turn it off use ` --disable-debug-info` flag. For example,
the given NMODL
```nmodl
1   FUNCTION func(x) {
2     func = x
3   }
4
5   PROCEDURE proc() {}
```
is transformed (running `./bin/nmodl <filename>.mod llvm --ir`) into
```llvm
define double @func(double %x1) !dbg !4 {
  ; ...
}

define i32 @proc() !dbg !6 {
  ; ...
}

!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!3}

!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "NMODL-LLVM", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
!1 = !DIFile(filename: "foo", directory: ".")
!2 = !{}
!3 = !{i32 2, !"Debug Version", i32 3}
!4 = distinct !DISubprogram(name: "func", linkageName: "func", scope: null, file: !1, line: 1, type: !5, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
!5 = !DISubroutineType(types: !2)
!6 = distinct !DISubprogram(name: "proc", linkageName: "proc", scope: null, file: !1, line: 5, type: !5, scopeLine: 5, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !2)
```

fixes #592 #612

Co-authored-by: Pramod Kumbhar <pramod.s.kumbhar@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants