gdb Pretty Print: generic encoded was failing on reference/pointer types #33612
Conversation
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @nikomatsakis (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
Hmm, do we have any unit testing for these pretty printers? |
|
We had tests for this at some point: I don't remember why they were removed. |
|
@bors: r+ |
|
📌 Commit 04baf25 has been approved by |
…lwoerister
gdb Pretty Print: generic encoded was failing on reference/pointer types
If you debug this program using **gdb**
```rust
fn main() {
let x = 10;
let y = Some(&x);
// additional code
}
```
And you try to print **y**'s value from the debugger, you get the following:
```
(gdb) print y
Python Exception <class 'gdb.error'> Cannot convert value to int.:
$1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}}
```
What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails.
Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration.
After this modification **gdb** prints **y** like this:
```
(gdb) print y
$1 = Some = {0x7fff5fbff97c}
```
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
If you debug this program using gdb
And you try to print y's value from the debugger, you get the following:
What happens is that inside debugger_pretty_printers_common.py the method
is_null_variantdoesn't have any special handling for pointer values so it ends up calling.as_integer()ondiscriminant_val(which holds a pointer) and fails.Considering it needs to handle pointers and return true when the pointer is null, I modified the
.as_integer()method in gdb_rust_pretty_printing.py to take pointers into consideration.After this modification gdb prints y like this:
Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.