Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Auto merge of #33612 - royalstream:royalstream-enc-enum-ptr, r=michae…
…lwoerister gdb Pretty Print: generic encoded was failing on reference/pointer types If you debug this program using **gdb** ```rust fn main() { let x = 10; let y = Some(&x); // additional code } ``` And you try to print **y**'s value from the debugger, you get the following: ``` (gdb) print y Python Exception <class 'gdb.error'> Cannot convert value to int.: $1 = {RUST$ENCODED$ENUM$0$None = Some = {0x7fff5fbff97c}} ``` What happens is that inside **debugger_pretty_printers_common.py** the method `is_null_variant` doesn't have any special handling for pointer values so it ends up calling `.as_integer()` on `discriminant_val` (which holds a pointer) and fails. Considering it needs to handle pointers and return _true_ when the pointer is _null_, I modified the `.as_integer()` method in **gdb_rust_pretty_printing.py** to take pointers into consideration. After this modification **gdb** prints **y** like this: ``` (gdb) print y $1 = Some = {0x7fff5fbff97c} ``` Now, it would be nice to print something useful (instead of a pointer address) but the pretty printer doesn't currently handle references/pointers so that's a completely different subject.
- Loading branch information