Add new record filter tool for public release of traces #6662

edeiana · 2024-02-20T04:33:38Z

We want to create a new tool to filter traces of Google workloads for public release.
The new public Google workload traces will contain more information compared to the previous version, while still preserving confidentiality of Google's IP.

Main features of new public traces:

Instruction categories will be used instead of the original instruction opcode;
Operand dependencies will be preserved using virtual registers + register size;
Syscall numbers will be removed as before, but we'll keep the blocking attribute, if a syscall is blocking;
Branch target addresses will be embedded in the trace, instead of being in a separate file;
Provide a map of all virtual addresses used in the trace and a tool to perform the virtual-to-physical mapping.

Simple record_filter_t::record_filter_func_t filter that modifies a field of every trace_entry_t record in a trace and leverages record_filter to output such modified records onto a "filtered" output trace. Issue: #6662

Containing-register IDs can be >=256, hence their value does not fit in the allotted 8 bits per register operand of regdeps encoding. This was causing a memory corruption in instr_convert_to_isa_regdeps() where src_reg_used and dst_reg_used have only 256 elements and are laid out next to each other in memory. Writing to index >=256 into one was overwriting the other. Fix: remap containing-register IDs to virtual-register IDs starting from 0 for all architectures. We still have only up to 198 unique containing registers (max number of containing registers for AARCH64), so remapping them allows to fit them in 8 bits. In the re-mapping (from DR_REG_ to DR_REG_V) we exclude DR_REG_INVALID to avoid issues with opnd_t operations for registers. We introduce 2 new public APIs: dr_reg_to_virtual() and get_virtual_register_name(). We use dr_reg_to_virtual() in instr_convert_to_isa_regdeps() to avoid the issue mentioned above. We also re-introduce setting the size for register operands in instr_convert_to_isa_reg_deps() and decode_isa_regdeps() as instr_t.operation_size because DR_REG_V don't have predefined size. We added tests to check that DR_REG_ with IDs >=256 don't cause problems. Issue: #6662

Containing-register IDs can be >=256, hence their value does not fit in the allotted 8 bits per register operand of regdeps encoding. This was causing a memory corruption in instr_convert_to_isa_regdeps() where src_reg_used and dst_reg_used have only 256 elements and are laid out next to each other in memory. Writing to index >=256 into one was overwriting the other. Fix: remap containing-register IDs to virtual-register IDs starting from 0 for all architectures. We still have only up to 198 unique containing registers (max number of containing registers for AARCH64), so remapping allows to fit them in 8 bits. In the re-mapping (from DR_REG_ to DR_REG_V) we exclude DR_REG_INVALID and DR_REG_NULL to avoid issues with opnd_t operations for registers. We introduce a private routine dr_reg_to_virtual() to do the mapping from real ISA to virtual register. We use it in instr_convert_to_isa_regdeps() to avoid the issue mentioned above. We modified the get_register_name() public API to use the global dcontext and its ISA mode to determine whether to return a real register name or a virtual one. The signature of the API remained the same, but we document the use of the global dcontext in doxygen. We also re-introduce setting the size for register operands in instr_convert_to_isa_reg_deps() and decode_isa_regdeps() as instr_t.operation_size because not all DR_REG_V have a predefined size based on their enum value (e.g., reserved DR_REG_XMM enum values). We added tests to check that DR_REG_ with IDs >=256 don't cause problems. Issue: #6662

AssadHashmi · 2024-04-25T14:10:13Z

We'd like to raise a point of discussion about the use of DR_REG_V as a prefix for virtual registers.
Most Arm programmers' documentation uses V for vector registers, e.g.ADD Vd.4S, Vn.4S, Vm.4S.

The DynamoRIO user documentation uses R for reasons of convention and generality, e.g.INSTR_CREATE_add_vector(dc, Rd, Rm, Rn, width), see:
https://dynamorio.org/dr__ir__macros__aarch64_8h.html#ad6fa6d2ab7764783481efd209cf11b76

A new or relatively inexperienced user may probably and intuitively but wrongly try something like this first:

INSTR_CREATE_add_vector(dc, opnd_create_reg(DR_REG_V0), opnd_create_reg(DR_REG_V1), opnd_create_reg(DR_REG_V2), OPND_CREATE_SINGLE())

Rather than the correct:

INSTR_CREATE_add_vector(dc, opnd_create_reg(DR_REG_Q0), opnd_create_reg(DR_REG_Q1), opnd_create_reg(DR_REG_Q2), OPND_CREATE_SINGLE())

or e.g.

INSTR_CREATE_add_vector(dc, opnd_create_reg(DR_REG_D20), opnd_create_reg(DR_REG_D10), opnd_create_reg(DR_REG_D14), OPND_CREATE_HALF())

This possible stumbling block is probably less of a problem for scalable vectors, e.g. INSTR_CREATE_add_sve(dc, Zd, Zn, Zm), because the documentation implies DR_REG_Z, see
https://dynamorio.org/dr__ir__macros__aarch64_8h.html#a14d0f0b7fad176b301539c3e9254771b

What do you think? Are we worrying unnecessarily?
Is the level of DR knowledge required to use instruction macros in e.g. clients, enough for developers to know which register names/IDs are correct?

derekbruening · 2024-04-25T14:25:37Z

s/DR_REG_V0/DR_REG_VIRT0/?

AssadHashmi · 2024-04-25T17:48:58Z

s/DR_REG_V0/DR_REG_VIRT0/?

That's fine.

edeiana · 2024-04-25T18:15:54Z

Thank you for pointing this out @AssadHashmi !
Will change from DR_REG_V to DR_REG_VIRT as @derekbruening suggested.

New record_filter_t::record_filter_func_t filter, which we call "encodings2regdeps", that modifies the encoding of trace_entry_t records from a real ISA to the synthetic regdeps ISA. "encodings2regdeps" can add or remove trace_entry_t records that contain encodings depending on the regdeps encoding size. Note that "encodings2regdeps" only changes the encoding of instructions, but it does not adjust their length (or changes the instruction PC), hence the output trace will have encoding sizes that do not match the instruction length. For this reason we disable the encoding size vs instruction length check in reader_t when the trace has DR_ISA_REGDEPS encodings. This filter is part of the "record_filter" tool and can be invoked with: ``` drrun -t drcachesim -simulator_type record_filter -filter_encodings2regdeps -indir path/to/input/trace -outdir path/to/output/trace ``` Issue: #6662

Fixes a size mismatch between dr_reg_fixer[] and d_r_reg_id_to_virtual[] maps in aarch64. Adds a check in encode_debug_checks() (in core/ir/${ARCH}/encode.c) for all architecures. Issue: #6662, #3544, #1569

Fixes a data race due to multiple dr_standalone_init() done in parallel (per shard) by encodings2regdeps_filter_t. dcontext is now initialize one time by record_filter_t and passed to its filters through the record_filter_info_t interface. Issue #6662

Fixes a data race due to multiple dr_standalone_init() done in parallel (per shard) by encodings2regdeps_filter_t. dcontext is now initialized one time by record_filter_t and passed to its filters through the record_filter_info_t interface. Avoids a data race in opcode_mix where all threads set the dcontext isa_mode to DR_ISA_REGDEPS for regdeps input traces. Issue #6662 #6812

Adds disassembling for DR_ISA_REGDEPS instructions. Specifically, when we disassemble DR_ISA_REGDEPS instructions, we print the instruction encoding (divided in 4 byte words, can span one or two lines, similar to x86), we substitute the opcode with categories, we print the operations size (e.g., `[4byte]`), and then the source and destination virtual register names (e.g., `%rv3`). Disassembled instructions look as follows: `00000812 06260606 load [8byte] %rv4 -> %rv4 %rv36` In general, they follow this pattern: [encoding (in 4 byte words)] [categories] [operation_size] [src_regs -> dst_regs] Issue: #6662

Modifies the view tool to handle OFFLINE_FILE_TYPE_ARCH_REGDEPS traces, leveraging the disassembly of DR_ISA_REGDEPS instructions. When visualizing DR_ISA_REGDEPS instructions, the view tool still prints the instruction length and PC, which for OFFLINE_FILE_TYPE_ARCH_REGDEPS traces are the same as those in the original trace. Then, after the PC, the instruction encoding, categories, operation size, and registers are printed following the disassembly format of DR_ISA_REGDEPS instructions (xref: #6799). DR_ISA_REGDEPS instructions printed by the view tool look as follows: ``` [...] ifetch 10 byte(s) @ 0x00007f86ef03d107 00001931 04020204 load store [4byte] %rv0 %rv2 %rv36 -> %rv0 [...] 00000026 ``` We also fix a formatting bug in DR_ISA_REGDEPS instruction disassembly, where we were missing a new line when the instruction encoding spills into a second line. Issue: #6662

The public version of a trace needs to preserve the TRACE_MARKER_TYPE_FUNC_[ID | ARG | RETVAL] markers related to SYS_futex system calls. We add this feature to encodings2regdeps_filter_t. This filter still drops the TRACE_MARKER_TYPE_FUNC_ markers related to other functions that are not SYS_futex. However, we still rely on type_filter_t to remove the additional markers that we don't want to preserve in the public trace. Issue #6662

Adds a new filter: `func_id_filter_t` to record_filter, which filters TRACE_MARKER_TYPE_FUNC_ markers based on the function ID. The filter is enabled by `-filter_keep_func_ids` followed by a list of integers that represent the function IDs bound to TRACE_MARKER_TYPE_FUNC_ markers to keep in the trace. Specifically, whenever we encounter a TRACE_MARKER_TYPE_FUNC_ID marker whose marker value is in the list we set a per-shard flag to indicate that all TRACE_MARKER_TYPE_FUNC_[ID | ARG | RETVAL | RETADDR] markers related to that function ID need to be preserved. We remove the TRACE_MARKER_TYPE_FUNC_ markers related to functions whose ID is not in the list. This filter can be invoked with: ``` drrun -t drmemtrace -tool record_filter -filter_keep_func_ids 1,2,3,4 -indir path/to/input/trace -outdir path/to/output/trace ``` To preserve TRACE_MARKER_TYPE_FUNC_ markers related to functions with ID: 1, 2, 3, 4, and remove the TRACE_MARKER_TYPE_FUNC_ markers for all other ID values. We use this filter to preserve markers related to SYS_futex functions in the public release of traces. Issue #6662

When filtering function related markers by function ID we need to provide the original marker value (i.e., function ID) in the trace. When filtering out system calls, the ID can be large due to TRACE_FUNC_ID_SYSCALL_BASE (which is 0x100000000ULL == 4294967296 for x64) that is added to the system call number, and can cause an std::out_of_range error. We now use std::stoll to handle large function IDs. Issue #6662

Implementation of a simple reader for virtual to physical address mapping in textproto format. The reader parses blocks in the following form: ``` address_mapping { virtual_address: 0x123 physical_address: 0x3 } ``` present in the provided textproto file to create a map in memory from virtual to physical addresses. The reader also parses the file to obtain the page size, page count, and number of bytes that were mapped. All the previous information is saved in a new struct `v2p_info_t`. We plan to include this virtual to physical mapping in textproto format in the release of public traces to aid users in experimenting with our TLB simulator using physical addresses. Adds unit tests and a sample v2p textproto file to check the correctness of the generated map. Issue #6662

We want to provide a virtual to physical mapping for the release of public traces to be used with our TLB simulator. We enable the TLB simulator to operate on the physical addresses provided by a v2p.textproto file, which is read by the previously added v2p_reader. Adds an end-to-end test "tool.drcacheoff.tlb_simulator_v2p" and a v2p.textproto mapping to clients/drcachesim/tests/drmemtrace.threadsig.aarch64.raw. Adds unit tests to check that the addresses used by the TLB simulator are the actual physical ones. Issue #6662

In REGDEPS public traces the encodings of instructions printed with the `view` tool are divided in 4 byte words, 2 words on the first line and an additional 2 on the second line (if necessary). Instructions that only have 1 word encodings (e.g., a branch) have their categories printed right after the encodings of their first word and don't align with the categories of 2 or more words instructions. For single word instructions we now add spacing equivalent to a second word. Issue #6662

In REGDEPS public traces the encodings of instructions printed with the `view` tool (which uses our disassembler) are divided in 4 byte words, 2 words on the first line and an additional 2 on the second line (if necessary). Instructions that only have 1 word encodings (e.g., a branch) have their categories printed right after the encodings of their first word and don't align with the categories of 2 or more word instructions. For single word instructions we now add spacing equivalent to a second word. So, we now have: ``` 102268 78352: 1271326 ifetch 3 byte(s) @ 0x00005605ec276c4a 00040110 00000206 math [8byte] %rv0 102269 78353: 1271326 ifetch 2 byte(s) @ 0x00005605ec276c4d 00002200 branch (taken) ``` Instead of: ``` 102268 78352: 1271326 ifetch 3 byte(s) @ 0x00005605ec276c4a 00040110 00000206 math [8byte] %rv0 102269 78353: 1271326 ifetch 2 byte(s) @ 0x00005605ec276c4d 00002200 branch (taken) ``` Expands `run_regdeps_test()` in clients/drcachesim/tests/view_test.cpp adding a 1 word encodings entry (a branch). Fixes suite/tests/api/ir_regdeps.c expected disasm output with this new spacing requirement. Issue #6662

The name `DR_REG_V` of enum values for REGDEPS virtual registers can be misleading, as the `V` can be confused with "Vector (register)", which are unrelated. We rename `DR_REG_V` to `DR_REG_VIRT` for better readability. Issue #6662

The name `DR_REG_V` of enum values for REGDEPS virtual registers can be misleading, as the `V` can be confused with "Vector (registers)", which are unrelated. We rename `DR_REG_V` to `DR_REG_VIRT` for better readability and to avoid confusion. Since this is technically a "breaking compatibility change" w.r.t. DynamoRIO version 10.93, we increase the version to 10.94. Issue #6662

Public traces (i.e., OFFLINE_FILE_TYPE_ARCH_REGDEPS traces) have a new constraint: TRACE_MARKER_TYPE_CPU_ID values must all be -1 as we don't want to expose a CPU schedule. We add this constraint as invariant in the invariant_checker. We also add an invariant for the trace file type, which should not contain any OFFLINE_FILE_TYPE_ARCH_* information except for OFFLINE_FILE_TYPE_ARCH_REGDEPS. Issue #6662

Public traces (i.e., OFFLINE_FILE_TYPE_ARCH_REGDEPS traces) have a new constraint: TRACE_MARKER_TYPE_CPU_ID values must all be -1 as we don't want to expose a CPU schedule. We add this constraint as invariant in the `invariant_checker`. We also add an invariant for the trace file type, which should not contain any OFFLINE_FILE_TYPE_ARCH_* information except for OFFLINE_FILE_TYPE_ARCH_REGDEPS. Issue #6662

Allows the v2p.textproto file to be in the trace directory of public traces to avoid the following error: ``` $ drrun -t drmemtrace -indir ${path_to_public_trace_dir} -tool invariant_checker Invalid header Failed to initialize scheduler: Failed to open <OMITTED>/v2p.textproto ERROR: failed to initialize analyzer ``` Issue #6662

Updates the documentation on Public Google Workload Traces. Gives a description of the new format: the #DR_ISA_REGDEPS synthetic ISA that instructions follow, what TRACE_MARKER_TYPE_* are present or have been modified, what is v2p.textproto. Issue #6662

We previously relied on load and store categories of REGDEPS instructions to indicate the presence of reads and writes that follows. Unfortunately, not all instructions that then perform a read or a write have a corresponding load or a store category, leading to false positive "Too many reads/writes" invariants firing. e.g., no store category for this instruction that performs a write operation: ``` 13123537 10416044: 7446 ifetch 4 byte(s) @ 0x0000aaaaefe73500 00000831 0e0d0d04 load [4byte] %rv11 %rv12 %rv13 -> %rv11 13123537 10416044: 7446 0000000f 13123538 10416044: 7446 read 4 byte(s) @ 0x0000115cffc33ba0 by PC 0x0000aaaaefe73500 13123539 10416044: 7446 write 4 byte(s) @ 0x0000115cffc33ba0 by PC 0x0000aaaaefe73500 ``` We disable these invariants for REGDEPS traces. Issue #6662

Adds instr_get_operation_size() and instr_set_operation_size() public APIs for DR_ISA_REGDEPS instructions, which are currently the only instructions for which operation_size is defined. We set operation_size of DR_ISA_REGDEPS instructions with no operands to OPSZ_0 in the convertion to REGDEPS process, so decoded and converted instructions look the same. Previously a converted instruction might have had an operation_size different than OPSZ_0 even with no register operands, that would then be corrected during encoding and subsequent decoding. This is mostly for convenience when testing. We update a test to also invoke the instr_length() API on DR_ISA_REGDEPS instructions. Issue #6662

Public traces should not have TRACE_MARKER_TYPE_SIGNAL_NUMBER and TRACE_MARKER_TYPE_UNCOMPLETED_INSTRUCTION markers. We add two invariants in invariant_checker to detect the presence of these markers in REGDEPS traces and raise an invariant error if necessary. Updated unit tests and end-to-end tests. Issue #6662

Adds instr_get_operation_size() and instr_set_operation_size() public APIs for DR_ISA_REGDEPS instructions, which are currently the only instructions for which operation_size is defined. We set operation_size of DR_ISA_REGDEPS instructions with no operands to OPSZ_0 in the convertion to REGDEPS process, so decoded and converted instructions look the same. Previously a converted instruction might have had an operation_size different than OPSZ_0 even with no register operands, that would then be corrected during encoding and subsequent decoding. This is mostly for convenience when testing. We update a test to also invoke the instr_length() API on DR_ISA_REGDEPS instructions. Issue #6662

) Public traces should never have TRACE_MARKER_TYPE_SIGNAL_NUMBER markers. Temporarily, we also disallow TRACE_MARKER_TYPE_UNCOMPLETED_INSTRUCTION markers until we can handle their partial encoding. Eventually we want to preserve them (xref: #7155). We add two invariants in invariant_checker to detect the presence of these markers in REGDEPS traces and raise an invariant error if necessary. Updated unit tests and end-to-end tests. Issue #6662 #7155

Allows the v2p.textproto file to be in the trace directory of public traces to avoid the following error: ``` $ drrun -t drmemtrace -indir ${path_to_public_trace_dir} -tool invariant_checker Invalid header Failed to initialize scheduler: Failed to open ${path_public_trace_dir}/v2p.textproto ERROR: failed to initialize analyzer ``` Issue #6662

The textproto format does not output fields with value == 0. In some cases a trace can use virtual address 0x0, so this case needs to be handled (previously we did not allow any missing fields from v2p.textproto). We do so relying on the fact that multiple missing virtual_address fields would try to map 0x0 multiple times, hence resulting in an error. Issue #6662

…oto (#7174) The textproto format does not have fields with value == 0. In some cases a trace can use virtual address 0x0, which would result in a missing virtual_address field in `v2p.textproto`. Previously we did not allow any missing fields in `v2p.textproto`, so this case needs to be handled. We do so relying on the fact that multiple missing virtual_address fields would try to map 0x0 multiple times, resulting in an error. We add the missing virtual_address case to the unit tests. Issue #6662

Update the version number to 11.3. Necessary for compatibility with the new public traces and their v2p.textproto file (xref: #7136 #7174). Issue #6662

edeiana added Type-Feature Component-DrMemtrace labels Feb 20, 2024

edeiana self-assigned this Feb 20, 2024

edeiana mentioned this issue Feb 21, 2024

i#6662 public traces, part 2: encoding_filter #6663

Merged

edeiana mentioned this issue Mar 6, 2024

i#6662 public traces, part 1: synthetic ISA #6691

Merged

edeiana mentioned this issue Apr 18, 2024

i#6662 regdeps ISA: virtual registers #6783

Merged

edeiana mentioned this issue May 8, 2024

i#6662 public traces, part 3: regdeps disasm #6799

Merged

edeiana mentioned this issue May 9, 2024

i#6662 virtual regs: bug fix #6805

Merged

edeiana mentioned this issue May 13, 2024

i#6662 encodings2regdeps: data race bug fix #6809

Merged

edeiana mentioned this issue May 17, 2024

i#6662 public traces, part 4: view tool #6816

Merged

edeiana mentioned this issue May 21, 2024

i#6662 public traces, part 5: func_id_filter_t #6820

Merged

edeiana mentioned this issue Jun 29, 2024

i#6662 public traces, part 6: invariant_checker #6861

Merged

edeiana mentioned this issue Jul 9, 2024

i#6662 func_id_filter: large function IDs #6870

Merged

edeiana mentioned this issue Oct 8, 2024

Invalidate TRACE_MARKER_TYPE_CPU_ID filter #7031

Closed

edeiana mentioned this issue Oct 29, 2024

i#6662: view tool spacing fix #7061

Merged

edeiana mentioned this issue Oct 31, 2024

i#6662: REGDEPS DR_REG_V renaming #7063

Merged

edeiana mentioned this issue Nov 28, 2024

i#6662 regdeps: Invariant checks for filetype and cpuid #7100

Merged

edeiana mentioned this issue Dec 15, 2024

i#6662 public traces: allow v2p.textproto in trace dir #7136

Merged

edeiana mentioned this issue Dec 16, 2024

i#6662 public traces: update doc #7139

Open

edeiana mentioned this issue Dec 16, 2024

i#6662 public traces: disable read/write counts in invariant_checker #7140

Closed

edeiana mentioned this issue Dec 17, 2024

i#6662 public traces: add instr_t operation_size API #7151

Merged

edeiana mentioned this issue Dec 18, 2024

i#6662 public traces: add signal and uncompleted instr invariants #7154

Merged

edeiana mentioned this issue Jan 6, 2025

i#6662 public traces: allow one missing virtual_address in v2p.textproto #7174

Merged

edeiana added a commit that referenced this issue Jan 7, 2025

Version 11.3

73e31a6

Update the version number to 11.3. Necessary for compatibility with the new public traces and their v2p.textproto file (xref: #7136 #7174). Issue #6662

edeiana mentioned this issue Jan 7, 2025

Version 11.3 #7176

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new record filter tool for public release of traces #6662

Add new record filter tool for public release of traces #6662

edeiana commented Feb 20, 2024 •

edited

Loading

AssadHashmi commented Apr 25, 2024

derekbruening commented Apr 25, 2024

AssadHashmi commented Apr 25, 2024

edeiana commented Apr 25, 2024

Add new record filter tool for public release of traces #6662

Add new record filter tool for public release of traces #6662

Comments

edeiana commented Feb 20, 2024 • edited Loading

AssadHashmi commented Apr 25, 2024

derekbruening commented Apr 25, 2024

AssadHashmi commented Apr 25, 2024

edeiana commented Apr 25, 2024

edeiana commented Feb 20, 2024 •

edited

Loading