Skip to content

[RISCV] Add branch folding before branch relaxation #134760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

[RISCV] Add branch folding before branch relaxation #134760

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3d2fee4
[RISCV] Add branch folding before branch relaxation
mikhailramalho Mar 27, 2025
47c9e17
Removed note added by update_llc_test_checks.py
mikhailramalho Apr 7, 2025
7075a8a
Merge remote-tracking branch 'origin/main' into riscv-late-opt-branch…
mikhailramalho Apr 7, 2025
515ce5b
Updated tests
mikhailramalho Apr 8, 2025
4e4ed6b
Create method to add the branch folding pass with the option to enabl…
mikhailramalho Apr 10, 2025
6afb625
Update tests
mikhailramalho Apr 10, 2025
aba8697
Whitespace
mikhailramalho Apr 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions llvm/include/llvm/CodeGen/Passes.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,8 @@ namespace llvm {
/// branches.
extern char &BranchFolderPassID;

MachineFunctionPass *createBranchFolderPass(bool EnableTailMerge);

/// BranchRelaxation - This pass replaces branches that need to jump further
/// than is supported by a branch instruction.
extern char &BranchRelaxationPassID;
Expand Down
12 changes: 10 additions & 2 deletions llvm/lib/CodeGen/BranchFolding.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,13 @@ namespace {

/// BranchFolderPass - Wrap branch folder in a machine function pass.
class BranchFolderLegacy : public MachineFunctionPass {
bool EnableTailMerge;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing new pass manager handling (also should handle print/parse of the pass parameter). Also this should be done as a separate step

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the new pass manager part covered by #128858?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That added the initial port to the new pm. This is now changing the pass arguments in the old PM, without the matching new PM change

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I'll work on it


public:
static char ID;

explicit BranchFolderLegacy() : MachineFunctionPass(ID) {}
explicit BranchFolderLegacy(bool EnableTailMerge = true)
: MachineFunctionPass(ID), EnableTailMerge(EnableTailMerge) {}

bool runOnMachineFunction(MachineFunction &MF) override;

Expand Down Expand Up @@ -152,7 +155,8 @@ bool BranchFolderLegacy::runOnMachineFunction(MachineFunction &MF) {
// TailMerge can create jump into if branches that make CFG irreducible for
// HW that requires structurized CFG.
bool EnableTailMerge = !MF.getTarget().requiresStructuredCFG() &&
PassConfig->getEnableTailMerge();
PassConfig->getEnableTailMerge() &&
this->EnableTailMerge;
Comment on lines 157 to +159
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This global state should be moved to only change the pass parameter

MBFIWrapper MBBFreqInfo(
getAnalysis<MachineBlockFrequencyInfoWrapperPass>().getMBFI());
BranchFolder Folder(
Expand Down Expand Up @@ -2080,3 +2084,7 @@ bool BranchFolder::HoistCommonCodeInSuccs(MachineBasicBlock *MBB) {
++NumHoist;
return true;
}

MachineFunctionPass *llvm::createBranchFolderPass(bool EnableTailMerge = true) {
return new BranchFolderLegacy(EnableTailMerge);
}
2 changes: 2 additions & 0 deletions llvm/lib/Target/RISCV/RISCVTargetMachine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
#include "RISCVTargetTransformInfo.h"
#include "TargetInfo/RISCVTargetInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BranchFoldingPass.h"
#include "llvm/CodeGen/GlobalISel/CSEInfo.h"
#include "llvm/CodeGen/GlobalISel/IRTranslator.h"
#include "llvm/CodeGen/GlobalISel/InstructionSelect.h"
Expand Down Expand Up @@ -570,6 +571,7 @@ void RISCVPassConfig::addPreEmitPass() {
addPass(createMachineCopyPropagationPass(true));
if (TM->getOptLevel() >= CodeGenOptLevel::Default)
addPass(createRISCVLateBranchOptPass());
addPass(createBranchFolderPass(false));
addPass(&BranchRelaxationPassID);
addPass(createRISCVMakeCompressibleOptPass());
}
Expand Down
20 changes: 0 additions & 20 deletions llvm/test/CodeGen/RISCV/GlobalISel/rv32zbb.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,11 +357,6 @@ define i64 @ctpop_i64(i64 %a) nounwind {
define i1 @ctpop_i64_ugt_two(i64 %a) nounwind {
; RV32I-LABEL: ctpop_i64_ugt_two:
; RV32I: # %bb.0:
; RV32I-NEXT: j .LBB6_2
; RV32I-NEXT: # %bb.1:
; RV32I-NEXT: sltiu a0, zero, 0
; RV32I-NEXT: ret
; RV32I-NEXT: .LBB6_2:
; RV32I-NEXT: srli a2, a0, 1
; RV32I-NEXT: lui a3, 349525
; RV32I-NEXT: lui a4, 209715
Expand Down Expand Up @@ -404,11 +399,6 @@ define i1 @ctpop_i64_ugt_two(i64 %a) nounwind {
;
; RV32ZBB-LABEL: ctpop_i64_ugt_two:
; RV32ZBB: # %bb.0:
; RV32ZBB-NEXT: j .LBB6_2
; RV32ZBB-NEXT: # %bb.1:
; RV32ZBB-NEXT: sltiu a0, zero, 0
; RV32ZBB-NEXT: ret
; RV32ZBB-NEXT: .LBB6_2:
; RV32ZBB-NEXT: cpop a0, a0
; RV32ZBB-NEXT: cpop a1, a1
; RV32ZBB-NEXT: add a0, a1, a0
Expand All @@ -422,11 +412,6 @@ define i1 @ctpop_i64_ugt_two(i64 %a) nounwind {
define i1 @ctpop_i64_ugt_one(i64 %a) nounwind {
; RV32I-LABEL: ctpop_i64_ugt_one:
; RV32I: # %bb.0:
; RV32I-NEXT: j .LBB7_2
; RV32I-NEXT: # %bb.1:
; RV32I-NEXT: snez a0, zero
; RV32I-NEXT: ret
; RV32I-NEXT: .LBB7_2:
; RV32I-NEXT: srli a2, a0, 1
; RV32I-NEXT: lui a3, 349525
; RV32I-NEXT: lui a4, 209715
Expand Down Expand Up @@ -470,11 +455,6 @@ define i1 @ctpop_i64_ugt_one(i64 %a) nounwind {
;
; RV32ZBB-LABEL: ctpop_i64_ugt_one:
; RV32ZBB: # %bb.0:
; RV32ZBB-NEXT: j .LBB7_2
; RV32ZBB-NEXT: # %bb.1:
; RV32ZBB-NEXT: snez a0, zero
; RV32ZBB-NEXT: ret
; RV32ZBB-NEXT: .LBB7_2:
; RV32ZBB-NEXT: cpop a0, a0
; RV32ZBB-NEXT: cpop a1, a1
; RV32ZBB-NEXT: add a0, a1, a0
Expand Down
6 changes: 5 additions & 1 deletion llvm/test/CodeGen/RISCV/O0-pipeline.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
; CHECK-NEXT: Expand large div/rem
; CHECK-NEXT: Expand fp
; CHECK-NEXT: Expand Atomic instructions
; CHECK-NEXT: RISC-V Zacas ABI fix
; CHECK-NEXT: RISC-V Zacas ABI fix
; CHECK-NEXT: Module Verifier
; CHECK-NEXT: Lower Garbage Collection Instructions
; CHECK-NEXT: Shadow Stack GC Lowering
Expand Down Expand Up @@ -62,6 +62,10 @@
; CHECK-NEXT: Insert fentry calls
; CHECK-NEXT: Insert XRay ops
; CHECK-NEXT: Implement the 'patchable-function' attribute
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
; CHECK-NEXT: Machine Block Frequency Analysis
; CHECK-NEXT: Control Flow Optimizer
; CHECK-NEXT: Branch relaxation pass
; CHECK-NEXT: RISC-V Make Compressible
; CHECK-NEXT: Contiguously Lay Out Funclets
Expand Down
6 changes: 5 additions & 1 deletion llvm/test/CodeGen/RISCV/O3-pipeline.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
; CHECK-NEXT: Expand large div/rem
; CHECK-NEXT: Expand fp
; CHECK-NEXT: Expand Atomic instructions
; CHECK-NEXT: RISC-V Zacas ABI fix
; CHECK-NEXT: RISC-V Zacas ABI fix
; CHECK-NEXT: Dominator Tree Construction
; CHECK-NEXT: Natural Loop Information
; CHECK-NEXT: Canonicalize natural loops
Expand Down Expand Up @@ -195,6 +195,10 @@
; CHECK-NEXT: Implement the 'patchable-function' attribute
; CHECK-NEXT: Machine Copy Propagation Pass
; CHECK-NEXT: RISC-V Late Branch Optimisation Pass
; CHECK-NEXT: MachineDominator Tree Construction
; CHECK-NEXT: Machine Natural Loop Construction
; CHECK-NEXT: Machine Block Frequency Analysis
; CHECK-NEXT: Control Flow Optimizer
; CHECK-NEXT: Branch relaxation pass
; CHECK-NEXT: RISC-V Make Compressible
; CHECK-NEXT: Contiguously Lay Out Funclets
Expand Down
108 changes: 54 additions & 54 deletions llvm/test/CodeGen/RISCV/atomic-signext.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4992,20 +4992,20 @@ define signext i32 @atomicrmw_max_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV32IA: # %bb.0:
; RV32IA-NEXT: andi a2, a1, 1
; RV32IA-NEXT: mv a1, a0
; RV32IA-NEXT: beqz a2, .LBB60_2
; RV32IA-NEXT: # %bb.1: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amomax.w a0, a0, (a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB60_2: # %else
; RV32IA-NEXT: bnez a2, .LBB60_4
; RV32IA-NEXT: # %bb.1: # %else
; RV32IA-NEXT: lw a0, 0(a1)
; RV32IA-NEXT: mv a2, a0
; RV32IA-NEXT: bgtz a0, .LBB60_4
; RV32IA-NEXT: # %bb.3: # %else
; RV32IA-NEXT: bgtz a0, .LBB60_3
; RV32IA-NEXT: # %bb.2: # %else
; RV32IA-NEXT: li a2, 1
; RV32IA-NEXT: .LBB60_4: # %else
; RV32IA-NEXT: .LBB60_3: # %else
; RV32IA-NEXT: sw a2, 0(a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB60_4: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amomax.w a0, a0, (a1)
; RV32IA-NEXT: ret
;
; RV64I-LABEL: atomicrmw_max_i32_monotonic_crossbb:
; RV64I: # %bb.0:
Expand Down Expand Up @@ -5056,19 +5056,19 @@ define signext i32 @atomicrmw_max_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV64IA: # %bb.0:
; RV64IA-NEXT: andi a2, a1, 1
; RV64IA-NEXT: mv a1, a0
; RV64IA-NEXT: beqz a2, .LBB60_2
; RV64IA-NEXT: # %bb.1: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amomax.w a0, a0, (a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB60_2: # %else
; RV64IA-NEXT: bnez a2, .LBB60_4
; RV64IA-NEXT: # %bb.1: # %else
; RV64IA-NEXT: lw a0, 0(a1)
; RV64IA-NEXT: mv a2, a0
; RV64IA-NEXT: bgtz a0, .LBB60_4
; RV64IA-NEXT: # %bb.3: # %else
; RV64IA-NEXT: bgtz a0, .LBB60_3
; RV64IA-NEXT: # %bb.2: # %else
; RV64IA-NEXT: li a2, 1
; RV64IA-NEXT: .LBB60_4: # %else
; RV64IA-NEXT: .LBB60_3: # %else
; RV64IA-NEXT: sw a2, 0(a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB60_4: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amomax.w a0, a0, (a1)
; RV64IA-NEXT: ret
br i1 %c, label %then, label %else

Expand Down Expand Up @@ -5140,20 +5140,20 @@ define signext i32 @atomicrmw_min_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV32IA: # %bb.0:
; RV32IA-NEXT: andi a2, a1, 1
; RV32IA-NEXT: mv a1, a0
; RV32IA-NEXT: beqz a2, .LBB61_2
; RV32IA-NEXT: # %bb.1: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amomin.w a0, a0, (a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB61_2: # %else
; RV32IA-NEXT: bnez a2, .LBB61_4
; RV32IA-NEXT: # %bb.1: # %else
; RV32IA-NEXT: lw a0, 0(a1)
; RV32IA-NEXT: mv a2, a0
; RV32IA-NEXT: blez a0, .LBB61_4
; RV32IA-NEXT: # %bb.3: # %else
; RV32IA-NEXT: blez a0, .LBB61_3
; RV32IA-NEXT: # %bb.2: # %else
; RV32IA-NEXT: li a2, 1
; RV32IA-NEXT: .LBB61_4: # %else
; RV32IA-NEXT: .LBB61_3: # %else
; RV32IA-NEXT: sw a2, 0(a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB61_4: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amomin.w a0, a0, (a1)
; RV32IA-NEXT: ret
;
; RV64I-LABEL: atomicrmw_min_i32_monotonic_crossbb:
; RV64I: # %bb.0:
Expand Down Expand Up @@ -5206,19 +5206,19 @@ define signext i32 @atomicrmw_min_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV64IA: # %bb.0:
; RV64IA-NEXT: andi a2, a1, 1
; RV64IA-NEXT: mv a1, a0
; RV64IA-NEXT: beqz a2, .LBB61_2
; RV64IA-NEXT: # %bb.1: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amomin.w a0, a0, (a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB61_2: # %else
; RV64IA-NEXT: bnez a2, .LBB61_4
; RV64IA-NEXT: # %bb.1: # %else
; RV64IA-NEXT: lw a0, 0(a1)
; RV64IA-NEXT: mv a2, a0
; RV64IA-NEXT: blez a0, .LBB61_4
; RV64IA-NEXT: # %bb.3: # %else
; RV64IA-NEXT: blez a0, .LBB61_3
; RV64IA-NEXT: # %bb.2: # %else
; RV64IA-NEXT: li a2, 1
; RV64IA-NEXT: .LBB61_4: # %else
; RV64IA-NEXT: .LBB61_3: # %else
; RV64IA-NEXT: sw a2, 0(a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB61_4: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amomin.w a0, a0, (a1)
; RV64IA-NEXT: ret
br i1 %c, label %then, label %else

Expand Down Expand Up @@ -5418,21 +5418,21 @@ define signext i32 @atomicrmw_umin_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV32IA: # %bb.0:
; RV32IA-NEXT: andi a2, a1, 1
; RV32IA-NEXT: mv a1, a0
; RV32IA-NEXT: beqz a2, .LBB63_2
; RV32IA-NEXT: # %bb.1: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amominu.w a0, a0, (a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB63_2: # %else
; RV32IA-NEXT: bnez a2, .LBB63_4
; RV32IA-NEXT: # %bb.1: # %else
; RV32IA-NEXT: lw a0, 0(a1)
; RV32IA-NEXT: li a3, 1
; RV32IA-NEXT: mv a2, a0
; RV32IA-NEXT: bltu a0, a3, .LBB63_4
; RV32IA-NEXT: # %bb.3: # %else
; RV32IA-NEXT: bltu a0, a3, .LBB63_3
; RV32IA-NEXT: # %bb.2: # %else
; RV32IA-NEXT: li a2, 1
; RV32IA-NEXT: .LBB63_4: # %else
; RV32IA-NEXT: .LBB63_3: # %else
; RV32IA-NEXT: sw a2, 0(a1)
; RV32IA-NEXT: ret
; RV32IA-NEXT: .LBB63_4: # %then
; RV32IA-NEXT: li a0, 1
; RV32IA-NEXT: amominu.w a0, a0, (a1)
; RV32IA-NEXT: ret
;
; RV64I-LABEL: atomicrmw_umin_i32_monotonic_crossbb:
; RV64I: # %bb.0:
Expand Down Expand Up @@ -5486,20 +5486,20 @@ define signext i32 @atomicrmw_umin_i32_monotonic_crossbb(ptr %a, i1 %c) nounwind
; RV64IA: # %bb.0:
; RV64IA-NEXT: andi a2, a1, 1
; RV64IA-NEXT: mv a1, a0
; RV64IA-NEXT: beqz a2, .LBB63_2
; RV64IA-NEXT: # %bb.1: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amominu.w a0, a0, (a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB63_2: # %else
; RV64IA-NEXT: bnez a2, .LBB63_4
; RV64IA-NEXT: # %bb.1: # %else
; RV64IA-NEXT: lw a0, 0(a1)
; RV64IA-NEXT: li a3, 1
; RV64IA-NEXT: mv a2, a0
; RV64IA-NEXT: bltu a0, a3, .LBB63_4
; RV64IA-NEXT: # %bb.3: # %else
; RV64IA-NEXT: bltu a0, a3, .LBB63_3
; RV64IA-NEXT: # %bb.2: # %else
; RV64IA-NEXT: li a2, 1
; RV64IA-NEXT: .LBB63_4: # %else
; RV64IA-NEXT: .LBB63_3: # %else
; RV64IA-NEXT: sw a2, 0(a1)
; RV64IA-NEXT: ret
; RV64IA-NEXT: .LBB63_4: # %then
; RV64IA-NEXT: li a0, 1
; RV64IA-NEXT: amominu.w a0, a0, (a1)
; RV64IA-NEXT: ret
br i1 %c, label %then, label %else

Expand Down
16 changes: 0 additions & 16 deletions llvm/test/CodeGen/RISCV/bfloat-br-fcmp.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,12 @@ declare bfloat @dummy(bfloat)
define void @br_fcmp_false(bfloat %a, bfloat %b) nounwind {
; RV32IZFBFMIN-LABEL: br_fcmp_false:
; RV32IZFBFMIN: # %bb.0:
; RV32IZFBFMIN-NEXT: j .LBB0_2
; RV32IZFBFMIN-NEXT: # %bb.1: # %if.then
; RV32IZFBFMIN-NEXT: ret
; RV32IZFBFMIN-NEXT: .LBB0_2: # %if.else
; RV32IZFBFMIN-NEXT: addi sp, sp, -16
; RV32IZFBFMIN-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IZFBFMIN-NEXT: call abort
;
; RV64IZFBFMIN-LABEL: br_fcmp_false:
; RV64IZFBFMIN: # %bb.0:
; RV64IZFBFMIN-NEXT: j .LBB0_2
; RV64IZFBFMIN-NEXT: # %bb.1: # %if.then
; RV64IZFBFMIN-NEXT: ret
; RV64IZFBFMIN-NEXT: .LBB0_2: # %if.else
; RV64IZFBFMIN-NEXT: addi sp, sp, -16
; RV64IZFBFMIN-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
; RV64IZFBFMIN-NEXT: call abort
Expand Down Expand Up @@ -581,20 +573,12 @@ if.then:
define void @br_fcmp_true(bfloat %a, bfloat %b) nounwind {
; RV32IZFBFMIN-LABEL: br_fcmp_true:
; RV32IZFBFMIN: # %bb.0:
; RV32IZFBFMIN-NEXT: j .LBB16_2
; RV32IZFBFMIN-NEXT: # %bb.1: # %if.else
; RV32IZFBFMIN-NEXT: ret
; RV32IZFBFMIN-NEXT: .LBB16_2: # %if.then
; RV32IZFBFMIN-NEXT: addi sp, sp, -16
; RV32IZFBFMIN-NEXT: sw ra, 12(sp) # 4-byte Folded Spill
; RV32IZFBFMIN-NEXT: call abort
;
; RV64IZFBFMIN-LABEL: br_fcmp_true:
; RV64IZFBFMIN: # %bb.0:
; RV64IZFBFMIN-NEXT: j .LBB16_2
; RV64IZFBFMIN-NEXT: # %bb.1: # %if.else
; RV64IZFBFMIN-NEXT: ret
; RV64IZFBFMIN-NEXT: .LBB16_2: # %if.then
; RV64IZFBFMIN-NEXT: addi sp, sp, -16
; RV64IZFBFMIN-NEXT: sd ra, 8(sp) # 8-byte Folded Spill
; RV64IZFBFMIN-NEXT: call abort
Expand Down
Loading