Skip to content

[VPlan] Implement VPInterleaveRecipe::computeCost. #106067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fhahn merged 4 commits into from
Oct 15, 2024
Merged

[VPlan] Implement VPInterleaveRecipe::computeCost. #106067

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7b70fb9
[VPlan] Implement VPInterleaveRecipe::computeCost.
fhahn Aug 26, 2024
f947915
Merge branch 'main' into vpinterleave-cost
fhahn Sep 3, 2024
7625952
Merge remote-tracking branch 'origin/main' into vpinterleave-cost
fhahn Oct 9, 2024
c1c20e7
Merge branch 'main' into vpinterleave-cost
fhahn Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2941,7 +2941,33 @@ void VPInterleaveRecipe::print(raw_ostream &O, const Twine &Indent,

InstructionCost VPInterleaveRecipe::computeCost(ElementCount VF,
VPCostContext &Ctx) const {
return Ctx.getLegacyCost(IG->getInsertPos(), VF);
Instruction *I = getInsertPos();
Type *ValTy = Ctx.Types.inferScalarType(
getNumDefinedValues() > 0 ? getVPValue(0) : getStoredValues()[0]);
auto *VectorTy = cast<VectorType>(ToVectorTy(ValTy, VF));
unsigned AS = getLoadStoreAddressSpace(I);
enum TTI::TargetCostKind CostKind = TTI::TCK_RecipThroughput;

unsigned InterleaveFactor = IG->getFactor();
auto *WideVecTy = VectorType::get(ValTy, VF * InterleaveFactor);

// Holds the indices of existing members in the interleaved group.
SmallVector<unsigned, 4> Indices;
for (unsigned IF = 0; IF < InterleaveFactor; IF++)
if (IG->getMember(IF))
Indices.push_back(IF);

// Calculate the cost of the whole interleaved group.
InstructionCost Cost = Ctx.TTI.getInterleavedMemoryOpCost(
I->getOpcode(), WideVecTy, IG->getFactor(), Indices, IG->getAlign(), AS,
CostKind, getMask(), NeedsMaskForGaps);

if (!IG->isReverse())
return Cost;

return Cost + IG->getNumMembers() *
Copy link
Member

@rengolin rengolin Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks a bit hard-coded. It would be better to get the attribute from IG and pass it to getShuffleCost, but this can be done in a future PR.

Copy link
Contributor Author

@fhahn fhahn Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC this extra computation could/should also be handled by getInterleavedMemoryOpCost above?

Ctx.TTI.getShuffleCost(TargetTransformInfo::SK_Reverse,
VectorTy, std::nullopt, CostKind, 0);
}

void VPCanonicalIVPHIRecipe::execute(VPTransformState &State) {
Expand Down