Skip to content

Commit

Permalink
bugfix: fix broken cpp integration caused by #553 (#570)
Browse files Browse the repository at this point in the history
Hi, when I try to rebase my current work, just found cpp integration
(benchmark and test) failed to build, this is introduced by the feature
#553.
Tests have been passed.

Co-authored-by: tsu-bin <tsubin@gmail.com>
  • Loading branch information
tsu-bin and tsu-bin authored Oct 30, 2024
1 parent 3e104bc commit e46d9a7
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions src/flashinfer_ops.cuh
Original file line number Diff line number Diff line change
Expand Up @@ -423,6 +423,7 @@ cudaError_t BatchPrefillWithPagedKVCacheWrapper(
ParamsT params(q, paged_kv, /*custom_mask=*/nullptr, qo_indptr,
/*qk_indptr=*/nullptr, q_offset, o, lse,
/*alibi_slopes=*/nullptr, num_qo_heads,
/*q_stride_n*/num_qo_heads*HEAD_DIM, /*q_stride_h*/HEAD_DIM,
/*window_left=*/-1, /*logits_soft_cap=*/0.f, sm_scale, rope_scale,
rope_theta);
params.request_indices = handler->GetRequestIndices<IdType>();
Expand Down Expand Up @@ -525,6 +526,7 @@ cudaError_t BatchDecodeWithPagedKVCacheWrapper(
/*use_custom_mask=*/false, /*use_sliding_window=*/true,
/*use_logits_soft_cap=*/false, /*use_alibi=*/false)>;
ParamsT params(q, q_offset, paged_kv, o, lse, /*alibi_slopes=*/nullptr, num_qo_heads,
/*q_stride_n*/num_qo_heads*HEAD_DIM, /*q_stride_h*/HEAD_DIM,
/*window_left=*/-1, /*logits_soft_cap=*/0.f, sm_scale, rope_scale,
rope_theta);
params.request_indices = handler->GetRequestIndices<IdType>();
Expand Down

0 comments on commit e46d9a7

Please sign in to comment.