You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I use Baichuan, I found that every Sequence has its own physical block even the input data has common prefix(The length of the prefix is 16 which is equal to the block_size). Is it expected? According to
, I think they will use shared physical block.
This is my "block_tables", every sequence has a common prefix but different physical block id
(Pdb) seq_group_metadata_list[0].block_tables {0: [7303, 7302]}
The text was updated successfully, but these errors were encountered:
YijiaZhao
changed the title
Every Sequence has its own physical block when they have common prefix.
Every Sequence has different physical block when they have common prefix.
Aug 6, 2023
YijiaZhao
changed the title
Every Sequence has different physical block when they have common prefix.
Sequences in a batch have different physical blocks when they have common prefix.
Aug 6, 2023
When I use Baichuan, I found that every Sequence has its own physical block even the input data has common prefix(The length of the prefix is 16 which is equal to the block_size). Is it expected? According to
, I think they will use shared physical block.
This is my "block_tables", every sequence has a common prefix but different physical block id
(Pdb) seq_group_metadata_list[0].block_tables {0: [7303, 7302]}
(Pdb) seq_group_metadata_list[1].block_tables {1: [7301, 7300]}
(Pdb) seq_group_metadata_list[2].block_tables {2: [7299, 7298]}
(Pdb) seq_group_metadata_list[3].block_tables {3: [7297, 7296]}
(Pdb) seq_group_metadata_list[4].block_tables {4: [7295, 7294]}
The text was updated successfully, but these errors were encountered: