Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zhwang/code llama debug #23

Open
wants to merge 178 commits into
base: corvo
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
178 commits
Select commit Hold shift + click to select a range
c7330c6
commit
sfc-gh-zhwang Oct 6, 2023
d5fc856
commit
sfc-gh-zhwang Oct 6, 2023
5baf8d6
commit
sfc-gh-zhwang Oct 6, 2023
138a883
commit
sfc-gh-zhwang Oct 6, 2023
88398e2
commit
sfc-gh-zhwang Oct 6, 2023
f1de67f
commit
sfc-gh-zhwang Oct 6, 2023
98ac852
commit
sfc-gh-zhwang Oct 6, 2023
3d96715
commit
sfc-gh-zhwang Oct 6, 2023
811b19f
commit
sfc-gh-zhwang Oct 6, 2023
120f46c
commit
sfc-gh-zhwang Oct 6, 2023
d850e49
commit
sfc-gh-zhwang Oct 6, 2023
751b7fc
commit
sfc-gh-zhwang Oct 6, 2023
d2946ee
commit
sfc-gh-zhwang Oct 6, 2023
d0cb0a5
commit
sfc-gh-zhwang Oct 6, 2023
9297c43
commit
sfc-gh-zhwang Oct 6, 2023
f92c384
commit
sfc-gh-zhwang Oct 6, 2023
5330e28
commit
sfc-gh-zhwang Oct 6, 2023
e0b452a
commit
sfc-gh-zhwang Oct 6, 2023
4785e3d
commit
sfc-gh-zhwang Oct 6, 2023
5e5c611
commit
sfc-gh-zhwang Oct 6, 2023
8cda3ab
commit
sfc-gh-zhwang Oct 6, 2023
d05cc05
commit
sfc-gh-zhwang Oct 6, 2023
fb89e18
commit
sfc-gh-zhwang Oct 6, 2023
5429b2c
commit
sfc-gh-zhwang Oct 6, 2023
fb72ca1
commit
sfc-gh-zhwang Oct 6, 2023
aa9f389
commit
sfc-gh-zhwang Oct 6, 2023
ef1c1e3
commit
sfc-gh-zhwang Oct 6, 2023
7aa5709
commit
sfc-gh-zhwang Oct 6, 2023
7918ff1
commit
sfc-gh-zhwang Oct 6, 2023
2c835da
commit
sfc-gh-zhwang Oct 6, 2023
7f32af3
commit
sfc-gh-zhwang Oct 6, 2023
6a2c0ef
commit
sfc-gh-zhwang Oct 6, 2023
bc3e0cd
commit
sfc-gh-zhwang Oct 6, 2023
4fc436a
commit
sfc-gh-zhwang Oct 6, 2023
ff1342b
commit
sfc-gh-zhwang Oct 6, 2023
37ece5d
commit
sfc-gh-zhwang Oct 6, 2023
5382b47
commit
sfc-gh-zhwang Oct 6, 2023
7bbc5c3
commit
sfc-gh-zhwang Oct 6, 2023
d81dbc6
commit
sfc-gh-zhwang Oct 6, 2023
fcea4c9
commit
sfc-gh-zhwang Oct 6, 2023
c6aa32a
commit
sfc-gh-zhwang Oct 6, 2023
4467d8f
commit
sfc-gh-zhwang Oct 6, 2023
06fea96
commit
sfc-gh-zhwang Oct 6, 2023
0935a9b
commit
sfc-gh-zhwang Oct 6, 2023
fcc7de4
commit
sfc-gh-zhwang Oct 6, 2023
b7aa0aa
commit
sfc-gh-zhwang Oct 7, 2023
a38721d
commit
sfc-gh-zhwang Oct 7, 2023
d809108
commit
sfc-gh-zhwang Oct 7, 2023
8ca884c
commit
sfc-gh-zhwang Oct 7, 2023
958c727
commit
sfc-gh-zhwang Oct 7, 2023
4d10783
commit
sfc-gh-zhwang Oct 7, 2023
37267d5
commit
sfc-gh-zhwang Oct 7, 2023
e16111b
commit
sfc-gh-zhwang Oct 7, 2023
ca4e514
commit
sfc-gh-zhwang Oct 7, 2023
a03b8f6
commit
sfc-gh-zhwang Oct 7, 2023
606e8e2
commit
sfc-gh-zhwang Oct 7, 2023
7b31bd6
commit
sfc-gh-zhwang Oct 7, 2023
5e64f57
commit
sfc-gh-zhwang Oct 7, 2023
f8d1081
commit
sfc-gh-zhwang Oct 7, 2023
78d2e2c
commit
sfc-gh-zhwang Oct 7, 2023
b550406
commit
sfc-gh-zhwang Oct 7, 2023
0cbfc56
commit
sfc-gh-zhwang Oct 7, 2023
6800070
commit
sfc-gh-zhwang Oct 7, 2023
1407bbb
commit
sfc-gh-zhwang Oct 7, 2023
51081e0
commit
sfc-gh-zhwang Oct 7, 2023
0189fe0
commit
sfc-gh-zhwang Oct 7, 2023
507efd6
commit
sfc-gh-zhwang Oct 7, 2023
68acfc7
commit
sfc-gh-zhwang Oct 7, 2023
af9f946
commit
sfc-gh-zhwang Oct 7, 2023
4914ae8
commit
sfc-gh-zhwang Oct 7, 2023
b6e316c
commit
sfc-gh-zhwang Oct 7, 2023
3624a29
commit
sfc-gh-zhwang Oct 7, 2023
1960ddc
commit
sfc-gh-zhwang Oct 7, 2023
0354934
commit
sfc-gh-zhwang Oct 7, 2023
0f8342b
commit
sfc-gh-zhwang Oct 7, 2023
b64e55f
commit
sfc-gh-zhwang Oct 7, 2023
b9af956
commit
sfc-gh-zhwang Oct 7, 2023
bbfe4c0
commit
sfc-gh-zhwang Oct 7, 2023
247b86c
commit
sfc-gh-zhwang Oct 7, 2023
f53f732
commit
sfc-gh-zhwang Oct 7, 2023
59f7e45
commit
sfc-gh-zhwang Oct 7, 2023
e51c6e4
commit
sfc-gh-zhwang Oct 7, 2023
20f3cd7
commit
sfc-gh-zhwang Oct 7, 2023
e1f02f5
commit
sfc-gh-zhwang Oct 7, 2023
ac281b6
commit
sfc-gh-zhwang Oct 7, 2023
132bcb7
commit
sfc-gh-zhwang Oct 7, 2023
2f8d384
commit
sfc-gh-zhwang Oct 7, 2023
0a90c30
commit
sfc-gh-zhwang Oct 7, 2023
2288d21
commit
sfc-gh-zhwang Oct 7, 2023
02f0fac
commit
sfc-gh-zhwang Oct 7, 2023
1ea3502
commit
sfc-gh-zhwang Oct 7, 2023
4b0ad70
commit
sfc-gh-zhwang Oct 7, 2023
84e236c
commit
sfc-gh-zhwang Oct 7, 2023
3d3e290
commit
sfc-gh-zhwang Oct 7, 2023
196b145
commit
sfc-gh-zhwang Oct 7, 2023
ba68c74
commit
sfc-gh-zhwang Oct 7, 2023
f3cefd8
commit
sfc-gh-zhwang Oct 7, 2023
396f981
commit
sfc-gh-zhwang Oct 7, 2023
cad0392
commit
sfc-gh-zhwang Oct 7, 2023
20d16a2
commit
sfc-gh-zhwang Oct 7, 2023
513f227
commit
sfc-gh-zhwang Oct 7, 2023
15a2323
commit
sfc-gh-zhwang Oct 7, 2023
f4c9711
commit
sfc-gh-zhwang Oct 7, 2023
44aac65
commit
sfc-gh-zhwang Oct 7, 2023
09fca7d
commit
sfc-gh-zhwang Oct 7, 2023
e631fd7
commit
sfc-gh-zhwang Oct 7, 2023
e5f14d9
commit
sfc-gh-zhwang Oct 7, 2023
a38772c
commit
sfc-gh-zhwang Oct 7, 2023
b85faee
commit
sfc-gh-zhwang Oct 7, 2023
c3601b7
commit
sfc-gh-zhwang Oct 7, 2023
ac51023
commit
sfc-gh-zhwang Oct 7, 2023
cc25af6
commit
sfc-gh-zhwang Oct 7, 2023
f18b378
commit
sfc-gh-zhwang Oct 7, 2023
ccbcc08
commit
sfc-gh-zhwang Oct 7, 2023
db2295d
commit
sfc-gh-zhwang Oct 7, 2023
ba0d4aa
commit
sfc-gh-zhwang Oct 7, 2023
ea56edf
commit
sfc-gh-zhwang Oct 7, 2023
52da87b
commit
sfc-gh-zhwang Oct 7, 2023
1d248c5
commit
sfc-gh-zhwang Oct 7, 2023
56275c3
commit
sfc-gh-zhwang Oct 7, 2023
b6f984e
commit
sfc-gh-zhwang Oct 7, 2023
5b3bd2f
commit
sfc-gh-zhwang Oct 7, 2023
4271826
commit
sfc-gh-zhwang Oct 7, 2023
b3343d1
commit
sfc-gh-zhwang Oct 7, 2023
d64fb07
commit
sfc-gh-zhwang Oct 7, 2023
8c591a7
commit
sfc-gh-zhwang Oct 7, 2023
dd1ad8e
commit
sfc-gh-zhwang Oct 7, 2023
16ed12b
commit
sfc-gh-zhwang Oct 7, 2023
d73ba7c
commit
sfc-gh-zhwang Oct 7, 2023
24dd780
commit
sfc-gh-zhwang Oct 7, 2023
5123c23
commit
sfc-gh-zhwang Oct 7, 2023
c5b5716
commit
sfc-gh-zhwang Oct 7, 2023
6f9b25f
commit
sfc-gh-zhwang Oct 7, 2023
078a316
commit
sfc-gh-zhwang Oct 7, 2023
2477208
commit
sfc-gh-zhwang Oct 7, 2023
87fdddd
commit
sfc-gh-zhwang Oct 7, 2023
b90b511
commit
sfc-gh-zhwang Oct 7, 2023
1a84d8d
commit
sfc-gh-zhwang Oct 7, 2023
3d3296d
commit
sfc-gh-zhwang Oct 7, 2023
3735c3d
commit
sfc-gh-zhwang Oct 8, 2023
fe64ade
commit
sfc-gh-zhwang Oct 8, 2023
f454ae0
commit
sfc-gh-zhwang Oct 8, 2023
40dffed
commit
sfc-gh-zhwang Oct 8, 2023
a9c927f
commit
sfc-gh-zhwang Oct 8, 2023
4c4baeb
commit
sfc-gh-zhwang Oct 8, 2023
ad61865
commit
sfc-gh-zhwang Oct 8, 2023
9fee4c1
commit
sfc-gh-zhwang Oct 8, 2023
708cab6
commit
sfc-gh-zhwang Oct 8, 2023
6ac423f
commit
sfc-gh-zhwang Oct 8, 2023
354f194
commit
sfc-gh-zhwang Oct 8, 2023
1ac63ec
commit
sfc-gh-zhwang Oct 8, 2023
245fed4
commit
sfc-gh-zhwang Oct 8, 2023
199c598
commit
sfc-gh-zhwang Oct 8, 2023
a181428
commit
sfc-gh-zhwang Oct 8, 2023
74a3406
commit
sfc-gh-zhwang Oct 8, 2023
4c69b5c
commit
sfc-gh-zhwang Oct 8, 2023
5f385c8
commit
sfc-gh-zhwang Oct 8, 2023
60ba998
commit
sfc-gh-zhwang Oct 8, 2023
559a41e
commit
sfc-gh-zhwang Oct 8, 2023
56a27f5
commit
sfc-gh-zhwang Oct 8, 2023
5838707
commit
sfc-gh-zhwang Oct 8, 2023
19fe4c0
commit
sfc-gh-zhwang Oct 8, 2023
d3a4ebc
commit
sfc-gh-zhwang Oct 8, 2023
0311a72
commit
sfc-gh-zhwang Oct 8, 2023
a8ae9d3
commit
sfc-gh-zhwang Oct 8, 2023
8b625cf
commit
sfc-gh-zhwang Oct 8, 2023
df90770
commit
sfc-gh-zhwang Oct 8, 2023
ffeeedb
commit
sfc-gh-zhwang Oct 8, 2023
7e28798
commit
sfc-gh-zhwang Oct 8, 2023
67c0b91
commit
sfc-gh-zhwang Oct 8, 2023
c6cd12c
commit
sfc-gh-zhwang Oct 8, 2023
7b836e9
commit
sfc-gh-zhwang Oct 8, 2023
53bb8b3
commit
sfc-gh-zhwang Oct 8, 2023
1078dbb
commit
sfc-gh-zhwang Oct 8, 2023
80cc0bb
commit
sfc-gh-zhwang Oct 8, 2023
aebe644
commit
sfc-gh-zhwang Oct 8, 2023
d25b683
commit
sfc-gh-zhwang Oct 8, 2023
aab7887
commit
sfc-gh-zhwang Oct 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
commit
  • Loading branch information
sfc-gh-zhwang committed Oct 8, 2023
commit 67c0b915ee6e4c7a3554b5d4c56db241b854486f
Original file line number Diff line number Diff line change
@@ -29,7 +29,7 @@
size_t smem_sz = mmha::smem_size_in_bytes<T>(params, THDS_PER_VALUE, THDS_PER_BLOCK); \
printf("smem_sz: %d\n", smem_sz); \
dim3 grid(params.num_heads, params.batch_size); \
cudaFuncSetAttribute(mmha::masked_groupedquery_attention_kernel, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_sz); \
cudaFuncSetAttribute(mmha::masked_groupedquery_attention_kernel<T, Dh, Dh_MAX, THDS_PER_KEY, THDS_PER_VALUE, THDS_PER_BLOCK, HAS_BEAMS>, cudaFuncAttributeMaxDynamicSharedMemorySize, smem_sz); \
mmha::masked_groupedquery_attention_kernel<T, \
Dh, \
Dh_MAX, \