Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support Gemma 2 #223

Merged
merged 36 commits into from
Jan 10, 2025
Merged

feat: Support Gemma 2 #223

merged 36 commits into from
Jan 10, 2025

Conversation

oreomaker
Copy link
Collaborator

Difference between Gemma and Gemma2

  • Gemma2 adds two feedforward layernorm before and after MLP
  • It changes the hidden_dim to 2304 while still using 2048 and 1024 for query dim and kv dim respectively.

oreomaker and others added 30 commits November 8, 2024 16:03
@oreomaker
dev: qnn multi input inference developing
e428bb5
@oreomaker
Merge branch 'main' into develop-zh
fca8584
@oreomaker
feat: qnn multi chunk prefilling in new frontend
f94937d 
refactor: qnn module setup skip in following chunks
todo: multi input
@yirongjie
Merge branch 'main' into develop-zh
83fefd3
@yirongjie
fix: clearCache in RoPE
5ff02cd
@yirongjie
fix: genarate with Padding
68ddb99
@oreomaker
feat: make chunk_size configurable in demo_phonelm_npu
28811fe
@oreomaker
Merge branch 'main' into develop-zh
d930d14
@oreomaker
chore: andoid submodule diff
9406c02
@oreomaker
fix: qwen npu demos execution type set
caf4f9c
@oreomaker
feat: KVCacheNPU GQA
dfac9d5
@oreomaker
refactor: simplify kvcache npu
2b20472
@oreomaker
fix: kvcache npu seq
fb95bcd
@oreomaker
Merge branch 'main' into develop-zh
865dcd6
@oreomaker
chore: android diff
3ec8d77
@oreomaker
dev: pipeline class init
3b0042d
@oreomaker
refactor: configurable chunk_size in qwen
05bcd77
@oreomaker
chore: remove qnn getBuildId
c3a0c4e
@oreomaker
chore: clean qnn backend include
9a56c4c
@oreomaker
dev: qnn new frontend pipeline(wrap implement)
f627381
@oreomaker
fix: kvcache nrep and stage switching bug in old frontend
bec0a63
@oreomaker
feat: new frontend pipeline
38fff58
@oreomaker
chore: qnn qwen executable change
8d07588
@oreomaker
feat: qnn prefill optimization, only do 1 seq lm_head
693700b
@oreomaker
refactor: main qwen npu token post process
be8258f
@oreomaker
Merge branch 'main' into develop-zh
f8def41
@oreomaker
fix: qnn old frontend modeling backend assign
7e20e8b
@oreomaker
fix: qnn total length and cur length conflict
1d526c1
@oreomaker
feat: configurable chunk size for HeadLinear
6070a4a
@yirongjie
Update CMakeLists.txt
a459c74
@yirongjie yirongjie changed the title Support Gemma 2 featSupport Gemma 2 Jan 10, 2025
@yirongjie yirongjie changed the title featSupport Gemma 2 feat: Support Gemma 2 Jan 10, 2025
@yirongjie yirongjie merged commit 036bbbb into UbiquitousLearning:main Jan 10, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yirongjie yirongjie yirongjie approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@oreomaker @yirongjie