Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gelu vk op tanh fix #5008

Merged
merged 1 commit into from
Sep 6, 2023
Merged

Conversation

FhqTreap
Copy link
Contributor

@FhqTreap FhqTreap commented Sep 5, 2023

try to fix #5001 (comment)

@codecov-commenter
Copy link

codecov-commenter commented Sep 5, 2023

Codecov Report

Merging #5008 (b6c782d) into master (cdcb0d2) will increase coverage by 5.21%.
Report is 2 commits behind head on master.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #5008       +/-   ##
===========================================
+ Coverage   89.50%   94.72%    +5.21%     
===========================================
  Files         298      768      +470     
  Lines       88989   228565   +139576     
===========================================
+ Hits        79652   216505   +136853     
- Misses       9337    12060     +2723     

see 646 files with indirect coverage changes

@nihui nihui merged commit e14cc27 into Tencent:master Sep 6, 2023
@nihui
Copy link
Member

nihui commented Sep 6, 2023

Thanks for your contribution !

@nihui
Copy link
Member

nihui commented Sep 18, 2023

all float constant needs to be wrapped in afp() in gelu compute shader

v = 0.5f * v * (1.0f + tanh(0.79788452f * (v + 0.044715f * v * v * v)));
v = afp(0.5f) * v * (afp(1.0f) + tanh(afp(0.79788452f) * (v + afp(0.044715f) * v * v * v)));

This was referenced Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants