Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

transpose 2d v1 #434

Closed
wants to merge 1 commit into from
Closed

transpose 2d v1 #434

wants to merge 1 commit into from

Conversation

zhiwei-fang
Copy link
Contributor

This is a special version for the current transpose operator. The current transpose operator will handle a general N-dimension transpose, while this PR implement a 2D version to speed up 2D transpose.
Thread coarsening and (static) shared memory have been used.
Benchmark result:

Running command: python /home/zhiwei/hidet/.github/scripts/bench/bench_op.py transpose2d --params 3000x4000 --dtype float16
type        id  name         runfile        param_id  param_name      dtype_id  dtype_name    hardware_config      latency
--------  ----  -----------  -----------  ----------  ------------  ----------  ------------  -----------------  ---------
operator     3  transpose2d  bench_op.py           7  3000x4000              1  float16                           0.181748

@zhiwei-fang zhiwei-fang requested review from xinli-git and hjjq March 1, 2024 18:51
@zhiwei-fang zhiwei-fang closed this Mar 1, 2024
@zhiwei-fang zhiwei-fang deleted the transpose branch March 1, 2024 22:54
vadiklyutiy added a commit that referenced this pull request Dec 19, 2024
@vadiklyutiy
[PERF] Specialize pow(x,2) as x*x. llama-7B (#434)
49780eb 
Right now `pow` with const exp argument is implemented simply. We
convert const to const tensor and run elementwise `pow` of 2 tensors. It
is simply but not always efficient.

llama2 (RMSNorm part) has `x*x` that implemented as `tensor.pow(2)`. 

Convert `pow(x,2)` to `x*x`.

Improvement on llama2-7B is around **0.237%**
vadiklyutiy added a commit that referenced this pull request Dec 20, 2024
@vadiklyutiy
[PERF] Specialize pow(x,2) as x*x. llama-7B (#434)
f421a43 
Right now `pow` with const exp argument is implemented simply. We
convert const to const tensor and run elementwise `pow` of 2 tensors. It
is simply but not always efficient.

llama2 (RMSNorm part) has `x*x` that implemented as `tensor.pow(2)`. 

Convert `pow(x,2)` to `x*x`.

Improvement on llama2-7B is around **0.237%**
vadiklyutiy added a commit that referenced this pull request Dec 26, 2024
@vadiklyutiy
[PERF] Specialize pow(x,2) as x*x. llama-7B (#434)
595405d 
Right now `pow` with const exp argument is implemented simply. We
convert const to const tensor and run elementwise `pow` of 2 tensors. It
is simply but not always efficient.

llama2 (RMSNorm part) has `x*x` that implemented as `tensor.pow(2)`. 

Convert `pow(x,2)` to `x*x`.

Improvement on llama2-7B is around **0.237%**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xinli-git xinli-git Awaiting requested review from xinli-git

@hjjq hjjq Awaiting requested review from hjjq

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zhiwei-fang