New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TVMWrapper] Add stream argument in BeginForward #164

Merged

yzh119 merged 1 commit into flashinfer-ai:main from MasterJH5574:03-07-cuda-stream-begin-forward

Mar 8, 2024

Collaborator

MasterJH5574 commented Mar 8, 2024

This PR supports passing CUDA stream in BeginForward for auxiliary data copy.


          [TVMWrapper] Add stream argument in BeginForward

dcc275f

This PR supports passing CUDA stream in BeginForward for auxiliary
data copy.

yzh119 approved these changes

View reviewed changes

yzh119 merged commit fabfcb5 into flashinfer-ai:main

yzh119 mentioned this pull request

chore(main): release 0.0.3 #120

Merged

yzh119 added a commit that referenced this pull request


          chore(main): release 0.0.3 (#120)

0d04571

🤖 I have created a release *beep* *boop*
---


##
[0.0.3](v0.0.3...v0.1.0)
(2024-03-08)


### Features

* adding `sm_scale` field for all attention APIs
([#145](#145))
([85d4018](85d4018))
* enable `head_dim=256` for attention kernels
([#132](#132))
([0372acc](0372acc))
* pytorch api of fp8 kv-cache
([#156](#156))
([66ee066](66ee066))
* support ALiBi
([#146](#146))
([383518b](383518b))

### Misc
* add stream argument in BeginForwardFunction of TVMWrapper
([#164](#164))
([fabfcb5](https://github.com/flashinfer-ai/flashinfer/tree/fabfcb5751dcc003137a5a7d2d5514f3afe2e302))

### Bug Fixes

* bugfix to pr 135
([#136](#136))
([3d55c71](3d55c71))
* fix bugs introduced in
[#132](#132)
([#135](#135))
([9b7b0b9](9b7b0b9))
* fix FindThrust.cmake
([#161](#161))
([30fa584](30fa584))


### Performance Improvements

* multiple q by sm_scale in decode kernels
([#144](#144))
([660c559](660c559))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: yzh119 <expye@outlook.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet