Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Cb streaming #641

Closed
wants to merge 32 commits into from
Closed

Add Cb streaming #641

wants to merge 32 commits into from

Conversation

Wovchena
Copy link
Collaborator

Merge after #612

Wovchena and others added 30 commits July 10, 2024 19:02
@Wovchena
Add ContinuousBatchingPipeline constructor similar to LLMPipeline
d2ccfe0 
That allows LLMPipeline to create ContinuousBatchingPipeline as a backend. There's also a constructor accepting ireq, which can be used if the model was already transformed appropriately for ContinuousBatchingPipeline. But it feels it's going to be misleading and it simpler just to throw if such constructor is called with ContinuousBatchingPipeline backend.
@Wovchena
Use CB as backend
ab0f43c
@Wovchena
Update bindings
05cf5d3
@Wovchena
comma
96fcf77
@Wovchena
Merge branch 'add-ContinuousBatchingPipeline-constructor-similar-to-L…
e5c2631 
…LMPipeline' into use-CB-as-backend
@Wovchena
Merge branch 'master' into add-ContinuousBatchingPipeline-constructor…
967da3a 
…-similar-to-LLMPipeline
@Wovchena
Merge branch 'add-ContinuousBatchingPipeline-constructor-similar-to-L…
c13d623 
…LMPipeline' into use-CB-as-backend
@Wovchena
pass
e278a14
@Wovchena
conflict
47fa22c
@Wovchena
conflict
2094ba6
@Wovchena
clean
78361a9
@Wovchena
verify status
691fefc
@Wovchena
conflict
1825328
@Wovchena
args
6a3275e
@Wovchena
test
a5f2cd6
@Wovchena
tests
771fc29
@Wovchena
tests
5c615bf
@Wovchena
test
3afc16d
@Wovchena
remove caching
67f4717
@popovaan
Clear beam search info.
d26723f
@Wovchena
Merge remote-tracking branch 'popovaan/clear_beam_info' into use-CB-a…
c28a023 
…s-backend
@Wovchena
-am cache
d223d68
@Wovchena
updte
238ea8b
@Wovchena
Revert "Merge remote-tracking branch 'popovaan/clear_beam_info' into …
5a4c878 
…use-CB-as-backend"

This reverts commit c28a023, reversing
changes made to 67f4717.
@Wovchena
revert spelling
cf35f19
@Wovchena
relax abs_tol
12061af
@Wovchena
Merge branch 'releases/2024/3' into use-CB-as-backend
cad13dc
@Wovchena
Merge branch 'releases/2024/3' into use-CB-as-backend
0ffdd6f
@Wovchena
lru_cache
6d7a468
@Wovchena
Merge branch 'releases/2024/3' into cb-streaming
036111c
Wovchena added a commit to Wovchena/openvino.genai-public that referenced this pull request Jul 18, 2024
@Wovchena
Add CB naive chat
25dad4e 
Merge after openvinotoolkit#641
Wovchena added a commit to Wovchena/openvino.genai-public that referenced this pull request Jul 18, 2024
@Wovchena
Add CB naive chat
532a804 
Merge after openvinotoolkit#641
Wovchena added a commit to Wovchena/openvino.genai-public that referenced this pull request Jul 18, 2024
@Wovchena
Add CB naive chat
c70b909 
Merge after openvinotoolkit#641
@Wovchena Wovchena mentioned this pull request Jul 18, 2024
Merged
@Wovchena Wovchena closed this Jul 23, 2024
@Wovchena
Copy link
Collaborator Author

Included in #644

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iefode iefode Awaiting requested review from iefode

@pavel-esir pavel-esir Awaiting requested review from pavel-esir

@mzegla mzegla Awaiting requested review from mzegla

@dkalinowski dkalinowski Awaiting requested review from dkalinowski

@popovaan popovaan Awaiting requested review from popovaan

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Wovchena @popovaan