Skip to content

Commit

Permalink
fix(UI): Updated ui.py. Frees up the CPU to not be bottlenecked.
Browse files Browse the repository at this point in the history
Updated ui.py to include a small sleep timer while building the stream deltas.  This recursive function fires off so quickly to eats up too much of the CPU.  This small sleep frees up the CPU to not be bottlenecked.  This value can go lower/shorter.  But 0.02 or 0.025 seems to work well. (#1589)

Co-authored-by: root <root@wesgitlabdemo.icl.gtri.org>
  • Loading branch information
icsy7867 and root authored Feb 16, 2024
1 parent 6bbec79 commit 24fb80c
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions private_gpt/ui/ui.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""This file should be imported only and only if you want to run the UI locally."""
import itertools
import logging
import time
from collections.abc import Iterable
from pathlib import Path
from typing import Any
Expand Down Expand Up @@ -87,6 +88,7 @@ def yield_deltas(completion_gen: CompletionGen) -> Iterable[str]:
elif isinstance(delta, ChatResponse):
full_response += delta.delta or ""
yield full_response
time.sleep(0.02)

if completion_gen.sources:
full_response += SOURCES_SEPARATOR
Expand Down

0 comments on commit 24fb80c

Please sign in to comment.