fix(UI): Updated ui.py. Frees up the CPU to not be bottlenecked.

Updated ui.py to include a small sleep timer while building the stream deltas.  This recursive function fires off so quickly to eats up too much of the CPU.  This small sleep frees up the CPU to not be bottlenecked.  This value can go lower/shorter.  But 0.02 or 0.025 seems to work well. (#1589)

Co-authored-by: root <root@wesgitlabdemo.icl.gtri.org>
This commit is contained in:
icsy7867 2024-02-16 06:52:14 -05:00 committed by GitHub
parent 6bbec79583
commit 24fb80ca38
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 0 deletions

View File

@ -1,6 +1,7 @@
"""This file should be imported only and only if you want to run the UI locally."""
import itertools
import logging
import time
from collections.abc import Iterable
from pathlib import Path
from typing import Any
@ -87,6 +88,7 @@ class PrivateGptUi:
elif isinstance(delta, ChatResponse):
full_response += delta.delta or ""
yield full_response
time.sleep(0.02)
if completion_gen.sources:
full_response += SOURCES_SEPARATOR