Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix greedy regex #47

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

davemurphysf
Copy link

@davemurphysf davemurphysf commented Mar 18, 2024

When processing an event right now, if a SSE event starts with a space or is just made up of space characters, the regex filtering the data: field will remove all of the spaces before it finds a non-space character.

I read the spec and says the space after the colon should be ignored, but all other characters until the newline.

Example streaming response from ChatGPT showing when this is an issue: when asking What is 10-3, it returns:

handleLLMNewToken { token: '' }
handleLLMNewToken { token: '10' }
handleLLMNewToken { token: ' -' }
handleLLMNewToken { token: ' ' }
handleLLMNewToken { token: '3' }
handleLLMNewToken { token: ' equals' }
handleLLMNewToken { token: ' ' }
handleLLMNewToken { token: '7' }
handleLLMNewToken { token: '.' }
handleLLMNewToken { token: '' }

Which will currently produce:

"10"
"-"  // <- missing space
// <-missing event with just a space
"3"
"equals"  // <- missing space
// <-missing event with just a space
"7"
"."

@davemurphysf davemurphysf marked this pull request as ready for review March 18, 2024 16:51
@EmilJunker
Copy link
Contributor

@davemurphysf The way you changed the regex, it will now only match the prefix "data: " with a space after the colon. When there is no space after the colon, e.g. "data:Hello", it won't parse the message correctly anymore. That is a big problem.

I think you want to use this regex: /data:?\s?/

@EmilJunker
Copy link
Contributor

@davemurphysf Also, beware that line 208 currently removes all trailing whitespace from the line, so if the data is just a space or ends with a space, you would still handle it incorrectly:

line = parts[i].trim();

For example, consider a token like " equals " with both a leading and trailing space.
It would show up in the response text as "data: equals " with two spaces after "data:" and another space at the end.
But it would be turned into "data: equals" by line 208.

@litongjava
Copy link

I've fixed.please use https://www.npmjs.com/package/react-native-ssec

@EmilJunker
Copy link
Contributor

@litongjava Why create a new package and not make a pull request to this one so existing users can befenit?

@litongjava
Copy link

litongjava commented Jun 4, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants