Add role splitting in markdown processor, remove old guidance regions

[extremeheat]

Breaking

• Remove old "guidance regions"
• Flow arguments for prompt handling have been updated.
  • prompt: string replaced with prompt: { system?: string, user?: string } or prompt: { text: string, roles: Record<string, Role> } to support new role handling.

Features

• Support "role" splitting in markdown processor. See docs/MarkdownProcessor.md's role section for doc.
  • calling loadPrompt() now accepts a new arg, options, which can take an a "role" field, set to a Record<string, string> map. The keys are the tokens to split by and the values are the roles.
• Add "requestChatCompletion" API method on CompletionService for sending messages to models
  • Supports caching
• Old "guidance regions" removed from requestCompletion() in favor of a new "guidance" role in messages which can be used now via "requestChatCompletion"
• Markdown processor now strips markdown comments out
• content and text are returned on all completion requests, whether chat or not

Fixes

• Fix caching issues in CompletionService for requestCompletion
• Fixes to markdown tokenization (handle comments, preformatted blocks)
• Fix Flow transformResponse

[extremeheat]

/review

[import-bot]

Hey extremeheat, thanks for the pull request! This looks like a great update. I have a few comments and suggestions:

1. In docs/MarkdownProcessing.md, I think it would be helpful to explicitly state that the guidance role should be the last message in the array. This is mentioned in the note at the end of the section, but it might be easy to miss.

   Here's the code I'm referring to:

   Note: there can only be one guidance message and it must be the last one. You should remove
   it from the messages array the next call you do to requestChatCompletion. This feature works
   best when used with the role parsing system above.

   I suggest adding a sentence like this to the main paragraph:

   "The guidance role should always be the last message in the messages array."

2. In src/Flow.js, the _hash function now uses JSON.stringify to hash the arguments. While this works, it might be more efficient to use a dedicated hashing library like crypto to create a hash of the arguments. This would avoid the overhead of stringifying the arguments, especially if they are large objects.

   Here's the code I'm referring to:

   _hash (...args) {
     const hash = crypto.createHash('sha1')
     hash.update(JSON.stringify(args))
     return hash.digest('hex')
   }

3. In src/tools/mdp.js, the segmentByRoles function uses a loop to iterate over the text and find the role tokens. This could potentially be optimized by using a regular expression to find all the role tokens at once.

   Here's the code I'm referring to:

   function segmentByRoles (text, roles) {
     // split the text into segments based on the roles
     const segments = []
     for (let i = 0; i < text.length; i++) {
       for (const role in roles) {
         if (text.slice(i, i + role.length) === role) {
           segments.push({ role, start: i, end: i + role.length })
           i += role.length
           break
         }
       }
     }

4. In src/tools/stripping.js, the tokenizeMarkdown function has a complex logic to handle code blocks and preformatted blocks. It might be worth considering using a dedicated markdown parsing library to handle this logic, which could potentially simplify the code and make it more robust.

   Here's the code I'm referring to:

   function tokenizeMarkdown (comment, options) {
     const tokens = []
     let tokenSoFar = ''
     let inCodeBlock = false
     let inCodeLang
     let inPreTag = false
     let linePadding = 0

Overall, I think this pull request is well-written and introduces some valuable features. I would be happy to accept it once the above comments are addressed.

[import-bot]

In docs/MarkdownProcessing.md, I think it would be helpful to explicitly state that the guidance role should be the last message in the array. This is mentioned in the note at the end of the section, but it might be easy to miss.

Here's the code I'm referring to:

Note: there can only be one guidance message and it must be the last one. You should remove
it from the messages array the next call you do to requestChatCompletion. This feature works
best when used with the role parsing system above.

I suggest adding a sentence like this to the main paragraph:

"The guidance role should always be the last message in the messages array."

[import-bot]

In src/Flow.js, the _hash function now uses JSON.stringify to hash the arguments. While this works, it might be more efficient to use a dedicated hashing library like crypto to create a hash of the arguments. This would avoid the overhead of stringifying the arguments, especially if they are large objects.

[import-bot]

In src/tools/mdp.js, the segmentByRoles function uses a loop to iterate over the text and find the role tokens. This could potentially be optimized by using a regular expression to find all the role tokens at once.

[import-bot]

In src/tools/stripping.js, the tokenizeMarkdown function has a complex logic to handle code blocks and preformatted blocks. It might be worth considering using a dedicated markdown parsing library to handle this logic, which could potentially simplify the code and make it more robust.