Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove binary state from high-level API and use Jinja templates #3147

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

Remove binary state from high-level API and use Jinja templates #3147

wants to merge 24 commits into from
+1,364 −1,700

Commits on Nov 5, 2024

  1. initial implementation of Jinja templates 

    - Python bindings use `jinja2`
- server.cpp is not implemented
- chatapi.cpp is not implemented

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    f80e5f9 View commit details
    Browse the repository at this point in the history

  2. add back trimming of trailing whitespace 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    593604a View commit details
    Browse the repository at this point in the history

  3. separate applyJinjaTemplate from chat model 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    52f1d2e View commit details
    Browse the repository at this point in the history

  4. implement chat naming and follow-up questions 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    15abe1f View commit details
    Browse the repository at this point in the history

  5. fix chat save/restore 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    495f322 View commit details
    Browse the repository at this point in the history

  6. implement the regenerate button 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    8569509 View commit details
    Browse the repository at this point in the history

  7. add TODO comment 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    827f9f8 View commit details
    Browse the repository at this point in the history

  8. fix off-by-one error 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    0b97849 View commit details
    Browse the repository at this point in the history

  9. implement system prompt and fix equality check 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    f7e07e0 View commit details
    Browse the repository at this point in the history

  10. gcc 12 apparently does not include the cend method for std::span. 

    Signed-off-by: Adam Treat <treat.adam@gmail.com>
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    2710d87 View commit details
    Browse the repository at this point in the history

  11. Remove extra information from jinja. 

    Signed-off-by: Adam Treat <treat.adam@gmail.com>
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    62a9751 View commit details
    Browse the repository at this point in the history

  12. Revert "Remove extra information from jinja." 

    This reverts commit 15e8fba.
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    9e93427 View commit details
    Browse the repository at this point in the history

  13. Update the sources in chatllm directly and make it thread safe to do so. 

    Signed-off-by: Adam Treat <treat.adam@gmail.com>
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    e6c5c71 View commit details
    Browse the repository at this point in the history

  14. Add a fixme for a potential crasher. 

    Signed-off-by: Adam Treat <treat.adam@gmail.com>
    @manyoso @cebtenzzre
    manyoso authored and cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    d75ffc2 View commit details
    Browse the repository at this point in the history

  15. remove unused EOS token from params, fix hardcoded BOS token 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    c91c990 View commit details
    Browse the repository at this point in the history

  16. jinja helpers: fix key casing 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    d2b6ba3 View commit details
    Browse the repository at this point in the history

  17. llamamodel: restore add_special=true 

    We still need this for models that don't include bos_token in their chat
template. Llama 3.1 8B Instruct sets this to false.

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    05cdcab View commit details
    Browse the repository at this point in the history

  18. get the local server working again 

    Importantly, the non-chat completions endpoint (`/v1/completions`) no
longer uses a system prompt or LocalDocs, as those are not applicable.

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    4c5d9c3 View commit details
    Browse the repository at this point in the history

  19. tests: fix the expected output of the non-completions test 

    Before this PR, GPT4All inserted a system message into non-chat
completions, and it attempted to use LocalDocs with them. It no longer
does either of these things because they do not make sense here.

This changes the output slightly, so the test needs to be updated.

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    121829b View commit details
    Browse the repository at this point in the history

  20. tests: simplify comparison 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    2ab57b1 View commit details
    Browse the repository at this point in the history

  21. server: fix missing 'else' 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    c842eb2 View commit details
    Browse the repository at this point in the history

  22. llamamodel: remove debugging offset 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    9343a5e View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. do not generate name or follow-up questions for server chats 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    f7f399a View commit details
    Browse the repository at this point in the history

  2. remove views::enumerate for GCC 12 compat 

    Signed-off-by: Jared Van Bortel <jared@nomic.ai>
    @cebtenzzre
    cebtenzzre committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    68b02a7 View commit details
    Browse the repository at this point in the history