Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[example] batched-bench "segmentation fault" #8839

Merged
merged 2 commits into from
Aug 4, 2024

Commits on Aug 3, 2024

  1. [example] batched-bench "segmentation fault"

    When `llama-batched-bench` is invoked _without_ setting `-npl`, "number
    of parallel prompts", it segfaults.
    
    The segfault is caused by invoking `max_element()` on a zero-length
    vector, `n_pl`
    
    This commit addresses that by first checking to see if the number of
    parallel prompts is zero, and if so sets the maximum sequence size to 1;
    otherwise, sets it to the original, the result of `max_element()`.
    
    Fixes, when running `lldb build/bin/llama-batched-bench -- -m models/Meta-Llama-3-8B.gguf`
    
    ```
    * thread ggerganov#1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
        frame #0: 0x000000010000366c llama-batched-bench`main(argc=3, argv=0x000000016fdff268) at batched-bench.cpp:72:28
       69  	    llama_context_params ctx_params = llama_context_params_from_gpt_params(params);
       70
       71  	    // ensure enough sequences are available
    -> 72  	    ctx_params.n_seq_max = *std::max_element(n_pl.begin(), n_pl.end());
    ```
    cunnie committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    8eca9b5 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Update examples/batched-bench/batched-bench.cpp

    Co-authored-by: compilade <git@compilade.net>
    ggerganov and compilade authored Aug 4, 2024
    Configuration menu
    Copy the full SHA
    adb79aa View commit details
    Browse the repository at this point in the history