chore: Bump mistral.rs, llama.cpp and tokenizers deps #2338
Conversation
grahamking
force-pushed
the
gk-mistralrs-bump
branch
from
dc0b15c to
e9f7dac
Compare
WalkthroughThis set of changes updates dependency versions in several Cargo.toml files across different engine libraries, including Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant MistralRsEngine
participant GGUFSpecificConfig
User->>MistralRsEngine: new()
MistralRsEngine->>GGUFSpecificConfig: default()
GGUFSpecificConfig-->>MistralRsEngine: returns config
MistralRsEngine-->>User: engine instance
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: .coderabbit.yaml ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (4)
🧰 Additional context used🧠 Learnings (5)📚 Learning: in docs/support_matrix.md, the nixl version difference between runtime dependencies (0.5.0) and buil...Applied to files:
📚 Learning: the codebase uses async-nats version 0.40, not the older nats crate. error handling should use async...Applied to files:
📚 Learning: the asyncenginecontextprovider trait in lib/runtime/src/engine.rs was intentionally changed from `se...Applied to files:
📚 Learning: in lib/runtime/src/component/client.rs, the current mutex usage in get_or_create_dynamic_instance_so...Applied to files:
📚 Learning: the ai-dynamo/dynamo project does not ship arm64 wheels, so platform markers to restrict dependencie...Applied to files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
🔇 Additional comments (6)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
WalkthroughThis change updates dependency versions across several Cargo.toml files for the Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant MistralRsEngine
participant GGUFSpecificConfig
User->>MistralRsEngine: new()
MistralRsEngine->>GGUFSpecificConfig: default()
GGUFSpecificConfig-->>MistralRsEngine: returns default config
MistralRsEngine-->>User: Engine initialized
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
lib/llm/Cargo.toml (1)
83-83: Verify transitive-API compatibility for bumped cratesAll three bumps (
candle-core 0.9.1,cudarc 0.16.6,tokenizers 0.21.4) look reasonable, but each of these crates introduced minor API tweaks in 0.9/0.16/0.21 lines.
Please run a full workspacecargo checkon the PR branch with all feature combinations (testing-full,testing-cuda, etc.) to catch any breakage early.No code changes needed here—just a heads-up.
Optional: document the minimal crate versions in
docs/support_matrix.mdto keep them discoverable.Also applies to: 94-94, 105-109
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
Cargo.lockis excluded by!**/*.locklib/bindings/python/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
lib/engines/llamacpp/Cargo.toml(1 hunks)lib/engines/mistralrs/Cargo.toml(1 hunks)lib/engines/mistralrs/src/lib.rs(1 hunks)lib/llm/Cargo.toml(3 hunks)
🧰 Additional context used
🧠 Learnings (6)
📓 Common learnings
Learnt from: dmitry-tokarev-nv
PR: ai-dynamo/dynamo#2179
File: docs/support_matrix.md:61-63
Timestamp: 2025-07-30T00:34:35.810Z
Learning: In docs/support_matrix.md, the NIXL version difference between runtime dependencies (0.5.0) and build dependencies (0.4.0) is intentional and expected, not an error that needs to be corrected.
📚 Learning: in docs/support_matrix.md, the nixl version difference between runtime dependencies (0.5.0) and buil...
Learnt from: dmitry-tokarev-nv
PR: ai-dynamo/dynamo#2179
File: docs/support_matrix.md:61-63
Timestamp: 2025-07-30T00:34:35.810Z
Learning: In docs/support_matrix.md, the NIXL version difference between runtime dependencies (0.5.0) and build dependencies (0.4.0) is intentional and expected, not an error that needs to be corrected.
Applied to files:
lib/engines/mistralrs/Cargo.tomllib/engines/llamacpp/Cargo.tomllib/llm/Cargo.toml
📚 Learning: the codebase uses async-nats version 0.40, not the older nats crate. error handling should use async...
Learnt from: kthui
PR: ai-dynamo/dynamo#1424
File: lib/runtime/src/pipeline/network/egress/push_router.rs:204-209
Timestamp: 2025-06-13T22:07:24.843Z
Learning: The codebase uses async-nats version 0.40, not the older nats crate. Error handling should use async_nats::error::Error variants, not nats::Error variants.
Applied to files:
lib/engines/mistralrs/Cargo.toml
📚 Learning: the asyncenginecontextprovider trait in lib/runtime/src/engine.rs was intentionally changed from `se...
Learnt from: ryanolson
PR: ai-dynamo/dynamo#1919
File: lib/runtime/src/engine.rs:168-168
Timestamp: 2025-07-14T21:25:56.930Z
Learning: The AsyncEngineContextProvider trait in lib/runtime/src/engine.rs was intentionally changed from `Send + Sync + Debug` to `Send + Debug` because the Sync bound was overly constraining. The trait should only require Send + Debug as designed.
Applied to files:
lib/engines/mistralrs/Cargo.tomllib/engines/mistralrs/src/lib.rs
📚 Learning: in lib/runtime/src/component/client.rs, the current mutex usage in get_or_create_dynamic_instance_so...
Learnt from: grahamking
PR: ai-dynamo/dynamo#1962
File: lib/runtime/src/component/client.rs:270-273
Timestamp: 2025-07-16T12:41:12.543Z
Learning: In lib/runtime/src/component/client.rs, the current mutex usage in get_or_create_dynamic_instance_source is temporary while evaluating whether the mutex can be dropped entirely. The code currently has a race condition between try_lock and lock().await, but this is acknowledged as an interim state during the performance optimization process.
Applied to files:
lib/engines/mistralrs/Cargo.toml
📚 Learning: the ai-dynamo/dynamo project does not ship arm64 wheels, so platform markers to restrict dependencie...
Learnt from: dmitry-tokarev-nv
PR: ai-dynamo/dynamo#2300
File: pyproject.toml:64-66
Timestamp: 2025-08-05T22:51:59.230Z
Learning: The ai-dynamo/dynamo project does not ship ARM64 wheels, so platform markers to restrict dependencies to x86_64 are not needed in pyproject.toml dependencies.
Applied to files:
lib/engines/llamacpp/Cargo.toml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: pre-merge-rust (.)
- GitHub Check: pre-merge-rust (lib/runtime/examples)
- GitHub Check: pre-merge-rust (lib/bindings/python)
- GitHub Check: Mirror Repository to GitLab
- GitHub Check: Build and Test - vllm
🔇 Additional comments (3)
lib/engines/llamacpp/Cargo.toml (1)
31-31: Confirm no lingeringdynamic-linkreferencesI ran a repository-wide search for “dynamic-link” and found no matches. However, absence of results isn’t definitive—please manually verify that:
- No downstream Cargo.toml still lists
dynamic-linkin afeaturessection- There are no
#[cfg(feature = "dynamic-link")]gates in the codebaseThis will ensure the workspace compiles cleanly after the
llama-cpp-2 = { version = "0.1.108", default-features = false }update.lib/engines/mistralrs/Cargo.toml (1)
42-42: Pinned commit updated – keep an eye on upstream API driftThe
mistralrsrev moves from8a4faf3→53ebc74. Good to stay current, but because we vendor by commit hash, any force-push upstream will silently break reproducible builds. Consider switching to a version tag once 0.6.x stabilises.lib/engines/mistralrs/src/lib.rs (1)
95-96: Simplification OK – semantics unchangedReplacing the manual
GGUFSpecificConfig { .. }construction withGGUFSpecificConfig::default()is equivalent (all fields wereNone). No functional impact observed.
|
FYI: looks like this change causes a build regression: Works: Doesn't work: |
|
This PR also adds support for CUDA 12.9. |
|
@da-x Did that double-feature work with cudarc 0.16.2? |
|
|
Nice. Would you mind send a PR and tagging @ryanolson ? He added cudarc for block manager. |
Summary by CodeRabbit