-
Notifications
You must be signed in to change notification settings - Fork 79
Description
Migration to Granite 4.0 Models 🚀
Why this matters:
The granite-3.x model family is approaching its End of Life (EOL). Default configurations, examples, and tests must be migrated to the granite-4.0 family.
What needs to be done:
Identify all usages of Granite v3 (and earlier) and update them to their v4 equivalents.
Observations:
There maybe some confusion in mellea/backends/model_ids.py. The constant IBM_GRANITE_4_MICRO_3B is mapped to ibm/granite-4-h-small. However, ibm/granite-4-h-small is a "Hybrid Small" model (larger and more capable than "Micro"). This naming mismatch should be corrected during the migration.
Additionally, explicit Granite 4 usages were found in:
test/stdlib/components/intrinsic/test_rag.py: Usesibm-granite/granite-4.0-microdirectly.test/conftest.py: Referencesgranite4:microin docstrings for Ollama checks.
These should be harmonized with the new default constants.
Action Items:
- Model IDs: Address the observation above. Create a new constant (e.g.,
IBM_GRANITE_4_HYBRID_SMALL) foribm/granite-4-h-smalland ensureIBM_GRANITE_4_MICRO_3Bpoints to a true micro model (if available) or is deprecated/renamed. - Audit Usages: Check all usages of
IBM_GRANITE_4_MICRO_3Bto determine if they intended to use the Micro model or the Hybrid Small model (since the constant currently points to Hybrid Small on Watsonx). - Backend Defaults: Update
WatsonxAIBackenddefault model toibm/granite-4-h-small(using the new correct constant).- Note: The
WatsonxAIBackenditself is deprecated in favor ofLiteLLM(as noted in the codebase). However, the default should still be fixed while it remains in use.
- Note: The
- Documentation: Update
docs/tutorial.mdand other guides to reference Granite 4. - Examples: Ensure all examples in
docs/examples/run out-of-the-box with Granite 4. - Tests: Verify
uv run pytestpasses with the new defaults.
Note that this is also the cause of the deprecation warning seen during testing LifecycleWarning: Model 'ibm/granite-3-3-8b-instruct' is in deprecated state... alternative models: ibm/granite-4-h-small.
Affected Files (Preliminary Scan):
References to "granite" were found in the following files.:
Configuration & Core
pyproject.tomlmellea/backends/model_ids.pymellea/backends/watsonx.pymellea/backends/ollama.pymellea/backends/huggingface.pymellea/backends/litellm.pymellea/backends/openai.py
Documentation & Examples
docs/tutorial.mddocs/AGENTS_TEMPLATE.mddocs/alora.mddocs/dev/mellea_library.mddocs/examples/aLora/101_example.pydocs/examples/notebooks/model_options_example.ipynbdocs/examples/notebooks/simple_email.ipynbdocs/examples/instruct_validate_repair/101_email_with_validate.py(and others indocs/examples/)docs/kv_smash/hf_example.py
Tests
test/backends/test_watsonx.pytest/backends/test_ollama.pytest/backends/test_huggingface.pytest/conftest.py
I can start working through this - or ok if we want to split up. Does the scope make sense?
Do we want to reain any v3 tests? I presume not