Use litellm to allow use of models from various providers #564
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… information cannot be retrieved. # Aider chat conversation: USER: create the `NoModelInfo` class ASSISTANT: ```diff --- aider/models/model.py +++ aider/models/model.py @@ ... @@ from dataclasses import dataclass, fields import litellm +from typing import Optional from PIL import Image from aider.dump import dump +class NoModelInfo(Exception): + """ + Exception raised when model information cannot be retrieved. + """ + def __init__(self, message: Optional[str] = None): + super().__init__(message or "No model information available.") + @DataClass class ModelSettings: name: str ```
…ortlib.resources.files` for accessing package resources in Python 3.9+. # Aider chat conversation: USER: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html ASSISTANT: To address the `DeprecationWarning` regarding `pkg_resources`, we will replace its usage with the recommended `importlib.resources` or `importlib.metadata` from the Python standard library. The specific replacement depends on the original use case of `pkg_resources`. In the provided code, `pkg_resources` is used to access package data, so we will use `importlib.resources`. Here are the diffs for the necessary changes: ```diff --- aider/repomap.py +++ aider/repomap.py @@ ... @@ import networkx as nx -import pkg_resources +import importlib.resources as pkg_resources from diskcache import Cache from grep_ast import TreeContext, filename_to_lang from pygments.lexers import guess_lexer_for_filename from pygments.token import Token from pygments.util import ClassNotFound @@ ... @@ # Load the tags queries try: - scm_fname = pkg_resources.resource_filename( - __name__, os.path.join("queries", f"tree-sitter-{lang}-tags.scm") - ) + scm_fname = pkg_resources.files(__package__).joinpath( + "queries", f"tree-sitter-{lang}-tags.scm" + ) except KeyError: return - query_scm = Path(scm_fname) + query_scm = scm_fname if not query_scm.exists(): return query_scm = query_scm.read_text() ``` These changes replace the deprecated `pkg_resources.resource_filename` with `importlib.resources.files`, which is the recommended way to access package resources in Python 3.9 and later. The `files()` function returns a `Path`-like object that can be used to read the contents of the resource.
|
JFYI I am successfully using this pull request with --opus, thank you for this work. |
…dels containing the given name or a slight misspelling.
# Aider chat conversation:
USER: implemenet the `fuzzy_match_models` function. takes the name of a LLM like "gpt-3.5" and searches the list of `models` to find models which contain the name or contain a slight misspelling of the name
ASSISTANT: Here is an implementation of the `fuzzy_match_models` function that searches for models containing the given name or a slight misspelling:
aider/models.py
```python
<<<<<<< SEARCH
def fuzzy_match_models(name):
models = litellm.most_cost.keys()
=======
import difflib
def fuzzy_match_models(name):
models = litellm.most_cost.keys()
# Check for exact match first
if name in models:
return [name]
# Check for models containing the name
matching_models = [model for model in models if name in model]
# If no matches found, check for slight misspellings
if not matching_models:
matching_models = difflib.get_close_matches(name, models, n=3, cutoff=0.8)
return matching_models
>>>>>>> REPLACE
```
This implementation does the following:
1. It first checks if the given `name` exactly matches any model in the `models` list. If an exact match is found, it returns a list containing only that model name.
2. If no exact match is found, it searches for models that contain the given `name` as a substring. It uses a list comprehension to filter the `models` list and find models that include `name`.
3. If still no matches are found, it uses the `difflib.get_close_matches()` function to find models that are close matches to the given `name`. It searches for up to 3 close matches with a similarity ratio of at least 0.8 (80%).
4. Finally, it returns the list of matching models found either by exact match, substring match, or close match.
This implementation allows for fuzzy matching of model names, considering both exact matches and slight misspellings.
… name from the command line and printing the results.
# Aider chat conversation:
USER: add a main func so i can run this script with a model name on the cmd line and print the results
ASSISTANT: Here are the changes to add a `main` function that allows running the script with a model name from the command line and printing the results:
aider/models.py
```python
<<<<<<< SEARCH
=======
import sys
def main():
if len(sys.argv) != 2:
print("Usage: python models.py <model_name>")
sys.exit(1)
model_name = sys.argv[1]
matching_models = fuzzy_match_models(model_name)
if matching_models:
print(f"Matching models for '{model_name}':")
for model in matching_models:
print(model)
else:
print(f"No matching models found for '{model_name}'.")
if __name__ == "__main__":
main()
>>>>>>> REPLACE
```
The `main` function does the following:
1. It checks if the script is called with exactly one command-line argument (the model name). If not, it prints the usage instructions and exits with an error code.
2. It retrieves the model name from the command-line argument using `sys.argv[1]`.
3. It calls the `fuzzy_match_models` function with the provided model name to get the list of matching models.
4. If matching models are found, it prints a message indicating the matching models for the given model name and then prints each matching model on a separate line.
5. If no matching models are found, it prints a message indicating that no matching models were found for the given model name.
The `if __name__ == "__main__":` block ensures that the `main` function is only executed when the script is run directly, not when it is imported as a module.
With these changes, you can run the script from the command line by providing a model name as an argument, and it will print the matching models or a message if no matches are found.
|
@paul-gauthier Is there a way to specify the litellm host that you know of? I have litellm running on a different node. The idea being to setup one litellm proxy with access to all LLM's for out team to track costs / etc. It runs for example on 192.168.3.226 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.