Improve vectorizer kwargs and typing

[tylerhutcherson]

Changes Made

1. Expanded Type Support:

   • Updated return type signatures across all vectorizers to properly reflect the ability to return either data lists (List[float]) or binary buffers (bytes)
   • Added special handling for Cohere's integer embedding types (List[int])

2. Standardized Interface:

   • Uniform type annotations and docstrings across all vectorizer implementations
   • Consistent default batch sizes (10) for better predictability

3. Improved Provider-Specific Support:

   • Enhanced kwargs forwarding to allow passing provider-specific parameters
   • Better warnings for deprecated parameters (like Cohere's embedding_types)

4. Fixed Type Checking:

   • Added strategic type ignores to resolve MyPy errors
   • Made minimal changes to consumer code to handle the expanded return types

Motivation

These changes create a more consistent and flexible vectorizer interface that:

• Accurately represents what the methods can return
• Accommodates provider-specific features (like Cohere's integer embeddings)
• Provides clearer documentation for users
• Maintains backward compatibility

Future Improvements

For future consideration:

• Introduce helper methods (like embed_as_list()) that guarantee specific return types when needed
• Add more robust type conversion in consumer code that relies on specific types
• Develop a cleaner separation between the base vectorizer interface and provider-specific extensions
• Consider a more structured approach to provider-specific parameters

[tylerhutcherson]

Looking for feedback on the approach here, team. Let me know what you think. This doesn't break the interface or user experience, but enables us to leverage the cohere model for int8 embeddings. It also exposes maybe some gaps in the current broader implementation that we should review in prep for a 1.x.x release later this year.

@abrookins @rbs333 @bsbodden @justin-cechmanek

[justin-cechmanek]

I like the standardization, and better support for passing through kwargs. No critiques from me.

[abrookins]

Nice cleanup! 🔥

[abrookins]

Helpful!