Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Alibaba Cloud full modal docking error #2302

Merged
merged 1 commit into from
Feb 17, 2025
Merged

fix: Alibaba Cloud full modal docking error #2302

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion apps/setting/models_provider/impl/aliyun_bai_lian_model_provider/model/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
from setting.models_provider.base_model_provider import MaxKBBaseModel
from setting.models_provider.impl.base_chat_open_ai import BaseChatOpenAI


class BaiLianChatModel(MaxKBBaseModel, BaseChatOpenAI):
@staticmethod
def is_cache_model():
Expand All @@ -14,6 +13,8 @@ def is_cache_model():
@staticmethod
def new_instance(model_type, model_name, model_credential: Dict[str, object], **model_kwargs):
optional_params = MaxKBBaseModel.filter_optional_params(model_kwargs)
if model_name == 'qwen-omni-turbo':
optional_params['streaming'] = True
return BaiLianChatModel(
model=model_name,
openai_api_base=model_credential.get('api_base'),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one potential issue in the provided Python code:

The new_instance method does not handle other models besides 'qwen-omni-turbo'. If another model requires streaming to be enabled, this logic should also include that functionality.

Here is a revised version of the new_instance method with added support for additional models:

from typing import Dict

def new_instance(model_type, model_name, model_credential: Dict[str, object], **model_kwargs):
    optional_params = MaxKBBaseModel.filter_optional_params(model_kwargs)
    
    try:
        # Apply default stream parameter based on known supported models
        if model_name == 'qwen-omni-turbo' or model_name == 'another_supported_model':
            optional_params['streaming'] = True
        
        return BaiLianChatModel(
            model=model_name,
            openai_api_base=model_credential.get('api_base'),
            openai_access_token=model_credential.get('access_key'),
            prompt_history_size=model_credential.get('prompt_history_size', 1),
            max_seq_len=model_credential.get('max_output_length', 4096),
            temperature=model_credential.get('temperature', 0.7),
            top_p=model_credential.get('top_p', 0.95),
            history_num=model_credential.get('history_capacity', 2),
            **optional_params
        )
    except AttributeError as e:
        print(f"An error occurred while setting up the model instance: {e}")

In this updated function, I've included placeholder values (another_supported_model) where you would need to add any additional models that might require streaming. The function catches AttributeError to ensure robustness even if there's an unexpected configuration change within your application. This ensures all relevant parameters are correctly set when creating a BaiLianChatModel.

Expand Down
2 changes: 2 additions & 0 deletions apps/setting/models_provider/impl/aliyun_bai_lian_model_provider/model/stt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ def new_instance(model_type, model_name, model_credential: Dict[str, object], **
optional_params['max_tokens'] = model_kwargs['max_tokens']
if 'temperature' in model_kwargs and model_kwargs['temperature'] is not None:
optional_params['temperature'] = model_kwargs['temperature']
if model_name == 'qwen-omni-turbo':
optional_params['streaming'] = True
return AliyunBaiLianSpeechToText(
model=model_name,
api_key=model_credential.get('api_key'),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code appears to be part of a function that initializes an instance of AliyunBaiLianSpeechToText with various parameters. It checks and includes certain key settings based on the input model_kwargs. Here are some observations:

  1. Conditional Streaming: The line if model_name == 'qwen-omni-turbo': optional_params['streaming'] = True ensures that streaming mode is enabled only when the model name is 'qwen-omni-turbo'. This is useful if there's specific behavior or integration requirements associated with this model.

  2. Temperature Check: The existing checks for max_tokens and temperature ensure that they are set correctly before being passed to the AliyunBaiLianSpeechToText constructor, which seems appropriate.

  3. Code Structure: The structure is neat and follows Python style guidelines. There's no apparent issue with readability or consistency in the code.

Summary of Changes:

No changes were suggested since the current implementation already addresses the expected use cases and best practices for setting up the parameters. If streaming isn't typically required across all instances of this speech-to-text service, it might make sense to remove the conditionals around enabling streaming.

If you need further modifications or optimizations, feel free to let me know!

Expand Down