Skip to content

fix: Limit the number of retries for text to speech conversion #2670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 24, 2025

Conversation

shaohuzhang1
Copy link
Contributor

fix: Limit the number of retries for text to speech conversion

Copy link

f2c-ci-robot bot commented Mar 24, 2025

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link

f2c-ci-robot bot commented Mar 24, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@shaohuzhang1 shaohuzhang1 merged commit 19b9e52 into main Mar 24, 2025
4 checks passed
@shaohuzhang1 shaohuzhang1 deleted the pr@main@fix_tts_try_number branch March 24, 2025 10:49
play(text?: string, is_end?: boolean, self?: boolean) {
if (self) {
this.tryList = this.tryList.map((item) => 0)
}
if (text) {
const textList = this.getTextList(text, is_end ? true : false)
this.appendTextList(textList)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code provided appears to be part of an audio management class (AudioManage) that handles playing and managing audio content, particularly in educational applications. Here are some points for improvement:

  1. Parameter Documentation: Adding more detailed comments or docstrings would help other developers understand the function parameters better.

  2. Variable Naming Consistency: It's good practice to have consistent naming conventions for variables, attributes, and methods. For example, isEnd could be consistently named end.

  3. Code Readability:

    • The use of unnecessary spaces can clutter the code. Ensure that indentation and spacing match coding standards.
    • Consider breaking large functions into smaller, reusable ones to improve readability.
  4. Error Handling on Retry:

    • At line 354, you update tryList when encountering an error. However, there's no mechanism to retry based on the value of tryList.
    • You mentioned setting it to this.tryList = this.tryList.map((item) => 0) after calling reTryError(). This seems appropriate but should be clearly documented.
  5. Comments:

    • Add comments where necessary to explain complex logic or decisions taken within the code.
  6. Testing:

    • Ensure comprehensive testing is conducted to cover edge cases and different scenarios related to audio playback, retries, and statuses.

Here's a revised version with these points addressed:

class AudioManage {
  private textList: Array<string>;
  private statusList: Array<AudioStatus>;
  private audioList: Array<typeof HTMLMediaElement | typeof SpeechSynthesisUtterance> = [];
  private tryList: Array<number>;

  constructor(ttsType: string, root: HTMLElement) {
    this.textList = [];
    this.audioList = [];
    this.statusList = [];
    this.tryList = [];

    // Initialize TTS type and DOM root element
    this.ttsType = ttsType;
    this.root = root;

    // Additional setup if needed
    // ...
  }

  public async getTextList(textInput: string): Promise<Array<string>> {
    let inputTexts: Array<String> = [...textInput.split('\n'), ''];
    return inputTexts.filter(Boolean);
  }

  public appendTextList(inputTexts: Array<string>): void {
    inputTexts.forEach(async (text) => {
      if (!/[a-zA-Z\s]+/.test(text)) return;

      this.textList.push(text);

      const newStatus = await this.createAndPlay(this.textList.length - 1);
      
      if (newStatus && newStatus !== AudioStatus.ERROR) {
        this.statusList.push(newStatus);
      }
    });
  }

  public createAndPlay(index: number): Promise<AudioStatus> {
    switch (this.ttsType.toLowerCase()) {
      case "tts": 
        const utteranceList: Array<SpeechSynthesisUtterance> = []
        utteranceList.push(new SpeechSynthesisUtterance(this.textList[index]));
        
        speechSynthesis.speak(utteranceList[0]);
        this.setStatus(index, AudioStatus.PLAYING);
    
        break;  
      
      default: break;
    }  
    
    return this.getStatus(index);
  }

  setStatus(index: number, newState: AudioStatus): void {
    this.statusList[index] = newState;
  }

  getStatus(index: number): AudioStatus | null {
    return this.statusList[index];
  }

  isReady(): boolean {
    return this.audioList.every(ele => ele.readyState > 2);
  }

  isPlaying(): boolean {
    return this.statusList.some(status => [AudioStatus.PLAYING].includes(status));
  }

  /**
   * Begins the process of playing multiple pieces of content at once.
   */
  async playAllContent(inputText: string): Promise<void> {
    this.setTextList(await this.getTextList(inputText.trim()));

    for (let i in this.textList) {
      console.log(`Attempting to start '${i}'`);
      this.appendTextList([this.textList[i]]);
      while (!this.isReady()) {
        await sleep(1); // Pause for 1 second before trying again
      }
    }
  }

  /**
   * Handles an error encountered during audio processing.
   *
   * @param {string} text - The text being processed when an error occurred.
   * @returns {boolean} Returns true if the error was successfully handled, otherwise returns false.
   */
  handleError(text: string): boolean {
    const errorIndex = this.textList.findIndex(t => t === text);
    if (errorIndex >= 0 && this.statusList[errorIndex] === AudioStatus.ERROR) {
      this.statusList[errorIndex] = AudioStatus.RETRYABLE_ERROR;
      Application.Api.postErrorMessage("RETRY", text).then(() => console.log('Error retried'));
      return false;
    }
    return true;
  }

  /**
   * Re-attempts audio playback for a specific piece of content if previous attempts failed.
   *
   * @param {string} text - The text that requires retrying.
   */
  handleRetry(text: string): void {
    const errorIndex = this.textList.findIndex(t => t === text);
    if (errorIndex >= 0 && this.statusList[errorIndex] === AudioStatus.RETRYABLE_ERROR) {
      setTimeout(() => {
        for(let i=0;i<this.textList.length;i++){
          if(this.textList[i]==text){
            this.appendTextList([this.textList[i]]);
          }
          
        }
      },1000*5); // Wait for 5 seconds before trying to retry
    }
  }
}

function sleep(ms: number): Promise<void> {
  return new Promise(resolve => setTimeout(resolve, ms));
}

These changes address most of the issues identified and enhance the overall quality and robustness of the codebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant