Convai Unreal Engine SDK Version 3.0.0

Added

• New lipsync component FaceSync which no longer requires ConvaiOVRLipsync plugin.
• Mac Lipsync support.
• Emotions.
• Object in Attention.
• New Convai Chatbot Component functions:
  • Invoke Speech.
  • Invoke Narrative Design Trigger.
  • Force Set Emotion.
  • Reset Emotion State.
  • Get Emotion Score.
  • Get Talking Time Elapsed.
  • Get Talking Time Remaining.
  • Set Lipsync Component.
  • Clear Action Queue.
• New Convai Chatbot Component Events:
  • On Narrative Section Recevied.
  • On Emotion State Changed.
• ConvaiGetLookedAtObjectOrCharacter function
• Actor loses focus with player after predefined time.
• Move To and Follow Actions failures will trigger a character response.
• Environment replication for multiplayer usecases.

Improved

• MetaHuman body Animations, Facial expressions and eye look at.
• Realusion Animation blueprint, lipsync and eye look at.
• Settings widget UI.
• Actions accuracy.
• Audio capture for low frame rate.

Fixed

• Packaging issue if Convai Player Component was in the scene prior to packaging.
• Crash when many interactions were happening simultaneously.
• ConvaiBasePlayerWithVoiceActivation Beta class was only working for the first interaction.
• Overall plugin stability and warnings.

Deprecated

• ConvaiOVRLipsync plugin and component - Please delete the plugin from your project and use the newFaceSync component instead.

Convai Unreal Engine SDK Version 2.9.0

Added

• Unreal Engine 5.3 Support.

Fixed

• Actions with more than 1 word was sometimes ignored.
• Player name would show as Unknown in multiplayer at some cases.

Convai Unreal Engine SDK Version 2.8.0

Added

• New Action System. (Tutorial Coming Soon!)
• Functions: Get All Convai Player Components and Get All Convai Chatbot Components.

Updated

• Improved MetaHuman body Animations and Facial expressions.
• Better OVR Lipsync for MetaHuman.
• Better capturing of text input as an extra parameter in Actions.

Fixed

• Rare crash caused by UE grabage collection during lipsync.
• Rare crash (Editor only) caused by exiting play mode before the AI character has finished its response.
• Random Freeze that happens when talking to an AI character for the first time.
• Crash when using Convai Speech to Text function with an imported audio.
• MetaHuman rotating 90 degrees when in play mode.

Convai Unreal Engine SDK Version 2.7.0

Added

• Full multiplayer support with real-time AI avatar conversations and voice chat.
• Environment awareness for AI avatars to do actions on surrounding objects and other players in multiplayer.
• Player name customization feature.
• Toggle for switching between client and server API keys.
• Multiplayer support for animations: Reallusion, Ready Player Me, and Unreal Engine Metahumans.
• Chat UI with multiplayer chat replication.

Updated

• Three distinct modes for Chat UI were added for a more intuitive user experience.

Fixed

• Game crash issue caused by curse words.
• Bug where non-English texts were not processed correctly.
• "Upstream Max Limit Exceeded" error when using the gRPC API.

Convai Unreal Engine SDK Version 2.6.1

Added

• High-Quality character voices support in multiplayer.

Updated

• Updated Documentation.
• Increased Actions accuracy.
• Replicate Chatbot events in multiplayer for easier integration in multiplayer apps.

Fixed

• LipSync lag.
• Interrupting a character in multilayer would sometimes not interrupt it on all other clients.

Convai Unreal Engine SDK Version 2.6.0

Added

• Mac Support.
• New MetaHuman body animations and Facial expressions.
• Reallusion characters support.

Updated

• Improved MetaHuman lipsync.

Fixed

• Sending non-English text won't get processed.
• Potential microphone issue when it gets detected as zero channel.

Convai Unreal Engine SDK Version 2.5.1

• Added official Android support to the plugin on the marketplace.
• Fixed issue with Behaviour Tree where unhandled actions were not sent to the character blueprint.

Convai Unreal Engine SDK Version 2.5.0-beta

Added

• Added three new chat widget UIs.
• Added Multilingual font support for non-English languages.

Convai Unreal Engine SDK Version 2.4.0

Added

• Added 'IsListening' function to detect when the character is actively listening to the player.
• Enhanced Mic Controls: Added the 'SetMicrophoneVolumeMultiplier' and 'GetMicrophoneVolumeMultiplier' functions to allow better control over audio settings. A mic gain slider has also been introduced for easy adjustments to the Mic widget.
• Extended Language Support: Increased Voice Capture Sample Rate to 16kHz, improving support for Korean Speech-To-Text.

Updated

• User Interface: The UI is now only initialized if the blueprint is controlled by a player, providing a smoother user experience.
• Blueprint Organization: Reorganized blueprints for easier navigation, complete with improved commentary for better understanding.
• Animation Tuning: The talking animation has been toned down to offer a more natural visual experience.

Fixed

• Player Component Acquisition: Resolved an issue where obtaining the player component in the mic widget was problematic.
• Packaging Issues: Addressed packaging issues with ConvaiOVRLipSync and a problem where fonts disappeared after packaging.
• Texture Display: Fixed a grey texture issue with the ReadyPlayerMe character in the demo project.
• Crash Issues: Fixed instances where crashes occurred due to logging an invalid variable and during mic recordings when no audio data was available.
• Component Initialization: Fixed ConvaiAudioComponent initialization errors.
• Missing Player Controller in the Demo Map.

Convai Unreal Engine SDK Version 2.3.0

Added

• Beta Android support.
• Audio permission on Android.
• Mini-Example demo project.
• Mic input device selection functionalities:
  • SetCaptureDeviceByName.
  • SetCaptureDeviceByIndex.
  • GetActiveCaptureDevice.
  • GetAvailableCaptureDeviceNames.
  • GetAvailableCaptureDeviceDetails.
  • GetCaptureDeviceInfo.
  • GetDefaultCaptureDeviceInfo.
• Mic settings menu widget blueprint with the ConvaiBasePlayer blueprint.
• language code and avatar image link outputs to the ConvaiGetCharacterDetails function.
• Convai Download Image function to download images.
• Convai attribution logo widget.

Updated

• Audio capturing by implementing ConvaiAudioCaptureComponent to replace IVoiceModule for audio capture.
• glTFRuntime in the MetaHuman demo project and ConvaiReadyPlayerMe plugin bundle.

Fixed

• C++ project error on restart.
• Microphone issues on some clients.
• Dark face for ReadyPlayerMe avatars.