Skip to content

Commit

Permalink
audio strength stream selection docs (#985)
Browse files Browse the repository at this point in the history
* audio strength stream selection doc

* review updates

* update

* review-updates

* language review

---------

Co-authored-by: saudsami <saudsami@gmail.com>
Co-authored-by: Saud <65331551+saudsami@users.noreply.github.com>
Co-authored-by: atovpeko <anastasia.tovpeko@agora.io>
  • Loading branch information
4 people authored Mar 18, 2024
1 parent 9b0265c commit 22dd441
Show file tree
Hide file tree
Showing 23 changed files with 135 additions and 22 deletions.
Binary file added assets/images/video-sdk/use-case-scenario.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,4 +1,51 @@
import * as data from '@site/data/variables';
import ProjectImplement from '@docs/shared/video-sdk/develop/audio-strength-stream-selection/project-implementation/index.mdx';
import Reference from '@docs/shared/video-sdk/develop/audio-strength-stream-selection/reference/index.mdx';
import NotAvailable from '@docs/shared/common/not-available.mdx'

By default, users joining a channel are automatically subscribed to the audio streams of all remote users. However, in scenarios with many users streaming simultaneously, such as large meetings, this consumes significant downstream bandwidth and system resources. To alleviate the bandwidth pressure and reduce system resource consumption on the receiving end, <Vg k ="COMPANY"/> offers audio-strength stream selection based on volume.

<PlatformWrapper notAllowed="unreal, blueprint, react-js">

## Understand the tech

After you enable audio-strength stream selection, Agora selects `N` audio streams with the highest volume, according to an audio-strength algorithm, and transmits them to the receiving end. The default value of `N` is 3 streams but you can customize it according to your requirements.
To accommodate diverse business scenarios, <Vg k ="COMPANY"/> also provides the streaming end with the option to choose whether its published audio stream participates in audio-strength stream selection or is directly transmitted to the receiving end.

## Prerequisites

Ensure that you have implemented the [SDK quickstart](../../video-calling/get-started/get-started-sdk) in your project.

<Admonition type="info" title="Information">
To enable this feature, contact [technical support](mailto:support@agora.io).
</Admonition>

## Implement stream selection

This section shows you how to implement audio-strength stream selection in common business scenarios.

### Enable audio-strength stream selection

When audio-strength stream selection is enabled, all audio streams in the channel participate in the selection by default. <Vg k ="COMPANY"/> transmits the top 3 audio streams with the highest volume to the receiving end. To customize the number of transmitted audio streams `N`, contact [technical support](mailto:support@agora.io). In scenarios with dozens or even hundreds of simultaneous streams, enabling audio-strength stream selection helps reduce pressure on the downstream bandwidth resource at the receiving end.

### Customize participation

<ProjectImplement/>

### Use case example

In a large enterprise meeting, with audio-strength stream selection enabled, certain key participants may designate their published audio streams to be excluded from selection based on audio strength. Instead, these streams are transmitted directly to the receiving end.

If there are 500 participants in the channel, and `N` is set to 4, it means that <Vg k ="COMPANY"/> transmits the top 4 audio streams with the highest volume to the receiving end. However, the main speaker A, representing headquarters, and other branch representatives B, C, D set their published audio streams not to participate in the selection. This setting reduces stuttering when playing audio streams at the receiving end. It also ensures that key users have their audio streams received by the audience, even if their speaking volume is low.

![Use Case Scenario](/images/video-sdk/use-case-scenario.png)

## Reference

This section provides integration considerations for the audio-strength stream selection feature.

<Reference/>

</PlatformWrapper>
<PlatformWrapper platform="unreal, blueprint, react-js">
<NotAvailable />
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@

<PlatformWrapper platform="android">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `updateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<PlatformWrapper platform="blueprint">
This feature is not available for Unreal blueprint.
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@

<PlatformWrapper platform="electron">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `updateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).


</PlatformWrapper>
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@

<PlatformWrapper platform="flutter">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `updateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import Unity from './unity.mdx'
import Unreal from './unreal.mdx'
import Web from './web.mdx';
import Windows from './windows.mdx'
import Blueprint from './blueprint.mdx'


<Android />
Expand All @@ -22,3 +23,4 @@ import Windows from './windows.mdx'
<Unreal />
<Web />
<Windows />
<Blueprint/>
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
<PlatformWrapper platform="ios">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel`, set `isAudioFilterable` to `false` in `AgoraRtcChannelMediaOptions` and pass the object in the `mediaOptions` parameter.

- After joining a channel, call `updateChannel` and set `isAudioFilterable` to `false` in `AgoraRtcChannelMediaOptions` and pass the object in the `mediaOptions` parameter.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
<PlatformWrapper platform="macos">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannelByToken` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `AgoraRtcChannelMediaOptions` and pass the object in the `mediaOptions` parameter.

- After joining a channel, call `updateChannelWithMediaOptions` and set `isAudioFilterable` to `false` in the `AgoraRtcChannelMediaOptions` and pass the object in the `mediaOptions` parameter.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
<PlatformWrapper platform="react-js">


This feature is not available for ReactJS.
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
<PlatformWrapper platform="react-native">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `updateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@

<PlatformWrapper platform="unity">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `JoinChannel` or `JoinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `UpdateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@

<PlatformWrapper platform="unreal">



This feature is not available for Unreal.
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<PlatformWrapper platform="web">

</PlatformWrapper>
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, contact [technical support](mailto:support@agora.io).
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
<PlatformWrapper platform="windows">
To exclude a user from audio-strength stream selection and ensure that their stream is transmitted to the receiving end regardless of its volume, use one of the following methods:

- When calling `joinChannel` or `joinChannelWithUserAccount` to join a channel, set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

- After joining a channel, call `updateChannelMediaOptions` and set `isAudioFilterable` to `false` in `ChannelMediaOptions` and pass the object in the `options` argument.

To receive all audio streams for a particular user, without any filtering by the audio-strength stream selection feature, pull all subscribed remote audio streams. To achieve this, you call the `setParameters` method with suitable JSON options. For details, contact [technical support](mailto:support@agora.io).

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@

<PlatformWrapper platform="android">

- <Link to="{{global.API_REF_ANDROID_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannel2">joinChannel</Link>
- <Link to="{{global.API_REF_ANDROID_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount">joinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_ANDROID_ROOT}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">updateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_ANDROID_ROOT}}/class_irtcengine.html#api_irtcengine_setparameters">setParameters</Link>

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@

<PlatformWrapper platform="electron">


- <Link to="{{global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannel2">joinChannel</Link>
- <Link to="{{global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount2">joinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">updateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_setparameters">setParameters</Link>

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@

<PlatformWrapper platform="flutter">

- <Link to="{{global.API_REF_FLUTTER_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannel2">joinChannel</Link>
- <Link to="{{global.API_REF_FLUTTER_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount2">joinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_FLUTTER_ROOT}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">updateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_FLUTTER_ROOT}}/class_irtcengine.html#api_irtcengine_setparameters">setParameters</Link>

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<PlatformWrapper platform="ios">



- <Link to="{{Global.API_REF_IOS_ROOT_RTC_KIT}}/agorartcenginekit/updatechannel(with:)">updateChannel</Link>
- <Link to="{{Global.API_REF_IOS_ROOT_RTC_KIT}}/agorartcenginekit/joinchannel(bytoken:channelid:useraccount:mediaoptions:joinsuccess:)">joinChannel</Link>
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<PlatformWrapper platform="macos">


- <Link to="{{Global.API_REF_MACOS_ROOT_RTC_KIT}}/agorartcenginekit/updatechannel(with:)">updateChannel</Link>
- <Link to="{{Global.API_REF_MACOS_ROOT_RTC_KIT}}/agorartcenginekit/joinchannel(bytoken:channelid:useraccount:mediaoptions:joinsuccess:)">joinChannel</Link>
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
<PlatformWrapper platform="react-native">

<PlatformWrapper platform="react-native">

- <Link to="{{global.API_REF_RN_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannel2">joinChannel</Link>
- <Link to="{{global.API_REF_RN_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount2">joinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_RN_ROOT}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">updateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_RN_ROOT}}/class_irtcengine.html#api_irtcengine_setparameters">setParameters</Link>

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@

<PlatformWrapper platform="unity">


- <Link to="{{global.API_REF_UNITY_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannel2">JoinChannel</Link>
- <Link to="{{global.API_REF_UNITY_ROOT}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount">JoinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_UNITY_ROOT}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">UpdateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_UNITY_ROOT}}/class_irtcengine.html#api_irtcengine_setparameters">SetParameters</Link>

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
<PlatformWrapper platform="windows">


- <Link to="{{global.API_REF_WINDOWS_ROOT_CPP}}/class_irtcengine.html#api_irtcengine_joinchannel2">joinChannel</Link>
- <Link to="{{global.API_REF_WINDOWS_ROOT_CPP}}/class_irtcengine.html#api_irtcengine_joinchannelwithuseraccount">joinChannelWithUserAccount</Link>
- <Link to="{{global.API_REF_WINDOWS_ROOT_CPP}}/class_irtcengine.html#api_irtcengine_updatechannelmediaoptions">updateChannelMediaOptions</Link>
- <Link to="{{global.API_REF_WINDOWS_ROOT_CPP}}/class_irtcengine.html#api_irtcengine_setparameters">setParameters</Link>
</PlatformWrapper>

0 comments on commit 22dd441

Please sign in to comment.