Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ActiveSpeakerObserver: fix memory leak and more improvements #942

Merged
merged 6 commits into from
Nov 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
* Transport: Remove duplicate call to method (PR #931).
* RTCP: Adjust maximum compound packet size (PR #934).
* Fix `bufferedAmount` type to be a number again (PR #936).
* `ActiveSpeakerObserver`: Fix memory leak (PR #942).


### 3.10.12
Expand Down
43 changes: 25 additions & 18 deletions worker/include/RTC/ActiveSpeakerObserver.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,9 @@
#include <vector>

// Implementation of Dominant Speaker Identification for Multipoint
// Videoconferencing by Ilana Volfin and Israel Cohen. This
// implementation uses the RTP Audio Level extension from RFC-6464
// for the input signal.
// This has been ported from DominantSpeakerIdentification.java in Jitsi.
// Videoconferencing by Ilana Volfin and Israel Cohen. This implementation uses
// the RTP Audio Level extension from RFC-6464 for the input signal. This has
// been ported from DominantSpeakerIdentification.java in Jitsi:
// https://github.com/jitsi/jitsi-utils/blob/master/src/main/java/org/jitsi/utils/dsi/DominantSpeakerIdentification.java
namespace RTC
{
Expand All @@ -24,7 +23,7 @@ namespace RTC
public:
Speaker();
void EvalActivityScores();
double GetActivityScore(int32_t interval);
double GetActivityScore(uint8_t interval);
void LevelChanged(uint32_t level, uint64_t now);
void LevelTimedOut(uint64_t now);

Expand All @@ -39,28 +38,36 @@ namespace RTC

public:
bool paused{ false };
double immediateActivityScore;
double mediumActivityScore;
double longActivityScore;
double immediateActivityScore{ 0 };
double mediumActivityScore{ 0 };
double longActivityScore{ 0 };
uint64_t lastLevelChangeTime{ 0 };

private:
uint8_t minLevel;
uint8_t nextMinLevel;
uint32_t nextMinLevelWindowLen{ 0 };
uint8_t minLevel{ 0u };
uint8_t nextMinLevel{ 0u };
uint32_t nextMinLevelWindowLen{ 0u };
std::vector<uint8_t> immediates;
std::vector<uint8_t> mediums;
std::vector<uint8_t> longs;
std::vector<uint8_t> levels;
size_t nextLevelIndex;
size_t nextLevelIndex{ 0u };
};

struct ProducerSpeaker
class ProducerSpeaker
{
public:
explicit ProducerSpeaker(RTC::Producer* producer);
~ProducerSpeaker();

public:
RTC::Producer* producer;
Speaker* speaker;
};

private:
static const size_t RelativeSpeachActivitiesLen{ 3u };

public:
ActiveSpeakerObserver(const std::string& id, RTC::RtpObserver::Listener* listener, json& data);
~ActiveSpeakerObserver() override;
Expand All @@ -84,13 +91,13 @@ namespace RTC
void OnTimer(Timer* timer) override;

private:
static constexpr int relativeSpeachActivitiesLen{ 3 };
double relativeSpeachActivities[relativeSpeachActivitiesLen];
std::string dominantId{ "" };
double relativeSpeachActivities[RelativeSpeachActivitiesLen];
std::string dominantId;
Timer* periodicTimer{ nullptr };
uint16_t interval{ 300u };
absl::flat_hash_map<std::string, struct ProducerSpeaker> mapProducerSpeaker;
uint64_t lastLevelIdleTime{ 0 };
// Map of ProducerSpeakers indexed by Producer id.
absl::flat_hash_map<std::string, ProducerSpeaker*> mapProducerSpeakers;
uint64_t lastLevelIdleTime{ 0u };
};
} // namespace RTC

Expand Down
Loading