Skip to content

Commit

Permalink
Reduces rate at which TryGetNextFrame returns NULL for WGC
Browse files Browse the repository at this point in the history
This CL is a follow-up of work done in
https://webrtc-review.googlesource.com/c/src/+/323882 where the goal
was to reduce the amount of FrameDropped error logs in
WebRTC.DesktopCapture.Win.WgcCaptureSessionGetFrameResult.

The previous work avoids FrameDropped logs for a minimized window
being captured with WGC but we still se a large amount of these error
(or rather warning) logs. See [1] which comes from Canary.

This CL does two different things to improve the situation:

1) It adds kFramePoolEmpty to the existing
GetFrameResult::kFrameDropped enum to point out that the warning
comes from the frame pool not being able to return a valid new frame.
It also makes it more clear that it does not cause an outer/final
error as WgcCapturerResult::kFrameDropped. We still keep the inner
GetFrameResult::kFrameDropped but it is only produced when the frame
pool returns NULL and our external queue is empty. Hence, a real
frame-drop error. Note that, it is still easy to provoke
kFramePoolEmpty simply by asking for a high resolution at a high rate.
The example in [2] comes from a 4K screen @30fps. Hence, we have not
fixed anything yet.

2) It also increases the size of the internal frame pool from 1 to 2.
This does lead to an almost zero rate of kFramePoolEmpt
warnings at the expense of a slightly reduced max capture rate. BUT,
with 1 as size, we can "see" a higher max capture rate but it is not
a true rate since it comes with a high rate of kFramePoolEmpty
errors. Hence, we "emulate" a high capture rate by simply feeding
copies of the last frame that we had stored in the external queue.
Using 2 leads to a more "true" rate of what we actually can capture
in terms of *new* frames and also a substantially lower rate of
kFramePoolEmpty.
In addition, with 1 as size, if we ask at a too high rate and provide
a copy of the last frame, our CPU adaptation will not reduce its rate
since we think that things are OK when it is actually not.

Also, the samples in [3] and [4] both use 2 as numberOfBuffers
as well.

Let me also mention that with this small change, I a have not been
able to provoke any kFramePoolEmpty error messages.

Finally, geDisplayMedia can be called called with constraints where
min and max framerate is defined. The mechanism which maintains the
min rate is implemented via the RequestRefreshFrame API and it can
be sent to the source (DesktopCaptureDevice) back to back with a
previous timer interrupt for a capture request. Without this change,
these RRFs were also a source of a large amount of
kFramePoolEmpty error logs. With 2 buffers instead; this is no
longer the case.

[1] https://screenshot.googleplex.com/7sfv6HdGXLwyxdj
[2] https://paste.googleplex.com/4795680001359872
[3] https://github.com/robmikh/Win32CaptureSample/blob/master/Win32CaptureSample/SimpleCapture.cpp
[4] https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/screen-capture#add-the-screen-capture-capability

(cherry picked from commit 4be5927)

Bug: chromium:1314868
Change-Id: I73b823b31a993fd2cd6e007b212826dfe1a80012
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/325521
Commit-Queue: Alexander Cooper <alcooper@chromium.org>
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Cr-Original-Commit-Position: refs/heads/main@{#41079}
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/326960
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/branch-heads/6099@{#1}
Cr-Branched-From: 507f1cc-refs/heads/main@{#41042}
  • Loading branch information
henrikand authored and WebRTC LUCI CQ committed Nov 10, 2023
1 parent 507f1cc commit b0cc68e
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 7 deletions.
14 changes: 10 additions & 4 deletions modules/desktop_capture/win/wgc_capture_session.cc
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ enum class GetFrameResult {
kGetContentSizeFailed = 9,
kResizeMappedTextureFailed = 10,
kRecreateFramePoolFailed = 11,
kMaxValue = kRecreateFramePoolFailed
kFramePoolEmpty = 12,
kMaxValue = kFramePoolEmpty
};

void RecordStartCaptureResult(StartCaptureResult error) {
Expand Down Expand Up @@ -332,10 +333,15 @@ HRESULT WgcCaptureSession::ProcessFrame() {
}

if (!capture_frame) {
// Avoid logging errors until at least one valid frame has been captured.
if (queue_.current_frame()) {
RTC_DLOG(LS_WARNING) << "Frame pool was empty => kFrameDropped.";
if (!queue_.current_frame()) {
// The frame pool was empty and so is the external queue.
RTC_DLOG(LS_ERROR) << "Frame pool was empty => kFrameDropped.";
RecordGetFrameResult(GetFrameResult::kFrameDropped);
} else {
// The frame pool was empty but there is still one old frame available in
// external the queue.
RTC_DLOG(LS_WARNING) << "Frame pool was empty => kFramePoolEmpty.";
RecordGetFrameResult(GetFrameResult::kFramePoolEmpty);
}
return E_FAIL;
}
Expand Down
8 changes: 5 additions & 3 deletions modules/desktop_capture/win/wgc_capture_session.h
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,12 @@ class WgcCaptureSession final {
return is_capture_started_;
}

// We only keep 1 buffer in the internal frame pool to reduce the latency as
// much as possible.
// We keep 2 buffers in the frame pool since it results in a good compromise
// between latency/capture-rate and the rate at which
// Direct3D11CaptureFramePool.TryGetNextFrame returns NULL and we have to fall
// back to providing a copy from our external queue instead.
// We make this public for tests.
static constexpr int kNumBuffers = 1;
static constexpr int kNumBuffers = 2;

private:
// Initializes `mapped_texture_` with the properties of the `src_texture`,
Expand Down

0 comments on commit b0cc68e

Please sign in to comment.