Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Impeller] Maintain separate queues of GLES operations for each thread in the reactor #56573

Merged
merged 4 commits into from
Nov 15, 2024

Conversation

jason-simmons
Copy link
Member

@jason-simmons jason-simmons commented Nov 14, 2024

Threads that add operations to the ReactorGLES assume that those operations will be executed serially.

But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.

This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions.

This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.

Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
Fixes flutter/flutter#158390

…d in the reactor

Threads that add operations to the ReactorGLES assume that those operations will be executed serially.

But prior to this change, the ReactorGLES added all operations into one queue.  The reactor would then execute those operations on any thread that can react.
This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations.  So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.

This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread.  If the thread can not currently react, then the operations will be queued until the thread enables reactions.

This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller.  This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.

Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

@jonahwilliams
Copy link
Member

We have a reactor_gles_unittest setup. I think this could be tested by enqueuing operations from two different threads, reacting on one, and verifying the other threads operation was not invoked

@@ -381,6 +381,7 @@ ImageDecoderImpeller::UnsafeUploadTextureToPrivate(
FML_DLOG(ERROR) << decode_error;
return std::make_pair(nullptr, decode_error);
}
command_buffer->WaitUntilScheduled();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a glFlush required here? Can you add a comment explaining why it's required. We aren't doing this elsewhere and it is going to add a performance hit to the decoder on opengles.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WaitUntilScheduled (which wraps glFlush) is required here. Without this, there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.

Added a comment.

Copy link
Member

@gaaclarke gaaclarke Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.

If that's the case, are we sure we don't need a glFinish then? The glFlush is just making sure the commands are submitted to the GPU, but glFinish would make it synchronize on them actually completing. With a glFlush couldn't the raster thread potentially be using the texture before the data is finished being uploaded? Or maybe the driver is synchronizing on it if it isn't finished yet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glFinish is expensive - I can see a major increase in latency when scrolling quickly through the Wonderous home screen pages on a Pixel 3. It takes several seconds for the app to catch up and render all pending images if each image decode does a glFinish.

However, IIUC the spec does not guarantee that a glFlush on the IO thread will ensure that the texture is safely usable when the raster thread consumes it on all GLES implementations.

Renamed CommandBuffer::WaitUntilScheduled to CommandBuffer::WaitUntilCompleted and changed it to call glFinish.

We should look into having GLES textures use glFenceSync/glWaitSync (flutter/flutter#158963)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes from the discord channel:

mutable Mutex ops_mutex_;
std::vector<Operation> ops_ IPLR_GUARDED_BY(ops_mutex_);
std::map<std::thread::id, std::vector<Operation>> ops_ IPLR_GUARDED_BY(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to instead key this with EGLContext? It sounded like Chinmay and Jonah had use cases in mind that revolved around passing the context between multiple threads in a worker pool?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently EGL contexts are not passed between threads.

ReactorGLES does support calling AddOperation on a thread that does not have an active EGLContext. Using the thread ID as a key allows queueing of operations when the EGLContext is unavailable.

@@ -105,8 +105,10 @@ bool ReactorGLES::RemoveWorker(WorkerID worker) {
}

bool ReactorGLES::HasPendingOperations() const {
auto thread_id = std::this_thread::get_id();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for future archeology and AIs: I was concerned about the overhead of this call, worrying it would require a syscall. I was wrong, Jason showed me that isn't the case. Pthread is keeping that info in user space.

@gaaclarke
Copy link
Member

I tested this patch on upstream/main too, this fixes flutter/flutter#158390. I'll add it to the description.

@jason-simmons
Copy link
Member Author

Added a test - PTAL

Copy link
Member

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me, thanks for adding a test. I just have one open question about wether we want a glFlush or glFinish in this case.

Copy link
Member

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks lgtm

@gaaclarke
Copy link
Member

I forgot to mention, this is big. Thanks Jason for wading through this.

@jason-simmons jason-simmons added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024
@auto-submit auto-submit bot merged commit 644f85e into flutter:main Nov 15, 2024
34 checks passed
@auto-submit auto-submit bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 15, 2024
gaaclarke added a commit that referenced this pull request Nov 15, 2024
This assert would have blocked the following bug:
flutter/flutter#158535

This shouldn't be landed until
#56573 lands.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 15, 2024
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Nov 16, 2024
github-merge-queue bot pushed a commit to flutter/flutter that referenced this pull request Nov 16, 2024
…159015)

Roll Flutter Engine from 619804c0fbb7 to f649330affa8 (47 revisions)

flutter/engine@619804c...f649330

2024-11-16 skia-flutter-autoroll@skia.org Roll Skia from 1ef3b910e064 to
6b0f264bde33 (6 revisions) (flutter/engine#56635)
2024-11-16 zanderso@users.noreply.github.com Revert "Remove
`android_jit_release_x86`." (flutter/engine#56634)
2024-11-15 chris@bracken.jp Move SetRenderTargetType to
EmbedderTestCompositor (flutter/engine#56626)
2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from
b62dadb14d82 to eccb15756020 (2 revisions) (flutter/engine#56630)
2024-11-15 andre.stein.1985@gmail.com [TextInput] Add
TextInputType.webSearch (#15762) (flutter/engine#56428)
2024-11-15 mdebbar@google.com [web] Send the correct view ID with
semantics actions (flutter/engine#56595)
2024-11-15 41930132+hellohuanlin@users.noreply.github.com [macos] early
return to suppress clang-tidy warning (flutter/engine#56588)
2024-11-15 chinmaygarde@google.com [Impeller] libImpeller: Tinker on the
README to cross reference the SDK and the example.
(flutter/engine#56623)
2024-11-15 chaopeng@google.com [fuchsia] MaybeRunInitialVsyncCallback
should only called once (flutter/engine#56429)
2024-11-15 magder@google.com Move mac clangd build from x64 to arm
(flutter/engine#56567)
2024-11-15 30870216+gaaclarke@users.noreply.github.com Added assert for
opengles thread safety (flutter/engine#56585)
2024-11-15 jonahwilliams@google.com [Impeller] enable OpenGL fallback on
Android. (flutter/engine#56591)
2024-11-15 jason-simmons@users.noreply.github.com [Impeller] Maintain
separate queues of GLES operations for each thread in the reactor
(flutter/engine#56573)
2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from
abe86ca4eb01 to b62dadb14d82 (2 revisions) (flutter/engine#56615)
2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from b730eb340852 to
1ef3b910e064 (4 revisions) (flutter/engine#56613)
2024-11-15 skia-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from
JkpuAsLzcmYLzf1iX... to UpSQzyXGUhMfedYIh... (flutter/engine#56611)
2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from
bf0eeb9d929c to abe86ca4eb01 (1 revision) (flutter/engine#56610)
2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from e5fda8472b21 to
b730eb340852 (1 revision) (flutter/engine#56608)
2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from 8ff65da4b8bf to
e5fda8472b21 (1 revision) (flutter/engine#56603)
2024-11-15 chris@bracken.jp Move TestMetalContext/Surface to testing
namespace (flutter/engine#56602)
2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from
343b9fc6eafe to bf0eeb9d929c (1 revision) (flutter/engine#56604)
2024-11-15 chinmaygarde@google.com [Impeller] Don't enable KHR_debug on
Angle. (flutter/engine#56601)
2024-11-15 chinmaygarde@google.com [Impeller] libImpeller: Reset the GL
state when transitioning control back to the embedder.
(flutter/engine#56597)
2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from 8946ea477b42 to
8ff65da4b8bf (2 revisions) (flutter/engine#56600)
2024-11-14 chinmaygarde@google.com [Impeller] Put test components in the
testing namespace. (flutter/engine#56594)
2024-11-14 chinmaygarde@google.com [Impeller] libImpeller: Add a warning
about OpenGL state being trampled. (flutter/engine#56599)
2024-11-14 chris@bracken.jp Embedder: Refactor EmbedderConfigBuilder by
backend (flutter/engine#56598)
2024-11-14 chris@bracken.jp Migrate Metal translation units to Obj-C++
(flutter/engine#56592)
2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from
db4d7d52db48 to 343b9fc6eafe (1 revision) (flutter/engine#56590)
2024-11-14 chris@bracken.jp iOS,macOS: Build/test
test_metal_surface_unittests (flutter/engine#56586)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 3222456e63dc to
8946ea477b42 (2 revisions) (flutter/engine#56587)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 16178bf63de7 to
3222456e63dc (3 revisions) (flutter/engine#56584)
2024-11-14 bruno.leroux@gmail.com [Android] Fix Slider semantics double
tap behaviors (flutter/engine#56452)
2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from
a3b6652100c8 to db4d7d52db48 (1 revision) (flutter/engine#56581)
2024-11-14 chris@bracken.jp Move Vulkan context to
EmbedderTestContextVulkan (flutter/engine#56571)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from a076435073fc to
16178bf63de7 (1 revision) (flutter/engine#56579)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 70553189e4e8 to
a076435073fc (1 revision) (flutter/engine#56577)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from fa52f2c1ddba to
70553189e4e8 (1 revision) (flutter/engine#56576)
2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from
3f05b3540cb2 to a3b6652100c8 (3 revisions) (flutter/engine#56575)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from ddbd8d1cb8d8 to
fa52f2c1ddba (3 revisions) (flutter/engine#56574)
2024-11-14 jonahwilliams@google.com [Android] choose 24 bit depth
buffer. (flutter/engine#56553)
2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 3750b8939c7f to
ddbd8d1cb8d8 (1 revision) (flutter/engine#56562)
2024-11-14 robert.ancell@canonical.com Move FlMouseCursorHandler from
FlView to FlEngine (flutter/engine#56026)
2024-11-14 jonahwilliams@google.com [Impeller] force software resize
usage for GLES backend. (flutter/engine#56511)
2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from
66ab1774bfe9 to 3f05b3540cb2 (2 revisions) (flutter/engine#56564)
2024-11-14 chris@bracken.jp Embedder: Refactor EmbedderTestContext
creation (flutter/engine#56563)
...
gaaclarke added a commit that referenced this pull request Nov 18, 2024
depends on #56573
fixes flutter/flutter#158523

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants