-
Notifications
You must be signed in to change notification settings - Fork 6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Impeller] Maintain separate queues of GLES operations for each thread in the reactor #56573
Conversation
…d in the reactor Threads that add operations to the ReactorGLES assume that those operations will be executed serially. But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa). The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread. This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions. This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned. Fixes flutter/flutter#158535 Fixes flutter/flutter#158388
It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group. |
We have a reactor_gles_unittest setup. I think this could be tested by enqueuing operations from two different threads, reacting on one, and verifying the other threads operation was not invoked |
@@ -381,6 +381,7 @@ ImageDecoderImpeller::UnsafeUploadTextureToPrivate( | |||
FML_DLOG(ERROR) << decode_error; | |||
return std::make_pair(nullptr, decode_error); | |||
} | |||
command_buffer->WaitUntilScheduled(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is a glFlush required here? Can you add a comment explaining why it's required. We aren't doing this elsewhere and it is going to add a performance hit to the decoder on opengles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WaitUntilScheduled
(which wraps glFlush
) is required here. Without this, there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.
Added a comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no assurance that the GPU will launch execution of the IO thread command buffer before the raster thread tries to access the texture.
If that's the case, are we sure we don't need a glFinish
then? The glFlush
is just making sure the commands are submitted to the GPU, but glFinish
would make it synchronize on them actually completing. With a glFlush
couldn't the raster thread potentially be using the texture before the data is finished being uploaded? Or maybe the driver is synchronizing on it if it isn't finished yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
glFinish
is expensive - I can see a major increase in latency when scrolling quickly through the Wonderous home screen pages on a Pixel 3. It takes several seconds for the app to catch up and render all pending images if each image decode does a glFinish
.
However, IIUC the spec does not guarantee that a glFlush
on the IO thread will ensure that the texture is safely usable when the raster thread consumes it on all GLES implementations.
Renamed CommandBuffer::WaitUntilScheduled
to CommandBuffer::WaitUntilCompleted
and changed it to call glFinish
.
We should look into having GLES textures use glFenceSync
/glWaitSync
(flutter/flutter#158963)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Notes from the discord channel:
- https://www.khronos.org/webgl/wiki/SharedResouces says glFinish is required
- skia source uses glFinish on opengles2 and glFenceSync on opengles3
mutable Mutex ops_mutex_; | ||
std::vector<Operation> ops_ IPLR_GUARDED_BY(ops_mutex_); | ||
std::map<std::thread::id, std::vector<Operation>> ops_ IPLR_GUARDED_BY( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to instead key this with EGLContext
? It sounded like Chinmay and Jonah had use cases in mind that revolved around passing the context between multiple threads in a worker pool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently EGL contexts are not passed between threads.
ReactorGLES does support calling AddOperation
on a thread that does not have an active EGLContext
. Using the thread ID as a key allows queueing of operations when the EGLContext
is unavailable.
@@ -105,8 +105,10 @@ bool ReactorGLES::RemoveWorker(WorkerID worker) { | |||
} | |||
|
|||
bool ReactorGLES::HasPendingOperations() const { | |||
auto thread_id = std::this_thread::get_id(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note for future archeology and AIs: I was concerned about the overhead of this call, worrying it would require a syscall. I was wrong, Jason showed me that isn't the case. Pthread is keeping that info in user space.
I tested this patch on upstream/main too, this fixes flutter/flutter#158390. I'll add it to the description. |
Added a test - PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything looks good to me, thanks for adding a test. I just have one open question about wether we want a glFlush or glFinish in this case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks lgtm
I forgot to mention, this is big. Thanks Jason for wading through this. |
This assert would have blocked the following bug: flutter/flutter#158535 This shouldn't be landed until #56573 lands. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [ ] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…each thread in the reactor (flutter/engine#56573)
…159015) Roll Flutter Engine from 619804c0fbb7 to f649330affa8 (47 revisions) flutter/engine@619804c...f649330 2024-11-16 skia-flutter-autoroll@skia.org Roll Skia from 1ef3b910e064 to 6b0f264bde33 (6 revisions) (flutter/engine#56635) 2024-11-16 zanderso@users.noreply.github.com Revert "Remove `android_jit_release_x86`." (flutter/engine#56634) 2024-11-15 chris@bracken.jp Move SetRenderTargetType to EmbedderTestCompositor (flutter/engine#56626) 2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from b62dadb14d82 to eccb15756020 (2 revisions) (flutter/engine#56630) 2024-11-15 andre.stein.1985@gmail.com [TextInput] Add TextInputType.webSearch (#15762) (flutter/engine#56428) 2024-11-15 mdebbar@google.com [web] Send the correct view ID with semantics actions (flutter/engine#56595) 2024-11-15 41930132+hellohuanlin@users.noreply.github.com [macos] early return to suppress clang-tidy warning (flutter/engine#56588) 2024-11-15 chinmaygarde@google.com [Impeller] libImpeller: Tinker on the README to cross reference the SDK and the example. (flutter/engine#56623) 2024-11-15 chaopeng@google.com [fuchsia] MaybeRunInitialVsyncCallback should only called once (flutter/engine#56429) 2024-11-15 magder@google.com Move mac clangd build from x64 to arm (flutter/engine#56567) 2024-11-15 30870216+gaaclarke@users.noreply.github.com Added assert for opengles thread safety (flutter/engine#56585) 2024-11-15 jonahwilliams@google.com [Impeller] enable OpenGL fallback on Android. (flutter/engine#56591) 2024-11-15 jason-simmons@users.noreply.github.com [Impeller] Maintain separate queues of GLES operations for each thread in the reactor (flutter/engine#56573) 2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from abe86ca4eb01 to b62dadb14d82 (2 revisions) (flutter/engine#56615) 2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from b730eb340852 to 1ef3b910e064 (4 revisions) (flutter/engine#56613) 2024-11-15 skia-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from JkpuAsLzcmYLzf1iX... to UpSQzyXGUhMfedYIh... (flutter/engine#56611) 2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from bf0eeb9d929c to abe86ca4eb01 (1 revision) (flutter/engine#56610) 2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from e5fda8472b21 to b730eb340852 (1 revision) (flutter/engine#56608) 2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from 8ff65da4b8bf to e5fda8472b21 (1 revision) (flutter/engine#56603) 2024-11-15 chris@bracken.jp Move TestMetalContext/Surface to testing namespace (flutter/engine#56602) 2024-11-15 skia-flutter-autoroll@skia.org Roll Dart SDK from 343b9fc6eafe to bf0eeb9d929c (1 revision) (flutter/engine#56604) 2024-11-15 chinmaygarde@google.com [Impeller] Don't enable KHR_debug on Angle. (flutter/engine#56601) 2024-11-15 chinmaygarde@google.com [Impeller] libImpeller: Reset the GL state when transitioning control back to the embedder. (flutter/engine#56597) 2024-11-15 skia-flutter-autoroll@skia.org Roll Skia from 8946ea477b42 to 8ff65da4b8bf (2 revisions) (flutter/engine#56600) 2024-11-14 chinmaygarde@google.com [Impeller] Put test components in the testing namespace. (flutter/engine#56594) 2024-11-14 chinmaygarde@google.com [Impeller] libImpeller: Add a warning about OpenGL state being trampled. (flutter/engine#56599) 2024-11-14 chris@bracken.jp Embedder: Refactor EmbedderConfigBuilder by backend (flutter/engine#56598) 2024-11-14 chris@bracken.jp Migrate Metal translation units to Obj-C++ (flutter/engine#56592) 2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from db4d7d52db48 to 343b9fc6eafe (1 revision) (flutter/engine#56590) 2024-11-14 chris@bracken.jp iOS,macOS: Build/test test_metal_surface_unittests (flutter/engine#56586) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 3222456e63dc to 8946ea477b42 (2 revisions) (flutter/engine#56587) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 16178bf63de7 to 3222456e63dc (3 revisions) (flutter/engine#56584) 2024-11-14 bruno.leroux@gmail.com [Android] Fix Slider semantics double tap behaviors (flutter/engine#56452) 2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from a3b6652100c8 to db4d7d52db48 (1 revision) (flutter/engine#56581) 2024-11-14 chris@bracken.jp Move Vulkan context to EmbedderTestContextVulkan (flutter/engine#56571) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from a076435073fc to 16178bf63de7 (1 revision) (flutter/engine#56579) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 70553189e4e8 to a076435073fc (1 revision) (flutter/engine#56577) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from fa52f2c1ddba to 70553189e4e8 (1 revision) (flutter/engine#56576) 2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from 3f05b3540cb2 to a3b6652100c8 (3 revisions) (flutter/engine#56575) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from ddbd8d1cb8d8 to fa52f2c1ddba (3 revisions) (flutter/engine#56574) 2024-11-14 jonahwilliams@google.com [Android] choose 24 bit depth buffer. (flutter/engine#56553) 2024-11-14 skia-flutter-autoroll@skia.org Roll Skia from 3750b8939c7f to ddbd8d1cb8d8 (1 revision) (flutter/engine#56562) 2024-11-14 robert.ancell@canonical.com Move FlMouseCursorHandler from FlView to FlEngine (flutter/engine#56026) 2024-11-14 jonahwilliams@google.com [Impeller] force software resize usage for GLES backend. (flutter/engine#56511) 2024-11-14 skia-flutter-autoroll@skia.org Roll Dart SDK from 66ab1774bfe9 to 3f05b3540cb2 (2 revisions) (flutter/engine#56564) 2024-11-14 chris@bracken.jp Embedder: Refactor EmbedderTestContext creation (flutter/engine#56563) ...
depends on #56573 fixes flutter/flutter#158523 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
Threads that add operations to the ReactorGLES assume that those operations will be executed serially.
But prior to this change, the ReactorGLES added all operations into one queue. The reactor would then execute those operations on any thread that can react. This could cause operations that were added to the reactor on the raster thread to be submitted to the GPU on the IO thread (or vice versa).
The reactor does not wait for the GPU to finish execution of those operations. So other operations added on the raster thread could be submitted by a reaction before the GPU has completed the operation that was submitted on the IO thread.
This PR ensures that operations added to the reactor on a given thread will be executed during a reaction on that same thread. If the thread can not currently react, then the operations will be queued until the thread enables reactions.
This also adds a call to CommandBuffer::WaitUntilScheduled to ImageDecoderImpeller. This ensures that the command buffer submitted on the IO thread is flushed before the image is returned.
Fixes flutter/flutter#158535
Fixes flutter/flutter#158388
Fixes flutter/flutter#158390