Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Merged by Bors] - gltf: load textures asynchronously using io task pool #1767

Closed
wants to merge 2 commits into from
Closed

[Merged by Bors] - gltf: load textures asynchronously using io task pool #1767

wants to merge 2 commits into from

Conversation

mockersf
Copy link
Member

While trying to reduce load time of gltf files, I noticed most of the loading time is spent transforming bytes into an actual texture.

This PR add asynchronously loading for them using io task pool in gltf loader. It reduces loading of a large glb file from 15 seconds to 6~8 on my laptop

To allow asynchronous tasks in an asset loader, I added a reference to the task pool from the asset server in the load context, which I can use later in the loader.

@alice-i-cecile alice-i-cecile added A-Assets Load files from disk to use for things like images, models, and sounds C-Performance A change motivated by improving speed, memory usage or compile times labels Mar 26, 2021
@mockersf
Copy link
Member Author

not sure how to make that works on wasm...

@mockersf mockersf force-pushed the gltf-async-textures branch from 27552c2 to f3c2724 Compare March 27, 2021 00:10
@mockersf
Copy link
Member Author

I sidestepped the wasm multithread issue (aka no multithread in wasm) by ... staying single threaded in wasm and choosing which to execute with a cfg

@FrankenApps
Copy link

I don't know if it's of any help, but did you already read this?

@mockersf mockersf force-pushed the gltf-async-textures branch from ef397e5 to 66b4849 Compare April 14, 2021 01:00
@mockersf
Copy link
Member Author

on my laptop, loading FlightHelmet.gltf
On main, it takes 4.5 seconds:

Apr 14 01:10:21.639  WARN bevy_gltf::loader: start loading textures
Apr 14 01:10:21.639  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:21.972  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:22.314  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:22.639  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:22.952  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:23.293  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:23.612  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:23.944  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:24.285  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:24.618  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:24.947  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:25.291  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:25.621  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:25.622  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:25.915  WARN bevy_gltf::loader: -> texture!
Apr 14 01:10:26.224  WARN bevy_gltf::loader: done loading textures

With this PR, 2.7 seconds:

Apr 14 01:15:15.142  WARN bevy_gltf::loader: start loading textures
Apr 14 01:15:15.142  WARN bevy_gltf::loader: texture!
Apr 14 01:15:15.143  WARN bevy_gltf::loader: texture!
Apr 14 01:15:15.483  WARN bevy_gltf::loader: texture!
Apr 14 01:15:15.492  WARN bevy_gltf::loader: texture!
Apr 14 01:15:15.815  WARN bevy_gltf::loader: texture!
Apr 14 01:15:15.824  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.155  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.156  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.478  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.501  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.502  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.803  WARN bevy_gltf::loader: texture!
Apr 14 01:15:16.822  WARN bevy_gltf::loader: texture!
Apr 14 01:15:17.160  WARN bevy_gltf::loader: texture!
Apr 14 01:15:17.516  WARN bevy_gltf::loader: texture!
Apr 14 01:15:17.856  WARN bevy_gltf::loader: done loading textures

We can see that two threads are doing the work.

@mockersf
Copy link
Member Author

why this can't work in wasm and need the workaround of not using the task pool: #1924

@cart
Copy link
Member

cart commented Apr 30, 2021

Cool this seems like the right approach for now. I added a quick TODO to remind us to unify impls when we can.

@cart
Copy link
Member

cart commented Apr 30, 2021

bors r+

bors bot pushed a commit that referenced this pull request Apr 30, 2021
@mockersf @cart
gltf: load textures asynchronously using io task pool (#1767)
c9b33e1 
While trying to reduce load time of gltf files, I noticed most of the loading time is spent transforming bytes into an actual texture.

This PR add asynchronously loading for them using io task pool in gltf loader. It reduces loading of a large glb file from 15 seconds to 6~8 on my laptop

To allow asynchronous tasks in an asset loader, I added a reference to the task pool from the asset server in the load context, which I can use later in the loader.

Co-authored-by: Carter Anderson <mcanders1@gmail.com>
@bors
Copy link
Contributor

bors bot commented Apr 30, 2021

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title gltf: load textures asynchronously using io task pool [Merged by Bors] - gltf: load textures asynchronously using io task pool Apr 30, 2021
@bors bors bot closed this Apr 30, 2021
ostwilkens pushed a commit to ostwilkens/bevy that referenced this pull request Jul 27, 2021
@mockersf @cart @ostwilkens
gltf: load textures asynchronously using io task pool (bevyengine#1767)
ba78fbb 
While trying to reduce load time of gltf files, I noticed most of the loading time is spent transforming bytes into an actual texture.

This PR add asynchronously loading for them using io task pool in gltf loader. It reduces loading of a large glb file from 15 seconds to 6~8 on my laptop

To allow asynchronous tasks in an asset loader, I added a reference to the task pool from the asset server in the load context, which I can use later in the loader.

Co-authored-by: Carter Anderson <mcanders1@gmail.com>
vabrador pushed a commit to vabrador/bevy that referenced this pull request Sep 15, 2021
@mockersf @cart @vabrador
gltf: load textures asynchronously using io task pool (bevyengine#1767)
c435418 
While trying to reduce load time of gltf files, I noticed most of the loading time is spent transforming bytes into an actual texture.

This PR add asynchronously loading for them using io task pool in gltf loader. It reduces loading of a large glb file from 15 seconds to 6~8 on my laptop

To allow asynchronous tasks in an asset loader, I added a reference to the task pool from the asset server in the load context, which I can use later in the loader.

Co-authored-by: Carter Anderson <mcanders1@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
A-Assets Load files from disk to use for things like images, models, and sounds C-Performance A change motivated by improving speed, memory usage or compile times
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mockersf @FrankenApps @cart @alice-i-cecile