Added Method to Allow Pipelined Asset Loading

[bushrat011899]

Objective

• Fixes Allow asset loaders to call other asset loaders - the "gzip" problem #10518

Solution

I've added a method to LoadContext, load_direct_with_reader, which mirrors the behaviour of load_direct with a single key difference: it is provided with the Reader by the caller, rather than getting it from the contained AssetServer. This allows for an AssetLoader to process its Reader stream, and then directly hand the results off to the LoadContext to handle further loading. The outer AssetLoader can control how the Reader is interpreted by providing a relevant AssetPath.

For example, a Gzip decompression loader could process the asset images/my_image.png.gz by decompressing the bytes, then handing the decompressed result to the LoadContext with the new path images/my_image.png.gz/my_image.png. This intuitively reflects the nature of contained assets, whilst avoiding unintended behaviour, since the generated path cannot be a real file path (a file and folder of the same name cannot coexist in most file-systems).

#[derive(Asset, TypePath)]
pub struct GzAsset {
    pub uncompressed: ErasedLoadedAsset,
}

#[derive(Default)]
pub struct GzAssetLoader;

impl AssetLoader for GzAssetLoader {
    type Asset = GzAsset;
    type Settings = ();
    type Error = GzAssetLoaderError;
    fn load<'a>(
        &'a self,
        reader: &'a mut Reader,
        _settings: &'a (),
        load_context: &'a mut LoadContext,
    ) -> BoxedFuture<'a, Result<Self::Asset, Self::Error>> {
        Box::pin(async move {
            let compressed_path = load_context.path();
            let file_name = compressed_path
                .file_name()
                .ok_or(GzAssetLoaderError::IndeterminateFilePath)?
                .to_string_lossy();
            let uncompressed_file_name = file_name
                .strip_suffix(".gz")
                .ok_or(GzAssetLoaderError::IndeterminateFilePath)?;
            let contained_path = compressed_path.join(uncompressed_file_name);

            let mut bytes_compressed = Vec::new();

            reader.read_to_end(&mut bytes_compressed).await?;

            let mut decoder = GzDecoder::new(bytes_compressed.as_slice());

            let mut bytes_uncompressed = Vec::new();

            decoder.read_to_end(&mut bytes_uncompressed)?;

            // Now that we have decompressed the asset, let's pass it back to the
            // context to continue loading

            let mut reader = VecReader::new(bytes_uncompressed);

            let uncompressed = load_context
                .load_direct_with_reader(&mut reader, contained_path)
                .await?;

            Ok(GzAsset { uncompressed })
        })
    }

    fn extensions(&self) -> &[&str] {
        &["gz"]
    }
}

Because this example is so prudent, I've included an asset_decompression example which implements this exact behaviour:

fn main() {
    App::new()
        .add_plugins(DefaultPlugins)
        .init_asset::<GzAsset>()
        .init_asset_loader::<GzAssetLoader>()
        .add_systems(Startup, setup)
        .add_systems(Update, decompress::<Image>)
        .run();
}

fn setup(mut commands: Commands, asset_server: Res<AssetServer>) {
    commands.spawn(Camera2dBundle::default());

    commands.spawn((
        Compressed::<Image> {
            compressed: asset_server.load("data/compressed_image.png.gz"),
            ..default()
        },
        Sprite::default(),
        TransformBundle::default(),
        VisibilityBundle::default(),
    ));
}

fn decompress<A: Asset>(
    mut commands: Commands,
    asset_server: Res<AssetServer>,
    mut compressed_assets: ResMut<Assets<GzAsset>>,
    query: Query<(Entity, &Compressed<A>)>,
) {
    for (entity, Compressed { compressed, .. }) in query.iter() {
        let Some(GzAsset { uncompressed }) = compressed_assets.remove(compressed) else {
            continue;
        };

        let uncompressed = uncompressed.take::<A>().unwrap();

        commands
            .entity(entity)
            .remove::<Compressed<A>>()
            .insert(asset_server.add(uncompressed));
    }
}

A key limitation to this design is how to type the internally loaded asset, since the example GzAssetLoader is unaware of the internal asset type A. As such, in this example I store the contained asset as an ErasedLoadedAsset, and leave it up to the consumer of the GzAsset to handle typing the final result, which is the purpose of the decompress system. This limitation can be worked around by providing type information to the GzAssetLoader, such as GzAssetLoader<Image, ImageAssetLoader>, but this would require registering the asset loader for every possible decompression target.

Aside from this limitation, nested asset containerisation works as an end user would expect; if the user registers a TarAssetLoader, and a GzAssetLoader, then they can load assets with compound containerisation, such as images.tar.gz.

---

Changelog

• Added LoadContext::load_direct_with_reader
• Added asset_decompression example

Notes

• While I believe my implementation of a Gzip asset loader is reasonable, I haven't included it as a public feature of bevy_asset to keep the scope of this PR as focussed as possible.
• I have included flate2 as a dev-dependency for the example; it is not included in the main dependency graph.

[nicopap]

Looks good to me, I really want the load_with_reader and load_with_reader_and_settigns methods though. I'll defer to other people wrt to the necessity of a decompression example.

[nicopap]

Also I think from_reader makes more sense than with_reader, but I don't think it's a big deal.

[bushrat011899]

Also I think from_reader makes more sense than with_reader, but I don't think it's a big deal.

I've chosen with_reader to match with_settings, which I think looks cleaner, but I'm also perfectly happy to rename these methods if anyone else has a stronger opinion on the matter.

[alice-i-cecile]

Useful API extension. I particularly like the example: assets badly needed more.

That error type is enterprise-grade :D