Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch interface #1778

Open
shyim opened this issue Mar 26, 2024 · 2 comments
Open

Batch interface #1778

shyim opened this issue Mar 26, 2024 · 2 comments
Labels

Comments

@shyim
Copy link
Contributor

shyim commented Mar 26, 2024

Feature Request

Scenario / Use-case

I want to copy 8k files to an external FS. This is right now pretty unuseable with Flysystem as the interface only allows one copy at once. So I cannot take any advantage of the real adapter, like async copying at once and so on.

My suggestion so far would be to have a interface: BatchFilesystemOperator which introduces new methods like:

writeBatch and takes a list of files and the adapter is able to write these files in a automatic.

when the adapter does not have this interface, we can just call the regular write in a loop.

interface BatchFilesystemOperator
{
    /**
     * @param array<string, resource> $files
     */
    public function writeBatch(array $files);
}
/**
     * @param array<string, resource> $files
     */
    public function writeBatch(array $files): void
    {
        if ($this->adapter instanceof BatchFilesystemOperator) {
            $this->adapter->writeBatch($files);
            return;
        }

        foreach ($files as $location => $contents) {
            $this->writeStream($location, $contents);
        }
    }

In the example AsyncAWS s3 adapter, we can implement this method and make use of async requests to make this faster for this adapter.

@frankdejonge frankdejonge added the V4 label Apr 7, 2024
@frankdejonge
Copy link
Member

Sounds like a reasonable addition, I've labelled it V4. I'll create a 4.x branch some time this week to start tracking it.

@shyim
Copy link
Contributor Author

shyim commented Apr 7, 2024

Let me know if I can help you in some way :). We did an ugly hack right now, would like to get rid of it 👍
https://github.com/shopware/shopware/blob/trunk/src/Core/Framework/Adapter/Filesystem/Plugin/CopyBatch.php#L72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants