You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm hitting an issue on Nextcloud when zipping files from S3. I've described the situation here: nextcloud/server#42248.
The problem here is that when the stream is a remote file, fread might return an empty string (even if it is a blocking stream, like in this case). This happened in my case, and the downloaded ZIP files contained truncated files.
while (!feof($stream) && $data = fread($stream, self::STREAM_CHUNK_SIZE)) {
...
}
So, if fread returns nothing (an empty string), then even if feof is false, indicating that it's not the end of the stream yet, it will still abort reading the file (S3 stream in this case).
If I change it like so:
while (!feof($stream)) {
$data = fread($stream, self::STREAM_CHUNK_SIZE);
...
}
Then my issue is fixed. The behavior was changed here: 22515e3.
One could argue that the underlying stream should be fixed/changed to block as long as necessary instead, but in my view, the library should either handle cases where fread returns nothing while feof is still false (by retrying), or return with an error to the caller, otherwise, as we see, it can silently produce truncated files. Another possible solution would be also passing the source file size (which is usually known by this time), and checking against that.
What are your thoughts about the issue, and my fix/workaround, and how do you think a proper fix (that could be merged) could be organized?
The text was updated successfully, but these errors were encountered:
Ziyann
changed the title
Premature stream termination when fread returns zero
Premature stream termination when fread returns nothing
Dec 16, 2023
This minimally invasive pull request fixes the issue: #20. @DeepDiver1975, what do you think about this approach?
Also, the TarStreamer library has the same behavior.
I'm hitting an issue on Nextcloud when zipping files from S3. I've described the situation here: nextcloud/server#42248.
The problem here is that when the stream is a remote file,
fread
might return an empty string (even if it is a blocking stream, like in this case). This happened in my case, and the downloaded ZIP files contained truncated files.So, if
fread
returns nothing (an empty string), then even iffeof
is false, indicating that it's not the end of the stream yet, it will still abort reading the file (S3 stream in this case).If I change it like so:
Then my issue is fixed. The behavior was changed here: 22515e3.
One could argue that the underlying stream should be fixed/changed to block as long as necessary instead, but in my view, the library should either handle cases where
fread
returns nothing whilefeof
is still false (by retrying), or return with an error to the caller, otherwise, as we see, it can silently produce truncated files. Another possible solution would be also passing the source file size (which is usually known by this time), and checking against that.What are your thoughts about the issue, and my fix/workaround, and how do you think a proper fix (that could be merged) could be organized?
The text was updated successfully, but these errors were encountered: