Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fs: make sure to write entire buffer #42434

Closed
wants to merge 13 commits into from
Closed

fs: make sure to write entire buffer #42434

Changes from 5 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
4307397
fs: make sure to write entire buffer
ronag Mar 22, 2022
b6b9680
fixup
ronag Mar 22, 2022
a5bf600
fixup
ronag Mar 22, 2022
523314a
fixup
ronag Mar 22, 2022
d4c7a90
fixup
ronag Mar 22, 2022
d0afc94
fixuP
ronag Mar 22, 2022
43f2529
fixuP
ronag Mar 22, 2022
4ff89b4
fixuP
ronag Mar 22, 2022
3869561
fixuP
ronag Mar 22, 2022
ab95708
fixuP
ronag Mar 22, 2022
43d9020
fixuP
ronag Mar 22, 2022
09b7994
fixuP
ronag Mar 22, 2022
be950ef
Update lib/internal/fs/promises.js
ronag Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 59 additions & 14 deletions lib/internal/fs/streams.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -386,22 +386,72 @@ WriteStream.prototype.open = openWriteFs;

WriteStream.prototype._construct = _construct;

function writeAll(data, pos, cb, retries = 0) {
const len = data.length;
this[kFs].write(this.fd, data, 0, len, pos, (er, bytesWritten) => {
if (er?.code === 'EAGAIN') {
er = null;
bytesWritten = 0;
}

if (this.destroyed || er) {
return cb(er);
ronag marked this conversation as resolved.
Show resolved Hide resolved
}
benjamingr marked this conversation as resolved.
Show resolved Hide resolved

ronag marked this conversation as resolved.
Show resolved Hide resolved
retries = bytesWritten ? 0 : retries + 1;

if (retries > 5) {
return cb(new Error('writev failed'));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we trying for 5 times here? Writing a non-zero number of bytes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment here.

}

this.bytesWritten += bytesWritten;

if (bytesWritten < len) {
writeAll(data.slice(bytesWritten), pos + bytesWritten, cb, retries);
} else {
cb();
}
});
}

function writevAll(chunks, pos, cb, retries = 0) {
this[kFs].writev(this.fd, chunks, this.pos, (er, bytesWritten) => {
if (er?.code === 'EAGAIN') {
er = null;
bytesWritten = 0;
}

if (this.destroyed || er) {
return cb(er);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will er here be something if this.destroyed is truthy? Otherwise, this could signal to the caller that the call succeeded?

}

this.bytesWritten += bytesWritten;

retries = bytesWritten ? 0 : retries + 1;

if (retries > 5) {
return cb(new Error('writev failed'));
}

if (bytesWritten < len) {
writevAll([Buffer.concat(chunks).slice(bytesWritten)], pos + bytesWritten, cb, retries);
} else {
cb();
}
});
}

WriteStream.prototype._write = function(data, encoding, cb) {
this[kIsPerformingIO] = true;
this[kFs].write(this.fd, data, 0, data.length, this.pos, (er, bytes) => {
writeAll.call(this, data, this.pos, (er) => {
this[kIsPerformingIO] = false;
if (this.destroyed) {
// Tell ._destroy() that it's safe to close the fd now.
cb(er);
return this.emit(kIoDone, er);
}

if (er) {
return cb(er);
}

this.bytesWritten += bytes;
cb();
cb(er);
});

if (this.pos !== undefined)
Expand All @@ -421,20 +471,15 @@ WriteStream.prototype._writev = function(data, cb) {
}

this[kIsPerformingIO] = true;
this[kFs].writev(this.fd, chunks, this.pos, (er, bytes) => {
writevAll.call(this, chunks, this.pos, (er) => {
this[kIsPerformingIO] = false;
if (this.destroyed) {
// Tell ._destroy() that it's safe to close the fd now.
cb(er);
return this.emit(kIoDone, er);
}

if (er) {
return cb(er);
}

this.bytesWritten += bytes;
cb();
cb(er);
});

if (this.pos !== undefined)
Expand Down