-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
src: clean up MaybeStackBuffer usage in i18n #11464
Conversation
/cc @nodejs/intl |
src/node_internals.h
Outdated
if (sizeof(T) > 1 && IsBigEndian()) { | ||
SPREAD_BUFFER_ARG(ret.ToLocalChecked(), retbuf); | ||
SwapBytes16(retbuf_data, retbuf_length); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I would prefer it if we did the SwapBytes16()
at the call side and kept Buffer::New()
s job restricted to turning the memory block into a Buffer
instance?
src/util.h
Outdated
if (storage > capacity()) { | ||
T* allocatedPtr = IsAllocated() ? buf_ : nullptr; | ||
buf_ = Realloc<T>(allocatedPtr, storage); | ||
capacity_ = storage; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right now, if the buffer wasn’t allocated before, the previous contents are lost (but not if it was malloc()
’ed). I guess this okay and intended behaviour, but maybe note that in the comment, too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR updated. Now contents up to length_ - 1
are preserved if buf_
was on the stack.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the ICU changes seem good to me. (Side note: I misread MaybeStackBuffer as ICU's MaybeStackArray at first… )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Rebased and changed Updated CI: https://ci.nodejs.org/job/node-test-pull-request/6535/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM modulo nits.
src/node_i18n.cc
Outdated
if (buf->IsAllocated()) { | ||
MaybeLocal<Object> ret = Buffer::New(isolate, buf->out(), len); | ||
if (!ret.IsEmpty()) buf->Release(); | ||
template<typename T> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tiny style nit: space before <
.
src/node_internals.h
Outdated
// underlying buffer. However, |buf| itself can be reused even after this call, | ||
// but its capacity, if increased through AllocateSufficientStorage, is not | ||
// guaranteed to stay the same. | ||
template<typename T> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
src/node_internals.h
Outdated
MaybeStackBuffer<T>* buf) { | ||
v8::MaybeLocal<v8::Object> ret; | ||
char* src = reinterpret_cast<char*>(buf->out()); | ||
const size_t len_in_bytes = buf->length() * sizeof(T); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sizeof(buf[0])
should work too and is arguably a little more Obviously Correct. Here, you don't know what T is without looking at the surrounding code.
src/util.h
Outdated
@@ -300,58 +301,81 @@ class MaybeStackBuffer { | |||
return buf_[index]; | |||
} | |||
|
|||
// Size of the buffer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somewhat superfluous comment.
src/util.h
Outdated
// There can only be 1 call to AllocateSufficientStorage or Invalidate | ||
// per instance. | ||
// Actual maximum capacity of the buffer with which SetLength() can be called | ||
// without first calling AllocateSufficientStorage() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you punctuate the comment?
src/util.h
Outdated
// Make sure enough space for `storage` entries is available. | ||
// This method can be called multiple times throughout the lifetime of the | ||
// buffer, but once this has been called Invalidate() cannot be used. | ||
// Contents of the buffer in the range [0, length()) are preserved. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/are/is/? 'Contents' is indefinite and therefore singular (I think? Not 100% sure.)
src/util.h
Outdated
if (storage > capacity()) { | ||
bool was_allocated = IsAllocated(); | ||
T* allocated_ptr = was_allocated ? buf_ : nullptr; | ||
buf_ = Realloc<T>(allocated_ptr, storage); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The explicit <T>
shouldn't be necessary. Not a big deal, though.
src/util.h
Outdated
buf_ = Realloc<T>(allocated_ptr, storage); | ||
capacity_ = storage; | ||
if (!was_allocated && length_ > 0) | ||
memcpy(buf_, buf_st_, length_ * sizeof(T)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sizeof(buf_[0])
?
src/util.h
Outdated
length_ = 0; | ||
buf_ = nullptr; | ||
} | ||
|
||
bool IsAllocated() { | ||
return buf_ != buf_st_; | ||
// If the buffer is stored in the heap rather than on the stack |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you punctuate the comments here and below?
test/cctest/util.cc
Outdated
EXPECT_FALSE(buf.IsAllocated()); \ | ||
EXPECT_GT(buf.capacity(), buf.length()); \ | ||
free(rawbuf); \ | ||
} while (0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider putting this in a template function rather than a macro. Rule of thumb: template when you can, macro when you must.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM with @bnoordhuis' comments addressed
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer
- Templatize AsBuffer() and create a generic version for inclusion in the Buffer class - Use MaybeStackBuffer::storage() - If possible, avoid double conversion in ToASCII()/ToUnicode() - More descriptive assertion error in tests
@bnoordhuis, comments addressed. PTAL. Updated CI: https://ci.nodejs.org/job/node-test-pull-request/6560/ |
return ret; | ||
|
||
if (buf->IsAllocated()) | ||
buf->Release(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be done in the if
block above?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ret.IsEmpty()
check has to happen before calling buf->Release()
, so unfortunately not.
src/util.h
Outdated
// length_ stores how much memory was allocated. | ||
CHECK_LE(length, length_); | ||
// capacity_ stores how much memory was allocated. | ||
CHECK_LE(length, capacity()); | ||
length_ = length; | ||
} | ||
|
||
void SetLengthAndZeroTerminate(size_t length) { | ||
// length_ stores how much memory was allocated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment also needs to be updated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. Will fix.
PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Templatize AsBuffer() and create a generic version for inclusion in the Buffer class - Use MaybeStackBuffer::storage() - If possible, avoid double conversion in ToASCII()/ToUnicode() - More descriptive assertion error in tests PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
Landed in 289e532...d4e1eaf. |
PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Templatize AsBuffer() and create a generic version for inclusion in the Buffer class - Use MaybeStackBuffer::storage() - If possible, avoid double conversion in ToASCII()/ToUnicode() - More descriptive assertion error in tests PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
Parts of this should likely be backported to v6.x (and possibly v4.x). |
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer Backport-PR-URL: #17365 PR-URL: #11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer Backport-PR-URL: #17365 PR-URL: #11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer Backport-PR-URL: #17365 PR-URL: #11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer Backport-PR-URL: #17365 PR-URL: #11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>
This PR can roughly be divided into two parts. It first cleans up
MaybeStackBuffer
itself:IsAllocated()
work for invalidated bufferscapacity()
method for finding out the actual capacity, not the current size, of the bufferAllocateSufficientStorage()
andInvalidate()
Release()
Then, it aims to make the usage of
MaybeStackBuffer
innode_i18n
more orthodox:MaybeStackBuffer<T>
-toBuffer
conversion function (AsBuffer()
) and move it toBuffer
namespace as a private constructorMaybeStackBuffer
, but there are multiple problems in doing that due to header/class dependency. On the other hand,Buffer
is as good as a home for the function as any.MaybeStackBuffer::storage()
instead of defining its own version ofkStackStorageSize
(which may get out of sync with the official one inMaybeStackBuffer
)len
parameter toAsBuffer()
, useMaybeStackBuffer::SetLength()
uniformlyToASCII()
/ToUnicode()
That last bullet point brings a nice performance enhancement for
url.domainToASCII()
/domainToUnicode()
with short strings (n
reduced to 1×106 since this method is not as susceptible to JIT fluctuations):Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passesAffected core subsystem(s)
src