Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: refacor MallocedBuffer to it's usage scope #23641

Closed
wants to merge 19 commits into from

Conversation

refack
Copy link
Contributor

@refack refack commented Oct 13, 2018

As discussed in #23543 (review).
MallocedBuffer is an unknown quantity as compared to std::unique_ptr.

  1. MallocedBuffer is a new utility and is node specific, so it's semantics are not clear to me, so I assume to others as well.
  2. std::unique_ptr is free of runtime cost (except for its dtor), while MallocedBuffer at minimum carries the runtime cost of construction, and the memory cost of MallocedBuffer::size.
  3. And probably most important to me - readability. std::unique_ptr is standardized and well documented. MallocedBuffer is only documented by its code.
  4. While refactoring is seems like it has no unit test coverage.

Ref: #23543 (review)
Ref: #23434

CI: https://ci.nodejs.org/job/node-test-pull-request/1​78​21

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

addaleax and others added 17 commits October 12, 2018 21:07
Original commit message:

    [api] Remove deprecated wasm methods

    These methods were deprecated in 7.0, now we can remove them.

    R=adamk@chromium.org

    Bug: v8:7868
    Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
    Change-Id: I60badb378a055152bdd27aed67d11ddf74fce174
    Reviewed-on: https://chromium-review.googlesource.com/1209283
    Reviewed-by: Adam Klein <adamk@chromium.org>
    Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
    Cr-Commit-Position: refs/heads/master@{nodejs#55695}

Refs: v8/v8@b0af309

PR-URL: nodejs#23415
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
PR-URL: nodejs#23455
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Covering the case when fs-read get invalid argument for file handle

PR-URL: nodejs#23601
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Minor cleanup in the lifetime for the platform worker initialization
synchronization barrier.

PR-URL: nodejs#23419
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
When using `assert.strictEqual`, the first argument must be the actual
value and the second argument must be the expected value.

PR-URL: nodejs#23448
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23449
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Beth Griggs <Bethany.Griggs@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Have actual first, expected second.

PR-URL: nodejs#23450
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reverse the argument for assertion. The first argument should be the
actual value and the second value should be the expected value. When
there is an AssertionError, the expected and actual value will be
labeled correctly.

PR-URL: nodejs#23451
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Catch statement defines err variable that is never used, so it
is safe to remove that.

PR-URL: nodejs#23452
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Converts RangeError assertions to use common.expectsError and includes
an assertion for the error code.

PR-URL: nodejs#23454
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23456
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23457
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23458
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Hitesh Kanwathirtha <digitalinfinity@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23459
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23461
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23463
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
PR-URL: nodejs#23465
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Oct 13, 2018
@refack refack self-assigned this Oct 13, 2018
@joyeecheung
Copy link
Member

joyeecheung commented Oct 13, 2018

I am not sure about arguing that std::unique_ptr as a standardized type is more readable than MallocedBuffer. The scope of MallocedBuffer is very well explained in the one-line comment:

// Simple RAII wrapper for contiguous data that uses malloc()/free().

Whereas with std::unique_ptr you don't get any answer about

  1. Is it wrapping a pointer that needs to be freed or deleted? (You may get the answer by looking at the template parameter, but first you need to see the parameter explicitly spelled somewhere, which means you need to spell std::unique_ptr<something, i_use_free>)
  2. Is it pointing to a contiguous chunk of memory, or an end of some other data structure?

@addaleax
Copy link
Member

  1. MallocedBuffer is a new utility and is node specific, so it's semantics are not clear to me, so I assume to others as well.

As @joyeecheung said, the semantics are defined right next to the class.

It's maturity is unknown (Ref: #23434)

I don’t think that bug lets you make conclusions about “maturity” – this is not a full-featured memory manager, this is a small utility class.

  1. std::unique_ptr is free of runtime cost (except for the dtor), while MallocedBuffer is unknown at minimum carries the runtime cost of construction, and the memory cost of MallocedBuffer::size.

There’s no reason to believe that there is any difference in runtime cost. The memory cost of extra 8 bytes is negligible, and is actually useful a lot of the time.


I don’t think comparing to std::unique_ptr makes sense in the first place.

This structure always represents a continuous array of data. It’s closest STL equivalent is std::vector, not std::unique_ptr. Sometimes, we need to use malloc(), though (because we can actually handle allocation failures, because we convert it to a Buffer, etc.), and writing custom STL allocators is painful and adds a lot more complexity than just using MallocedBuffer().

Copy link
Member

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1, see above.

@refack
Copy link
Contributor Author

refack commented Oct 13, 2018

I am not sure about arguing that std::unique_ptr as a standardized type is more readable than MallocedBuffer. The scope of MallocedBuffer is very well explained in the one-line comment:

I am not sure about arguing that std::unique_ptr as a standardized type is more readable than MallocedBuffer. The scope of MallocedBuffer is very well explained in the one-line comment:

// Simple RAII wrapper for contiguous data that uses malloc()/free().

Whereas with std::unique_ptr you don't get any answer about

  1. Is it wrapping a pointer that needs to be freed or deleted? (You may get the answer by looking at the template parameter, but first you need to see the parameter explicitly spelled somewhere, which means you need to spell std::unique_ptr<something, i_use_free>)
  2. Is it pointing to a contiguous chunk of memory, or an end of some other data structure?

It might be my perspective, but https://en.cppreference.com/w/cpp/memory/unique_ptr seems to answer these quastion, and leave much less ambiguity.

@refack
Copy link
Contributor Author

refack commented Oct 13, 2018

I don’t think comparing to std::unique_ptr makes sense in the first place.

This structure always represents a continuous array of data. It’s closest STL equivalent is std::vector, not std::unique_ptr. Sometimes, we need to use malloc(), though (because we can actually handle allocation failures, because we convert it to a Buffer, etc.), and writing custom STL allocators is painful and adds a lot more complexity than just using MallocedBuffer().

As I see it (again this might point to ambiguity) MallocedBuffer is a: Simple RAII wrapper for contiguous data that uses malloc()/free(). So the analogy to std::unique_ptr is not too far fetched (and is exemplified in #23543).

I think that the main difference between this and std::vector is that vector by design hides memory management, while MallocedBuffer put it front and center.

@addaleax
Copy link
Member

I think that the main difference between this and std::vector is that vector by design hides memory management, while MallocedBuffer put it front and center.

Yes. So let’s use it when it matters that we use a specific memory allocator.

@refack
Copy link
Contributor Author

refack commented Oct 13, 2018

I think that the main difference between this and std::vector is that vector by design hides memory management, while MallocedBuffer put it front and center.

Yes. So let’s use it when it matters that we use a specific memory allocator.

I agree that it makes sense to abstract that, I just disagree on the implementation. IMHO we should minimize our own code. Something like:

template<typename T>
struct Free {
  void operator()(T* ptr) const { free(ptr); }
};

// Specialization of std::unique_ptr that used Malloc<t>
template<typename T>
using malloced_unique_ptr = std::unique_ptr<T, Free<T>>;

template<typename T>
malloced_unique_ptr make_malloced_unique(size_t number_of_t) {
  return malloced_unique_ptr(Malloc<T>(number_of_t));
}

@addaleax
Copy link
Member

@refack I think the main downside to that is that it does not provide size explicitly.

I just disagree on the implementation.

This PR seems to do a lot more than changing the implementation.

@refack
Copy link
Contributor Author

refack commented Oct 13, 2018

@refack I think the main downside to that is that it does not provide size explicitly.

As I see it we have two use cases:

  1. A "buffer" where we want to keep track of its size, store or keep track in a wider scope. There MallocedBuffer is probably a better fit.
  2. An RAII wrapper for Malloc, where size is tracked independently, and probably has a release() in scope. That's where IMHO MallocedBuffer is not the best abstraction.

@refack
Copy link
Contributor Author

refack commented Oct 13, 2018

This PR seems to do a lot more than changing the implementation.

Ok, so an alternative suggested at #23642

@joyeecheung
Copy link
Member

joyeecheung commented Oct 13, 2018

It might be my perspective, but https://en.cppreference.com/w/cpp/memory/unique_ptr seems to answer these quastion, and leave much less ambiguity.

@refack That is a long document which just explains std::unique_ptr, can you point out how it answer those questions?

Maybe I wasn't being clear: std::unique_ptr may be able to provide the means to answer questions about the nature of the underlying pointer through template parameters, but that doesn't mean it's readable and easy to use, and the answers ultimately come from the users, not the std::unique_ptr type itself, whereas MallocedBuffer only does one job and the name iis expressive enough.

(Also, I think it's even possible to implement MallocedBuffer on top of unique_ptr and I think there is still value doing it that instead of just using unique_ptr)

EDIT: oh, yeah there is a (sort-of) implementation #23641 (comment) already

@refack
Copy link
Contributor Author

refack commented Oct 20, 2018

Abandoned

@refack refack closed this Oct 20, 2018
@refack refack deleted the refactor-MallocedBuffer branch October 20, 2018 21:22
@refack refack removed their assignment Oct 20, 2018
refack added a commit to refack/node that referenced this pull request Oct 20, 2018
@refack refack added the stalled Issues and PRs that are stalled. label Oct 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. stalled Issues and PRs that are stalled.
Projects
None yet
Development

Successfully merging this pull request may close these issues.