Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] FFI - Giving Buffer more low-level C functionality #1750

Closed
wants to merge 61 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
c1d2975
buffer: add readInt64BE, readInt64LE, readUInt64BE, and readUInt64LE
TooTallNate May 19, 2015
1871fbf
buffer: add writeInt64BE, writeInt64LE, writeUInt64BE, and writeUInt64LE
TooTallNate May 19, 2015
ca55494
buffer: add address() function
TooTallNate May 19, 2015
6c32b7e
buffer: add 0-length NULL pointer Buffer to exports
TooTallNate May 19, 2015
1f65088
buffer: add the hex address to the inspect() output
TooTallNate May 19, 2015
b8eac6d
buffer: add isNull() function
TooTallNate May 19, 2015
a9cdb38
buffer: remove "base" variable name from template
TooTallNate May 20, 2015
e338a92
buffer: return the int64 write binding values
TooTallNate May 20, 2015
af8bff4
buffer: add readPointerLE and readPointerBE
TooTallNate May 20, 2015
0f1e401
buffer: add writePointerLE and writePointerBE
TooTallNate May 20, 2015
2d54386
buffer: use uintptr_t for address()
TooTallNate May 20, 2015
db39bc2
buffer: fix JS_MAX_INT and JS_MIN_INT values
TooTallNate May 20, 2015
34feb80
buffer: add HasInstance check to IsNull()
TooTallNate May 21, 2015
89e7d61
buffer: add HasInstance check to Address()
TooTallNate May 21, 2015
5e9c2a4
buffer: allow offset to be passed to isNull and address
TooTallNate May 21, 2015
db6aea9
buffer: fix offset checking logic
TooTallNate May 21, 2015
1fdf531
buffer: inline WrapNullPointer
TooTallNate May 22, 2015
ff5a094
buffer: use nullptr
TooTallNate May 22, 2015
dea8c4d
buffer: asterisks lean to the left
TooTallNate May 22, 2015
4dae6be
buffer: remove unnecessary #defines
TooTallNate May 22, 2015
619d136
buffer: fix most of the cpplint errors
TooTallNate May 22, 2015
6e07bbc
buffer: remove EscapableHandleScope in readPointer
TooTallNate May 22, 2015
e6f3262
buffer: add CHECK calls
TooTallNate May 22, 2015
565a1e8
buffer: remove variable redefinition
TooTallNate May 22, 2015
66d68ef
buffer: use node::Utf8Value
TooTallNate May 22, 2015
51e9556
buffer: don't use C++ streams, use printf %lx
TooTallNate May 22, 2015
2b6ec58
buffer: don't rely on snprintf return value
TooTallNate May 22, 2015
c84e890
test: fix tests for new Buffer#inspect() behavior
TooTallNate May 22, 2015
001b072
doc: add docs for Buffer read/writePointer
TooTallNate May 22, 2015
86ff08c
doc: buffer docs tweaks
TooTallNate May 22, 2015
319e593
doc: add Buffer readInt64* docs
TooTallNate May 22, 2015
924da7f
doc: add fake memory addresses to old examples
TooTallNate May 22, 2015
ed4d220
buffer: return `null` from readPointer upon NULL
TooTallNate May 23, 2015
13c77bd
buffer: remove static `buffer.NULL` Buffer
TooTallNate May 23, 2015
61c5c7c
doc: document Buffer#readPointer `null` behavior
TooTallNate May 23, 2015
a9e8f62
buffer: remove Buffer#isNull()
TooTallNate May 23, 2015
f0cec38
buffer: remove `offset` param from `address()`
TooTallNate May 23, 2015
3f0d7a0
doc: fix buffer typo
TooTallNate May 23, 2015
ea44b75
buffer: fix checkPointer when null is passed in
TooTallNate May 23, 2015
7326597
doc: add Buffer#address() docs
TooTallNate May 23, 2015
2ace08f
doc: fix Buffer#address() output in example
TooTallNate May 23, 2015
e7f8d77
doc: add Buffer writeInt64*() docs
TooTallNate May 23, 2015
4ee5354
Revert "doc: add fake memory addresses to old examples"
TooTallNate May 23, 2015
f364f1b
doc: add fake memory addresses to old examples
TooTallNate May 23, 2015
13b5d67
doc: add another fake memory address to example
TooTallNate May 23, 2015
f451ee7
test: add Buffer#address() test case
TooTallNate May 23, 2015
39d468c
test: add Buffer int64 read/write tests
TooTallNate May 23, 2015
0c80b8c
test: add writeInt64 TypeError bad input tests
TooTallNate May 23, 2015
1867bd6
test: add Buffer read/writePointer test cases
TooTallNate May 23, 2015
b929ed0
buffer: use memcpy instead of reinterpret_cast
TooTallNate May 23, 2015
0e5cbe5
buffer: use MDN links for max/min JS int values
TooTallNate May 28, 2015
d6e37cd
buffer: do not free() buffers from readPointer*()
TooTallNate May 28, 2015
01a3ad3
doc: remove backticks from code examples
TooTallNate Jun 1, 2015
df5bda7
buffer: remove __STDC_FORMAT_MACROS
TooTallNate Jun 1, 2015
8477e98
buffer: revert unnecessary whitespace fix
TooTallNate Jun 1, 2015
3887226
buffer: add instanceof check to address()
TooTallNate Jun 1, 2015
c79e4f7
buffer: remove "pointer" functions
TooTallNate Jun 4, 2015
91e35b4
buffer: revert unnecessary whitespace change
TooTallNate Jun 4, 2015
797ef99
buffer: use base of 10 for the strto(u)ll calls
TooTallNate Jun 4, 2015
475aabb
buffer: change C++-land throw to a CHECK()
TooTallNate Jun 4, 2015
ce00983
buffer: make strto(u)ll usage cross-platform
TooTallNate Jun 5, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 117 additions & 11 deletions doc/api/buffer.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ Supports up to 48 bits of accuracy. For example:

var b = new Buffer(6);
b.writeUIntBE(0x1234567890ab, 0, 6);
// <Buffer 12 34 56 78 90 ab>
// <Buffer@0x102800408 12 34 56 78 90 ab>

Set `noAssert` to `true` to skip validation of `value` and `offset`. Defaults
to `false`.
Expand Down Expand Up @@ -283,7 +283,7 @@ Example:
});

console.log(copy);
// <Buffer 74 65 73 74>
// <Buffer@0x102800408 74 65 73 74>

### buf[index]

Expand Down Expand Up @@ -498,6 +498,37 @@ Example:
// 0x03042342
// 0x42230403

### buf.readUInt64LE(offset[, noAssert])
### buf.readUInt64BE(offset[, noAssert])

* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number or String

Reads an unsigned 64 bit integer from the buffer at the specified offset with
specified endian format.

If the value is greater than `Number.MAX_SAFE_INTEGER` then a String is
returned, otherwise a Number is returned. Because of this, it is assumed
that you will pass the return value to a JavaScript Int64 module, rather
than interact with the value directly.

Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.

Example:

var buf = new Buffer(8);
buf[0] = buf[1] = buf[2] = buf[3] = 0x00;
buf[4] = buf[5] = buf[6] = buf[7] = 0xff;
// <Buffer@0x10187aa08 00 00 00 00 ff ff ff ff>

b.readUInt64BE(0);
// 4294967295

b.readUInt64LE(0);
// '1844674406941458432'

### buf.readInt8(offset[, noAssert])

* `offset` Number
Expand Down Expand Up @@ -544,6 +575,28 @@ may be beyond the end of the buffer. Defaults to `false`.
Works as `buffer.readUInt32*`, except buffer contents are treated as two's
complement signed values.

### buf.readInt64LE(offset[, noAssert])
### buf.readInt64BE(offset[, noAssert])

* `offset` Number or String
* `noAssert` Boolean, Optional, Default: false
* Return: Number

Reads a signed 64 bit integer from the buffer at the specified offset with
specified endian format.

If the value is less than `Number.MIN_SAFE_INTEGER` or greater than
`Number.MAX_SAFE_INTEGER` then a String is returned, otherwise a Number
is returned. Because of this, it is assumed that you will pass the
return value to a JavaScript Int64 module, rather than interact with the
value directly.

Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.

Works as `buffer.readUInt64*`, except buffer contents are treated as two's
complement signed values.

### buf.readFloatLE(offset[, noAssert])
### buf.readFloatBE(offset[, noAssert])

Expand Down Expand Up @@ -624,7 +677,7 @@ Example:

console.log(buf);

// <Buffer 03 04 23 42>
// <Buffer@0x102800408 03 04 23 42>

### buf.writeUInt16LE(value, offset[, noAssert])
### buf.writeUInt16BE(value, offset[, noAssert])
Expand Down Expand Up @@ -654,8 +707,8 @@ Example:

console.log(buf);

// <Buffer de ad be ef>
// <Buffer ad de ef be>
// <Buffer@0x102800408 de ad be ef>
// <Buffer@0x102800408 ad de ef be>

### buf.writeUInt32LE(value, offset[, noAssert])
### buf.writeUInt32BE(value, offset[, noAssert])
Expand Down Expand Up @@ -683,8 +736,33 @@ Example:

console.log(buf);

// <Buffer fe ed fa ce>
// <Buffer ce fa ed fe>
// <Buffer@0x102800408 fe ed fa ce>
// <Buffer@0x102800408 ce fa ed fe>

### buf.writeUInt64LE(value, offset[, noAssert])
### buf.writeUInt64BE(value, offset[, noAssert])

* `value` Number or String
* `offset` Number
* `noAssert` Boolean, Optional, Default: false

Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid unsigned 64 bit integer, or a String
representation of one.

Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.

Example:

var buf = new Buffer(8)

b.writeUInt64LE('0xffffffffffff')

console.log(b);
// <Buffer@0x10187aa08 ff ff ff ff ff ff 00 00>

### buf.writeInt8(value, offset[, noAssert])

Expand Down Expand Up @@ -739,6 +817,25 @@ should not be used unless you are certain of correctness. Defaults to `false`.
Works as `buffer.writeUInt32*`, except value is written out as a two's
complement signed integer into `buffer`.

### buf.writeInt64(value, offset[, noAssert])
### buf.writeInt64(value, offset[, noAssert])

* `value` Number or String
* `offset` Number
* `noAssert` Boolean, Optional, Default: false

Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid signed 64 bit integer, or a String
representation of one.

Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.

Works as `buffer.writeUInt64*`, except value is written out as a two's
complement signed integer into `buffer`.

### buf.writeFloatLE(value, offset[, noAssert])
### buf.writeFloatBE(value, offset[, noAssert])

Expand All @@ -765,8 +862,8 @@ Example:

console.log(buf);

// <Buffer 4f 4a fe bb>
// <Buffer bb fe 4a 4f>
// <Buffer@0x102800408 4f 4a fe bb>
// <Buffer@0x102800408 bb fe 4a 4f>

### buf.writeDoubleLE(value, offset[, noAssert])
### buf.writeDoubleBE(value, offset[, noAssert])
Expand Down Expand Up @@ -794,8 +891,8 @@ Example:

console.log(buf);

// <Buffer 43 eb d5 b7 dd f9 5f d7>
// <Buffer d7 5f f9 dd b7 d5 eb 43>
// <Buffer@0x102800408 43 eb d5 b7 dd f9 5f d7>
// <Buffer@0x102800408 d7 5f f9 dd b7 d5 eb 43>

### buf.fill(value[, offset][, end])

Expand All @@ -810,6 +907,15 @@ buffer.
var b = new Buffer(50);
b.fill("h");

### buffer.address()

Returns a String representation of the hex address of the pointer in memory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like including the address on inspect() isn't useful for the> 95% of use cases, and since there's now the address() function it can be retrieved at any time.


var b = new Buffer(1);

b.address();
// '10202ee08'

### buffer.values()

Creates iterator for buffer values (bytes). This function is called automatically
Expand Down
2 changes: 1 addition & 1 deletion doc/api/errors.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ fs.readFile('/some/file/that/does-not-exist', function nodeStyleCallback(err, da

fs.readFile('/some/file/that/does-exist', function(err, data) {
console.log(err) // null
console.log(data) // <Buffer: ba dd ca fe>
console.log(data) // <Buffer@0x102800408 ba dd ca fe>
})
```

Expand Down
107 changes: 106 additions & 1 deletion lib/buffer.js
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ Buffer.prototype.inspect = function inspect() {
if (this.length > max)
str += ' ... ';
}
return '<' + this.constructor.name + ' ' + str + '>';
return '<' + this.constructor.name + '@0x' + this.address() + ' ' + str + '>';
};


Expand Down Expand Up @@ -454,6 +454,13 @@ Buffer.prototype.fill = function fill(val, start, end) {
};


Buffer.prototype.address = function address() {
if (!(this instanceof Buffer))
throw new TypeError('this must be a Buffer');
return binding.address(this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should have preliminary checks here that check that this is a Buffer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a note, both this check and the HasInstance() checks are necessary. Because the object must be an instance of Buffer, and it must contain external array data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JS-side instanceof Buffer checks seem to current only happen in the write*() functions.
fill(), slice(), toString(), etc. don't seem to have these checks. Is that intentional?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. They should in fact have these checks.

EDIT: I'll take care of the other cases in a different PR. Thanks for bringing them up.

};


// XXX remove in v0.13
Buffer.prototype.get = util.deprecate(function get(offset) {
offset = ~~offset;
Expand Down Expand Up @@ -818,6 +825,38 @@ Buffer.prototype.readDoubleBE = function readDoubleBE(offset, noAssert) {
};


Buffer.prototype.readInt64LE = function readInt64LE(offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkOffset(offset, 8, this.length);
return binding.readInt64LE(this, offset);
};


Buffer.prototype.readInt64BE = function readInt64BE(offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkOffset(offset, 8, this.length);
return binding.readInt64BE(this, offset);
};


Buffer.prototype.readUInt64LE = function readUInt64LE(offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkOffset(offset, 8, this.length);
return binding.readUInt64LE(this, offset);
};


Buffer.prototype.readUInt64BE = function readUInt64BE(offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkOffset(offset, 8, this.length);
return binding.readUInt64BE(this, offset);
};


function checkInt(buffer, value, offset, ext, max, min) {
if (!(buffer instanceof Buffer))
throw new TypeError('buffer must be a Buffer instance');
Expand Down Expand Up @@ -1071,6 +1110,72 @@ Buffer.prototype.writeDoubleBE = function writeDoubleBE(val, offset, noAssert) {
return offset + 8;
};


function checkInt64(buffer, value, offset, ext) {
if (!(buffer instanceof Buffer))
throw new TypeError('buffer must be a Buffer instance');
if (!(typeof value === 'string' || typeof value === 'number'))
throw new TypeError('must pass a "string" or "number" for value');
if (offset + ext > buffer.length)
throw new RangeError('index out of range');
}


Buffer.prototype.writeInt64LE = function writeInt64LE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkInt64(this, val, offset, 8);
return binding.writeInt64LE(this, val, offset);
};


Buffer.prototype.writeInt64BE = function writeInt64BE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkInt64(this, val, offset, 8);
return binding.writeInt64BE(this, val, offset);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all these write*64*() operations, you should check if val is a number and if it's within the 2^51 range that can be handled by JS. If that's the case then just do the bit twiddling here. Just an idea. If you don't implement it then I will in my rewrite. :)

};


Buffer.prototype.writeUInt64LE = function writeUInt64LE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkInt64(this, val, offset, 8);
return binding.writeUInt64LE(this, val, offset);
};


Buffer.prototype.writeUInt64BE = function writeUInt64BE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkInt64(this, val, offset, 8);
return binding.writeUInt64BE(this, val, offset);
};


function checkPointer(buffer, value, offset) {
if (!(buffer instanceof Buffer))
throw new TypeError('buffer must be a Buffer instance');
if (!(value === null || value instanceof Buffer))
throw new TypeError('value must be a Buffer instance or null');
}

Buffer.prototype.writePointerLE = function writePointerLE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkPointer(this, val, offset);
return binding.writePointerLE(this, val, offset);
};


Buffer.prototype.writePointerBE = function writePointerBE(val, offset, noAssert) {
offset = offset >>> 0;
if (!noAssert)
checkPointer(this, val, offset);
return binding.writePointerBE(this, val, offset);
};


// ES6 iterator

var ITERATOR_KIND_KEYS = 1;
Expand Down
Loading