Skip to content

Commit 231e095

Browse files
HADOOP-18407. Improve readVectored() api spec (#4760)
part of HADOOP-18103. Contributed By: Mukund Thakur
1 parent a9e5fb3 commit 231e095

File tree

2 files changed

+17
-0
lines changed

2 files changed

+17
-0
lines changed

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PositionedReadable.java

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,16 @@ default int maxReadSizeForVectorReads() {
114114
* As a result of the call, each range will have FileRange.setData(CompletableFuture)
115115
* called with a future that when complete will have a ByteBuffer with the
116116
* data from the file's range.
117+
* <p>
118+
* The position returned by getPos() after readVectored() is undefined.
119+
* </p>
120+
* <p>
121+
* If a file is changed while the readVectored() operation is in progress, the output is
122+
* undefined. Some ranges may have old data, some may have new and some may have both.
123+
* </p>
124+
* <p>
125+
* While a readVectored() operation is in progress, normal read api calls may block.
126+
* </p>
117127
* @param ranges the byte ranges to read
118128
* @param allocate the function to allocate ByteBuffer
119129
* @throws IOException any IOE.

hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstream.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -454,6 +454,13 @@ Also, clients are encouraged to use `WeakReferencedElasticByteBufferPool` for
454454
allocating buffers such that even direct buffers are garbage collected when
455455
they are no longer referenced.
456456

457+
The position returned by `getPos()` after `readVectored()` is undefined.
458+
459+
If a file is changed while the `readVectored()` operation is in progress, the output is
460+
undefined. Some ranges may have old data, some may have new, and some may have both.
461+
462+
While a `readVectored()` operation is in progress, normal read api calls may block.
463+
457464
Note: Don't use direct buffers for reading from ChecksumFileSystem as that may
458465
lead to memory fragmentation explained in HADOOP-18296.
459466

0 commit comments

Comments
 (0)