Skip to content

Commit 3ece8ee

Browse files
dongjoon-hyunHyukjinKwon
authored andcommitted
[SPARK-29124][CORE] Use MurmurHash3 bytesHash(data, seed) instead of bytesHash(data)
### What changes were proposed in this pull request? This PR changes `bytesHash(data)` API invocation with the underlaying `byteHash(data, arraySeed)` invocation. ```scala def bytesHash(data: Array[Byte]): Int = bytesHash(data, arraySeed) ``` ### Why are the changes needed? The original API is changed between Scala versions by the following commit. From Scala 2.12.9, the semantic of the function is changed. If we use the underlying form, we are safe during Scala version migration. - scala/scala@846ee2b#diff-ac889f851e109fc4387cd738d52ce177 ### Does this PR introduce any user-facing change? No. ### How was this patch tested? This is a kind of refactoring. Pass the Jenkins with the existing tests. Closes #25821 from dongjoon-hyun/SPARK-SCALA-HASH. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
1 parent 05988b2 commit 3ece8ee

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ private[spark] object XORShiftRandom {
6060
/** Hash seeds to have 0/1 bits throughout. */
6161
private[random] def hashSeed(seed: Long): Long = {
6262
val bytes = ByteBuffer.allocate(java.lang.Long.BYTES).putLong(seed).array()
63-
val lowBits = MurmurHash3.bytesHash(bytes)
63+
val lowBits = MurmurHash3.bytesHash(bytes, MurmurHash3.arraySeed)
6464
val highBits = MurmurHash3.bytesHash(bytes, lowBits)
6565
(highBits.toLong << 32) | (lowBits.toLong & 0xFFFFFFFFL)
6666
}

0 commit comments

Comments
 (0)