Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: Move random data generation methods from CometCastSuite to new DataGenerator class #426

Merged
merged 5 commits into from
May 15, 2024
Merged

tests: Move random data generation methods from CometCastSuite to new DataGenerator class #426

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
892d66d
add new DataGenerator class
andygrove May 14, 2024
23082da
move more data gen methods
andygrove May 14, 2024
b109871
ignore some failing tests, update compat docs
andygrove May 14, 2024
edf28d9
address feedback
andygrove May 14, 2024
cf779df
fix regression
andygrove May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 24 additions & 87 deletions spark/src/test/scala/org/apache/comet/CometCastSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ import org.apache.comet.expressions.{CometCast, Compatible}
class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
import testImplicits._

/** Create a data generator using a fixed seed so that tests are reproducible */
private val gen = new DataGenerator(new Random(42))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to make this instance a field in the Companion object of RandomDataGenerator?

So that other test suites can all use the same random data generator by default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand what you are suggesting, I think you are saying to add this:

object DataGenerator {
  val DEFAULT = new DataGenerator(new Random(42))
}

And then reference that from the test suite?

private val gen = DataGenerator.DEFAULT

I think this could lead to some non-deterministic behavior. For example, running CometCastSuite on its own from my IDE would start with the freshly created generator, but running the same suite after after test suites have run would result in different inputs.

I think it would be better for each test to create a new instance of the generator with a fixed seed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we could define it as a def rather than a val to force a new creation each time. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the new suggested approach, that’s better and should be used.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @advancedxy. I have implemented your suggestion.


private val dataSize = 1000

// we should eventually add more whitespace chars here as documented in
Expand Down Expand Up @@ -478,7 +481,7 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
test("cast StringType to BooleanType") {
val testValues =
(Seq("TRUE", "True", "true", "FALSE", "False", "false", "1", "0", "", null) ++
generateStrings("truefalseTRUEFALSEyesno10" + whitespaceChars, 8)).toDF("a")
gen.generateStrings(dataSize, "truefalseTRUEFALSEyesno10" + whitespaceChars, 8)).toDF("a")
castTest(testValues, DataTypes.BooleanType)
}

Expand Down Expand Up @@ -519,53 +522,53 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
// test with hand-picked values
castTest(castStringToIntegralInputs.toDF("a"), DataTypes.ByteType)
// fuzz test
castTest(generateStrings(numericPattern, 4).toDF("a"), DataTypes.ByteType)
castTest(gen.generateStrings(dataSize, numericPattern, 4).toDF("a"), DataTypes.ByteType)
}

test("cast StringType to ShortType") {
// test with hand-picked values
castTest(castStringToIntegralInputs.toDF("a"), DataTypes.ShortType)
// fuzz test
castTest(generateStrings(numericPattern, 5).toDF("a"), DataTypes.ShortType)
castTest(gen.generateStrings(dataSize, numericPattern, 5).toDF("a"), DataTypes.ShortType)
}

test("cast StringType to IntegerType") {
// test with hand-picked values
castTest(castStringToIntegralInputs.toDF("a"), DataTypes.IntegerType)
// fuzz test
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.IntegerType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.IntegerType)
}

test("cast StringType to LongType") {
// test with hand-picked values
castTest(castStringToIntegralInputs.toDF("a"), DataTypes.LongType)
// fuzz test
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.LongType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.LongType)
}

ignore("cast StringType to FloatType") {
// https://github.com/apache/datafusion-comet/issues/326
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.FloatType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.FloatType)
}

ignore("cast StringType to DoubleType") {
// https://github.com/apache/datafusion-comet/issues/326
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.DoubleType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.DoubleType)
}

ignore("cast StringType to DecimalType(10,2)") {
// https://github.com/apache/datafusion-comet/issues/325
val values = generateStrings(numericPattern, 8).toDF("a")
val values = gen.generateStrings(dataSize, numericPattern, 8).toDF("a")
castTest(values, DataTypes.createDecimalType(10, 2))
}

test("cast StringType to BinaryType") {
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.BinaryType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.BinaryType)
}

ignore("cast StringType to DateType") {
// https://github.com/apache/datafusion-comet/issues/327
castTest(generateStrings(datePattern, 8).toDF("a"), DataTypes.DateType)
castTest(gen.generateStrings(dataSize, datePattern, 8).toDF("a"), DataTypes.DateType)
}

test("cast StringType to TimestampType disabled by default") {
Expand All @@ -581,7 +584,10 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
ignore("cast StringType to TimestampType") {
// https://github.com/apache/datafusion-comet/issues/328
withSQLConf((CometConf.COMET_CAST_ALLOW_INCOMPATIBLE.key, "true")) {
val values = Seq("2020-01-01T12:34:56.123456", "T2") ++ generateStrings(timestampPattern, 8)
val values = Seq("2020-01-01T12:34:56.123456", "T2") ++ gen.generateStrings(
dataSize,
timestampPattern,
8)
castTest(values.toDF("a"), DataTypes.TimestampType)
}
}
Expand Down Expand Up @@ -630,7 +636,7 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}

test("cast BinaryType to StringType - valid UTF-8 inputs") {
castTest(generateStrings(numericPattern, 8).toDF("a"), DataTypes.StringType)
castTest(gen.generateStrings(dataSize, numericPattern, 8).toDF("a"), DataTypes.StringType)
}

// CAST from DateType
Expand Down Expand Up @@ -739,67 +745,31 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}

private def generateFloats(): DataFrame = {
val r = new Random(0)
val values = Seq(
Float.MaxValue,
Float.MinPositiveValue,
Float.MinValue,
Float.NaN,
Float.PositiveInfinity,
Float.NegativeInfinity,
1.0f,
-1.0f,
Short.MinValue.toFloat,
Short.MaxValue.toFloat,
0.0f) ++
Range(0, dataSize).map(_ => r.nextFloat())
withNulls(values).toDF("a")
withNulls(gen.generateFloats(dataSize)).toDF("a")
}

private def generateDoubles(): DataFrame = {
val r = new Random(0)
val values = Seq(
Double.MaxValue,
Double.MinPositiveValue,
Double.MinValue,
Double.NaN,
Double.PositiveInfinity,
Double.NegativeInfinity,
0.0d) ++
Range(0, dataSize).map(_ => r.nextDouble())
withNulls(values).toDF("a")
withNulls(gen.generateDoubles(dataSize)).toDF("a")
}

private def generateBools(): DataFrame = {
withNulls(Seq(true, false)).toDF("a")
}

private def generateBytes(): DataFrame = {
val r = new Random(0)
val values = Seq(Byte.MinValue, Byte.MaxValue) ++
Range(0, dataSize).map(_ => r.nextInt().toByte)
withNulls(values).toDF("a")
withNulls(gen.generateBytes(dataSize)).toDF("a")
}

private def generateShorts(): DataFrame = {
val r = new Random(0)
val values = Seq(Short.MinValue, Short.MaxValue) ++
Range(0, dataSize).map(_ => r.nextInt().toShort)
withNulls(values).toDF("a")
withNulls(gen.generateShorts(dataSize)).toDF("a")
}

private def generateInts(): DataFrame = {
val r = new Random(0)
val values = Seq(Int.MinValue, Int.MaxValue) ++
Range(0, dataSize).map(_ => r.nextInt())
withNulls(values).toDF("a")
withNulls(gen.generateInts(dataSize)).toDF("a")
}

private def generateLongs(): DataFrame = {
val r = new Random(0)
val values = Seq(Long.MinValue, Long.MaxValue) ++
Range(0, dataSize).map(_ => r.nextLong())
withNulls(values).toDF("a")
withNulls(gen.generateLongs(dataSize)).toDF("a")
}

private def generateDecimalsPrecision10Scale2(): DataFrame = {
Expand Down Expand Up @@ -864,17 +834,6 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
.drop("str")
}

private def generateString(r: Random, chars: String, maxLen: Int): String = {
val len = r.nextInt(maxLen)
Range(0, len).map(_ => chars.charAt(r.nextInt(chars.length))).mkString
}

// TODO return DataFrame for consistency with other generators and include null values
private def generateStrings(chars: String, maxLen: Int): Seq[String] = {
val r = new Random(0)
Range(0, dataSize).map(_ => generateString(r, chars, maxLen))
}

private def generateBinary(): DataFrame = {
val r = new Random(0)
val bytes = new Array[Byte](8)
Expand Down Expand Up @@ -907,28 +866,6 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}
}

// TODO Commented out to work around scalafix since this is currently unused.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this commented out code since it is no longer needed

// private def castFallbackTestTimezone(
// input: DataFrame,
// toType: DataType,
// expectedMessage: String): Unit = {
// withTempPath { dir =>
// val data = roundtripParquet(input, dir).coalesce(1)
// data.createOrReplaceTempView("t")
//
// withSQLConf(
// (SQLConf.ANSI_ENABLED.key, "false"),
// (CometConf.COMET_CAST_ALLOW_INCOMPATIBLE.key, "true"),
// (SQLConf.SESSION_LOCAL_TIMEZONE.key, "America/Los_Angeles")) {
// val df = data.withColumn("converted", col("a").cast(toType))
// df.collect()
// val str =
// new ExtendedExplainInfo().generateExtendedInfo(df.queryExecution.executedPlan)
// assert(str.contains(expectedMessage))
// }
// }
// }

private def castTimestampTest(input: DataFrame, toType: DataType) = {
withTempPath { dir =>
val data = roundtripParquet(input, dir).coalesce(1)
Expand Down
91 changes: 91 additions & 0 deletions spark/src/test/scala/org/apache/comet/DataGenerator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.comet

import scala.util.Random

class DataGenerator(r: Random) {

/** Generate a random string using the specified characters */
def generateString(chars: String, maxLen: Int): String = {
val len = r.nextInt(maxLen)
Range(0, len).map(_ => chars.charAt(r.nextInt(chars.length))).mkString
}

/** Generate random strings */
def generateStrings(n: Int, maxLen: Int): Seq[String] = {
Range(0, n).map(_ => r.nextString(maxLen))
}

/** Generate random strings using the specified characters */
def generateStrings(n: Int, chars: String, maxLen: Int): Seq[String] = {
Range(0, n).map(_ => generateString(chars, maxLen))
}

def generateFloats(n: Int): Seq[Float] = {
Seq(
Float.MaxValue,
Float.MinPositiveValue,
Float.MinValue,
Float.NaN,
Float.PositiveInfinity,
Float.NegativeInfinity,
1.0f,
-1.0f,
Short.MinValue.toFloat,
Short.MaxValue.toFloat,
0.0f) ++
Range(0, n).map(_ => r.nextFloat())
}

def generateDoubles(n: Int): Seq[Double] = {
Seq(
Double.MaxValue,
Double.MinPositiveValue,
Double.MinValue,
Double.NaN,
Double.PositiveInfinity,
Double.NegativeInfinity,
0.0d) ++
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we cover

      1.0d,
       -1.0d,
       Int.MinValue.toDouble,
       Int.MaxValue.toDouble,
       0.0d,
       -0.0d

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. But I think it could be addressed in a follow-up pr.

Range(0, n).map(_ => r.nextDouble())
}

def generateBytes(n: Int): Seq[Byte] = {
Seq(Byte.MinValue, Byte.MaxValue) ++
Range(0, n).map(_ => r.nextInt().toByte)
}

def generateShorts(n: Int): Seq[Short] = {
val r = new Random(0)
Seq(Short.MinValue, Short.MaxValue) ++
Range(0, n).map(_ => r.nextInt().toShort)
}

def generateInts(n: Int): Seq[Int] = {
Seq(Int.MinValue, Int.MaxValue) ++
Range(0, n).map(_ => r.nextInt())
}

def generateLongs(n: Int): Seq[Long] = {
Seq(Long.MinValue, Long.MaxValue) ++
Range(0, n).map(_ => r.nextLong())
}

}
Loading