Spark: Deprecate SparkAppenderFactory

[ajantha-bhat]

It is unused currently and Spark integration uses SparkFileWriterFactory instead of this.

[ajantha-bhat]

While working on partition stats, I came across FileWriterFactory and AppenderFactory. After digging a bit, found that Spark was using AppenderFactory before but now uses FileWriterFactory. But the implementation of AppenderFactory still exist and confusing for users. Hence, deprecating it before removal in the next version.

[ajantha-bhat]

Also, this class is not used by the Spark engine integration. Hence, we can safely remove it.

Looking more into it, we might have lost metrics related function in FileWriterFactory that was present in AppenderFactory. Maybe I can work on it in the future for FileWriterFactory.

[amoghjaha-db]

Since the class is package private do we want to just remove it upfront? It looks like it's currently used in some JMH benchmarks WritersBenchmark, and TestSparkMergingMetrics would need to be updated to test the SparkFileWriterFactory class instead.

[ajantha-bhat]

@amoghjaha-db:

Since the class is package private do we want to just remove it upfront

I did get this thought initially and checked how we handled previously for other classes in this module. The package private classes like SparkFileScan was deprecated first and then removed. So, I went ahead and followed the same.

It looks like it's currently used in some JMH benchmarks WritersBenchmark, and TestSparkMergingMetrics would need to be updated to test the SparkFileWriterFactory class instead.

As I mentioned in one of the comments, SparkFileWriterFactory doesn't have metrics related function. So, we cannot update the test to use it. We have to delete the tests as well.

[aokolnychyi]

This is indeed our legacy writers. We can deprecate and then remove to be safe.

[aokolnychyi]

Thanks, @ajantha-bhat!