[SPARK-40714][SQL] Remove PartitionAlreadyExistsException

[MaxGekk]

What changes were proposed in this pull request?

In the PR, I propose to remove PartitionAlreadyExistsException and use PartitionsAlreadyExistException instead of it.

Why are the changes needed?

1. To simplify user apps. After the changes, users don't need to catch both exceptions PartitionsAlreadyExistException as well as PartitionAlreadyExistsException .
2. To improve code maintenance since don't need to support almost the same code.
3. To avoid errors like the PR [SPARK-40702][SQL] Fix partition specs in PartitionsAlreadyExistException #38152 fixed PartitionsAlreadyExistException but not PartitionAlreadyExistsException.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

By running the affected test suites:

$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *SupportsPartitionManagementSuite"
$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *.AlterTableAddPartitionSuite"

[LuciferYang]

+1,LGTM

[dongjoon-hyun]

Although these interfaces are Experimental, this looks like a big change on SupportsAtomicPartitionManagement and SupportsPartitionManagement.

[cloud-fan]

Does Spark try-catch the errors thrown by these 2 APIs? If yes then we may have bugs as the existing data source implementations may still throw PartitionAlreadyExistsException

[MaxGekk]

In the current implementation, Spark invokes createPartition()/createPartitions() from AddPartitionExec (ALTER TABLE .. ADD PARTITION) where all exceptions from the methods are propagated to users directly. But it doesn't matter in our case because AddPartitionExec.run() checks partitions exist BEFORE the invokes, see:

spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/AddPartitionExec.scala

Lines 41 to 47 in 5d74ace

	val (existsParts, notExistsParts) =
	partSpecs.partition(p => table.partitionExists(p.ident))
	if (existsParts.nonEmpty && !ignoreIfExists) {
	throw new PartitionsAlreadyExistException(
	table.name(), existsParts.map(_.ident), table.partitionSchema())
	}

So, this PR doesn't affect ALTER TABLE .. ADD PARTITION but it affects v2 ALTER TABLE .. RENAME PARTITION which propagates the exception directly to users.

[cloud-fan]

I think in this case it's OK to change the type of the exception that is propagated to users.

[MaxGekk]

this looks like a big change on SupportsAtomicPartitionManagement and SupportsPartitionManagement.

I think it is right time to change the APIs since they are not broadly used so far. I guess, it will be difficult to predict which of PartitionAlreadyExistsException and PartitionsAlreadyExistException to expect from particular Spark DDL, and user apps will be written in the way to handle both exceptions. I think we should avoid this from the beginning, and don't introduce unnecessary complexity in the user code.

[MaxGekk]

Merging to master. Thank you, @LuciferYang @cloud-fan @dongjoon-hyun for review.