Spark: Add view support to SparkSessionCatalog #11388
Conversation
nastra
force-pushed
the
spark-session-catalog-with-views
branch
4 times, most recently
from
13f39eb to
47ac7f9
Compare
nastra
force-pushed
the
spark-session-catalog-with-views
branch
from
47ac7f9 to
c89d454
Compare
nastra
force-pushed
the
spark-session-catalog-with-views
branch
4 times, most recently
from
e0eb166 to
1401a5f
Compare
open-api/src/testFixtures/java/org/apache/iceberg/rest/RCKUtils.java
Outdated
Show resolved
Hide resolved
open-api/src/testFixtures/java/org/apache/iceberg/rest/RESTCatalogServer.java
Outdated
Show resolved
Hide resolved
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/BaseCatalog.java
Outdated
Show resolved
Hide resolved
nastra
force-pushed
the
spark-session-catalog-with-views
branch
from
1401a5f to
2d8e5de
Compare
nastra
force-pushed
the
spark-session-catalog-with-views
branch
from
2d8e5de to
37f6c7b
Compare
| properties); | ||
| } else if (isViewCatalog()) { | ||
| try { | ||
| getSessionCatalog().dropView(ident); |
There was a problem hiding this comment.
I'm not sure if we want to allow this. If a catalog implementation doesn't support replace view, this could technically be a lossy operation. For example, if someone has DROP but not CREATE permissions, then we would drop the view and not be able to create leaving this in a bad state. I feel like it's safer to just fail and require that the user perform the drop.
There was a problem hiding this comment.
this was mostly following Spark's example in https://github.com/apache/spark/blob/6cd13344bffd411289ec20170ae2da2035bb2859/sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/ViewCatalog.java#L124-L152 but I agree that we maybe don't want to do this non-atomically. If needed, we can always add that back later
nastra
force-pushed
the
spark-session-catalog-with-views
branch
from
730cf39 to
252de8b
Compare
* Spark: Add view support to SparkSessionCatalog * Don't replace view non-atomically
No description provided.