We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Running ALTER TABLE table_name SET TBLPROPERTIES () does not persist the changes in the _delta_log/.
ALTER TABLE table_name SET TBLPROPERTIES ()
_delta_log/.
import org.apache.spark.sql.delta.actions.Action import org.apache.spark.sql.delta.DeltaLog import org.apache.spark.sql.delta.actions.CommitInfo import org.apache.spark.sql.delta.util.FileNames val df = loadTestData(spark) val tmpDir = "/tmp/test" df.write.format("qbeast").option("columnsToIndex", "user_id,price").save(tmpDir) spark.sql(s"CREATE TABLE t1 USING QBEAST LOCATION $tmpDir") spark.sql("ALTER TABLE t1 SET TBLPROPERTIES ('k' = 'v')") val deltaLog = DeltaLog.forTable(spark, tablePath) val conf = deltaLog.newDeltaHadoopConf() val snapshot = deltaLog.update() val v = snapshot.version val isCommitted = (deltaLog.store .read(FileNames.deltaFile(deltaLog.logPath, v), conf) .map(Action.fromJson) .exists { case c: CommitInfo => println(c) c.operation == "SET TBLPROPERTIES" && c.operationParameters == Map("properties" -> """{"k":"v"}""") case _ => false } ) isCommitted shouldBe true
The text was updated successfully, but these errors were encountered:
osopardo1
Successfully merging a pull request may close this issue.
What went wrong?
Running
ALTER TABLE table_name SET TBLPROPERTIES ()
does not persist the changes in the_delta_log/.
How to reproduce?
The text was updated successfully, but these errors were encountered: