Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [ErrorDataSinkExecutor] 扩展达梦数据库存储错误数据至 Mysql 时,使用了源数据源的 Dialect 实现类而非错误数据源的 Dialect 实现类拼接 SQL 导致执行错误 #481

Open
1 of 2 tasks
AlexChen68 opened this issue Nov 12, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@AlexChen68
Copy link
Contributor

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

我的需求是检测达梦数据库并将错误数据存储下来,但是我发现达梦数据库默认不支持存储错误数据至 Mysql,因此我通过手动修改 DmDialectinvalidateItemCanOutput 方法来开启;

在这过程中,我发现存储错误数据需要新建错误数据表以及插入错误数据,它们的 SQL 在拼接时,使用到了 DmDialectquoteIdentifier 方法来包装数据库字段;但是通过 debug 发现,获取的 Dialect 类型为数据来源的实现类,而不是用来存储错误数据的实现类,如下图:

issues_1

这会导致新建数据表和插入数据的 SQL 使用 DM 的 quoteIdentifier 方法拼接字段,而拼接完的 SQL 无法在 Mysql 执行,是否有办法获取错误数据源的 Dialect 实现类?

DataVines Version

dev 最新版本

DataVines Config

Mysql8 作为元数据库;
DM8 作为待检测的数据源;
Mysql8 作为错误数据存储源。

Running Command

使用本地 idea 启动

Error Exception

[ERROR] 2024-11-12 17:14:02.920 [JobExecutionUniqueId=JOB-20241112-client_local_22:[306] - sink error data error : 
java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"id" BIGINT,"name" TEXT,"create_user" BIGINT,"create_time" TEXT,"update_user" BI' at line 1
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
	at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:782)
	at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:666)
	at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:94)
	at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
	at io.datavines.engine.local.connector.executor.ErrorDataSinkExecutor.createTable(ErrorDataSinkExecutor.java:344)
	at io.datavines.engine.local.connector.executor.ErrorDataSinkExecutor.sinkErrorData(ErrorDataSinkExecutor.java:170)
	at io.datavines.engine.local.connector.executor.ErrorDataSinkExecutor.execute(ErrorDataSinkExecutor.java:71)
	at io.datavines.engine.local.connector.BaseJdbcSink.output(BaseJdbcSink.java:98)
	at io.datavines.engine.local.api.LocalExecution.execute(LocalExecution.java:146)
	at io.datavines.engine.core.BaseDataVinesBootstrap.parseConfigAndExecute(BaseDataVinesBootstrap.java:79)
	at io.datavines.engine.core.BaseDataVinesBootstrap.execute(BaseDataVinesBootstrap.java:43)
	at io.datavines.engine.local.executor.LocalEngineExecutor.execute(LocalEngineExecutor.java:46)
	at io.datavines.server.dqc.executor.runner.JobRunner.run(JobRunner.java:67)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

Engine Type

Java Version

JDK8

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@AlexChen68 AlexChen68 added the bug Something isn't working label Nov 12, 2024
@AlexChen68 AlexChen68 changed the title [Bug] [ErrorDataSinkExecutor] 扩展达梦数据库存储错误数据至 Mysql 时,使用了 DM 的 Dialect 而非 Mysql 的 Dialect 拼接 SQL 导致执行错误 [Bug] [ErrorDataSinkExecutor] 扩展达梦数据库存储错误数据至 Mysql 时,使用了源数据源的 Dialect 实现类而非错误数据源的 Dialect 实现类拼接 SQL 导致执行错误 Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant