Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[doris] 类型com.qlangtech.tis.plugin.datax.seq.SeqKey转型错误 #270

Closed
baisui1981 opened this issue Oct 2, 2023 · 0 comments
Closed
Labels
bug Something isn't working
Milestone

Comments

@baisui1981
Copy link
Member

baisui1981 commented Oct 2, 2023

由于扩展点 com.qlangtech.tis.plugin.datax.seq.SeqKey 是在 插件内部tis-datax-doris-plugin定义的 SeqKey.java

异常复现:

  1. 使用OffSeqKey 执行doris 全量导入
  2. 切换使用OnSeqKey执行全量导入 如下图
    截屏2023-10-02 09 54 15

在assemble节点上出现如下错误:

Caused by: com.thoughtworks.xstream.converters.ConversionException: Cannot convert type com.qlangtech.tis.plugin.datax.seq.OnSeqKey to type com.qlangtech.tis.plugin.datax.seq.SeqKey
---- Debugging information ----
message             : Cannot convert type com.qlangtech.tis.plugin.datax.seq.OnSeqKey to type com.qlangtech.tis.plugin.datax.seq.SeqKey
class               : com.qlangtech.tis.plugin.datax.doris.DorisSelectedTab
required-type       : com.qlangtech.tis.plugin.datax.doris.DorisSelectedTab
converter-type      : com.qlangtech.tis.util.XStream2$TISCompositConvert
path                : /com.qlangtech.tis.plugin.KeyedPluginStore/plugins/com.qlangtech.tis.plugin.datax.doris.DorisSelectedTab/seqKey
line number         : 25
class[1]            : java.util.ArrayList
required-type[1]    : java.util.ArrayList
converter-type[1]   : com.thoughtworks.xstream.converters.collections.CollectionConverter
-------------------------------
	at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doUnmarshal(AbstractReflectionConverter.java:452)
	at com.qlangtech.tis.util.XStream2$TISCompositConvert.doUnmarshal(XStream2.java:252)
	at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.unmarshal(AbstractReflectionConverter.java:277)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readBareItem(AbstractCollectionConverter.java:132)
	at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:117)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.addCurrentElementToCollection(CollectionConverter.java:99)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:92)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.populateCollection(CollectionConverter.java:86)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:81)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.qlangtech.tis.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:402)
	at com.qlangtech.tis.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:344)
	at com.qlangtech.tis.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:278)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1421)
	at com.qlangtech.tis.util.XStream2.unmarshal(XStream2.java:70)
	at com.qlangtech.tis.extension.impl.XmlFile.unmarshal(XmlFile.java:119)
	at com.qlangtech.tis.plugin.PluginStore.load(PluginStore.java:399)
	... 29 common frames omitted

故障定位

参考单元测试,TestDorisSelectedTab.java

由于执行了 TIS.clean(),导致 pluginManager中UberClassloader会重新初始化,导致之前已经加载的class会被重新加载,在内存中生成两个版本的class(虽然类名是相同的)

解决办法

对TIS.clean() 进行重载,引入参数 removeTIS:boolean,清除TIS过程中是否删除TIS实例。调用端可使用如下方法:

TIS.clean(false)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant