Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hadoop3.x版本,hdfs connector不支持orc #565

Closed
liukunyuan opened this issue Dec 15, 2021 · 4 comments
Closed

hadoop3.x版本,hdfs connector不支持orc #565

liukunyuan opened this issue Dec 15, 2021 · 4 comments
Labels
bug Something isn't working

Comments

@liukunyuan
Copy link

因为hive的版本是1.1.1,不支持3.x版本的hadoop,会失败
image

@liukunyuan liukunyuan added the bug Something isn't working label Dec 15, 2021
@liukunyuan
Copy link
Author

image

@jctanking
Copy link

我也遇到这个问题非常棘手

@liumengkai
Copy link
Contributor

@liukunyuan @jctanking
我们尝试出来一种方法是修改hdfs-connector中的hive版本,然后修改一下其中的方法调用参数
将hdfs-connector 中 Hive version修改为3.1.0

<!--		<hive.version>1.1.1</hive.version>-->
		<hive.version>3.1.0</hive.version>

修改HdfsOrcInputSplit.class

OrcSplit orcSplit = new OrcSplit(null, 0, 0, null, null, false, false, new ArrayList<>());

修改为

OrcSplit orcSplit = new OrcSplit(null, null, 0, 0, null, null, false, false, new ArrayList<>(), 0, 0, null);

然后重新编译运行即可

@FlechazoW
Copy link
Member

PR merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants