JDBC support #283
JDBC support #283
Conversation
|
good job ! |
1 similar comment
|
good job ! |
|
I run it ,but error occurs below, is there anybody can help me ? |
|
elasticsearch dependency in pom.xml is declared |
|
@allwefantasy many thanks, I have add elasticsearch dependency, and that problem seems solved. But another error happends: have any idea? ps: I have found the solution to the problem: down the druid version to 1.0.15; thans, I can run it successful now! |
|
Good job |
|
@allwefantasy I see you have used TransportClient in ElasticSearchConnection.java; and this is a natural way, cause mysql protocol tcp is TCP based, and so transportClient. But there is a limitation: users can only connect to the transport port (default 9300), other than the http port(9200). The limitation is more obvious when in the product environment where many companies demand that searching must be proxyed by Nginx. Have any idea? |
|
@allwefantasy If I want to use ‘join‘, how to page？'LIMIT' cannot do that，for example: |
|
We don't support paging /scrolling for not native elasticsearch queries now |
|
I want to know can the feather support cluster? |
|
yes, you can do that,for example(elasticsearch.yml): |
|
The es-sql website can run normally |
|
@will0815 Have you modified the JDBC URL eg. |
|
good job |
|
How to driver? |
|
It seems like it misses the java.sql.Driver implementation. |
|
when i want to select from a type,just like: 'select * from index1.type2 where xxx ' , how can i use this jdbc method? But the example only show how to select from index. thx~ |
|
@RaniRaven |
|
@stopit can you try |
|
good job ! |
|
I fail to understand ... |
|
At first,I think can't support it elasticsearch 5.1.2. |
|
@allwefantasy How to support the sql like And I get the Exception: thx! detail info #400 |
|
jdbc对于es7.1.1支持吗？因为我看es7已经关闭了对于transport的连接方式，我个人还是尝试了一下用transport去连接，导入了很多依赖的jar包，但是最后还是没有成功，提示连接关闭或无效，不知道对于es7.x有没有什么特殊配置，还是根本不支持？ |
|
PreBuiltXPackTransportClient已经废弃了，要是用的话得申请证书并做下配置：#383 正考虑着切换到highlevelrestclient |
|
@allwefantasy 你好，请教个问题，我在试用sql查询时，如果sql中试用sum等运算是，无法返回结果，如下图 如果我使用http://host:9200/_xpack/sql?format=json，直接使用图中的sql也是没有问题的， 请问这个问题怎么解决 |
|
这个字段类型是text应该不能sum，你要改下mapping吧
…---Original---
From: "bjzhenglq"<notifications@github.com>
Date: Wed, Jun 10, 2020 17:20 PM
To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>;
Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>;
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
@allwefantasy 你好，请教个问题，我在试用sql查询时，如果sql中试用sum等运算是，无法返回结果，如下图
会提示“Fielddata is disabled on text fields by default”，如果是直接sql查询没有问题，String sql = "SELECT name , score FROM score_list_202006";
如果我使用http://host:9200/_xpack/sql?format=json，直接使用图中的sql也是没有问题的，
请问这个问题怎么解决
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
说错了等我查一下啊
…---Original---
From: "bjzhenglq"<notifications@github.com>
Date: Wed, Jun 10, 2020 17:20 PM
To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>;
Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>;
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
@allwefantasy 你好，请教个问题，我在试用sql查询时，如果sql中试用sum等运算是，无法返回结果，如下图
会提示“Fielddata is disabled on text fields by default”，如果是直接sql查询没有问题，String sql = "SELECT name , score FROM score_list_202006";
如果我使用http://host:9200/_xpack/sql?format=json，直接使用图中的sql也是没有问题的，
请问这个问题怎么解决
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
这个字段实际上是个整型的，直接rest接口形式是可以sum的，但是图片中的就是不行 |
|
我没怎么用过sql都是用dsl，那会不会是建立的es链接不是rest请求我看你用的连接池，可能是transport请求呢
…---Original---
From: "bjzhenglq"<notifications@github.com>
Date: Wed, Jun 10, 2020 17:30 PM
To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>;
Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>;
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
这个字段类型是text应该不能sum，你要改下mapping吧
…
---Original--- From: "bjzhenglq"<notifications@github.com> Date: Wed, Jun 10, 2020 17:20 PM To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>; Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283) @allwefantasy 你好，请教个问题，我在试用sql查询时，如果sql中试用sum等运算是，无法返回结果，如下图 会提示“Fielddata is disabled on text fields by default”，如果是直接sql查询没有问题，String sql = "SELECT name , score FROM score_list_202006"; 如果我使用http://host:9200/_xpack/sql?format=json，直接使用图中的sql也是没有问题的， 请问这个问题怎么解决 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.
这个字段实际上是个整型的，直接rest接口形式是可以sum的，但是图片中的就是不行
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
你看下连接池底层请求是不是http吧
…---Original---
From: "bjzhenglq"<notifications@github.com>
Date: Wed, Jun 10, 2020 17:30 PM
To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>;
Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>;
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
这个字段类型是text应该不能sum，你要改下mapping吧
…
---Original--- From: "bjzhenglq"<notifications@github.com> Date: Wed, Jun 10, 2020 17:20 PM To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>; Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283) @allwefantasy 你好，请教个问题，我在试用sql查询时，如果sql中试用sum等运算是，无法返回结果，如下图 会提示“Fielddata is disabled on text fields by default”，如果是直接sql查询没有问题，String sql = "SELECT name , score FROM score_list_202006"; 如果我使用http://host:9200/_xpack/sql?format=json，直接使用图中的sql也是没有问题的， 请问这个问题怎么解决 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.
这个字段实际上是个整型的，直接rest接口形式是可以sum的，但是图片中的就是不行
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
检查下，name字段是不是text类型的，对text类型字段做聚合或者排序是需要开启Fielddata的。 |
|
preparedStatement支持set参数吗？ sql = "select * from person where name = ?";
statement = connection.prepareStatement(sql);
statement.setString(1, "小明");
resultSet = statement.executeQuery();会报错 直接查sql="select * from person where name = '小明'"没有问题。 |
|
应该不能这样写吧，那个是mysql的预编译，这个是es不一样的，应该用java吧sql语句编好了，然后掉方法查询，这个里面的方法会把sql转换成es查询语句去查询
…---Original---
From: "石门小张"<notifications@github.com>
Date: Sun, Sep 27, 2020 15:23 PM
To: "NLPchina/elasticsearch-sql"<elasticsearch-sql@noreply.github.com>;
Cc: "Comment"<comment@noreply.github.com>;"ananxuan"<493513100@qq.com>;
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
preparedStatement支持set参数吗？
sql = "select * from person where name = ?"; statement = connection.prepareStatement(sql); statement.setString(1, "小明"); resultSet = statement.executeQuery();
会报错
Caused by: org.nlpcn.es4sql.exception.SqlParseException: Failed to parse SqlExpression of type class com.alibaba.druid.sql.ast.expr.SQLVariantRefExpr. expression value: ? at org.nlpcn.es4sql.parse.WhereParser.parseValue(WhereParser.java:616) at org.nlpcn.es4sql.parse.WhereParser.explanCond(WhereParser.java:266) at org.nlpcn.es4sql.parse.WhereParser.parseWhere(WhereParser.java:93) at org.nlpcn.es4sql.parse.WhereParser.findWhere(WhereParser.java:64) at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:91) at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:63) at org.nlpcn.es4sql.query.ESActionFactory.create(ESActionFactory.java:65) at org.nlpcn.es4sql.SearchDao.explain(SearchDao.java:46) at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:92) at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:40) ... 94 common frames omitted
直接查sql="select * from person where name = '小明'"没有问题。
druid版本1.1.16，试了最新的1.1.24也不行，这种情况有什么解决方法吗？
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
支持预编译的： |
|
原来使用的是6.8.2.0，已升级，感谢。
smile_zchi@163.com
From: shiyuan
Date: 2020-10-25 21:51
To: NLPchina/elasticsearch-sql
CC: 石门小张; Comment
Subject: Re: [NLPchina/elasticsearch-sql] JDBC support (#283)
preparedStatement支持set参数吗？
sql = "select * from person where name = ?";
statement = connection.prepareStatement(sql);
statement.setString(1, "小明");
resultSet = statement.executeQuery();
会报错
Caused by: org.nlpcn.es4sql.exception.SqlParseException: Failed to parse SqlExpression of type class com.alibaba.druid.sql.ast.expr.SQLVariantRefExpr. expression value: ?
at org.nlpcn.es4sql.parse.WhereParser.parseValue(WhereParser.java:616)
at org.nlpcn.es4sql.parse.WhereParser.explanCond(WhereParser.java:266)
at org.nlpcn.es4sql.parse.WhereParser.parseWhere(WhereParser.java:93)
at org.nlpcn.es4sql.parse.WhereParser.findWhere(WhereParser.java:64)
at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:91)
at org.nlpcn.es4sql.parse.SqlParser.parseSelect(SqlParser.java:63)
at org.nlpcn.es4sql.query.ESActionFactory.create(ESActionFactory.java:65)
at org.nlpcn.es4sql.SearchDao.explain(SearchDao.java:46)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.getObjectResult(ElasticSearchDruidPooledPreparedStatement.java:92)
at com.alibaba.druid.pool.ElasticSearchDruidPooledPreparedStatement.executeQuery(ElasticSearchDruidPooledPreparedStatement.java:40)
... 94 common frames omitted
直接查sql="select * from person where name = '小明'"没有问题。
druid版本1.1.16，试了最新的1.1.24也不行，这种情况有什么解决方法吗？
支持预编译的：
2f581c2
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
我的依赖是这样 测试类如下： 一直报下面的错，请问是哪的原因呢（服务器es版本为7.10.2） |
|
包版本不一致，x-pack-transport是7.10.2，但elasticsearch是7.6.2 |
|
@shi-yuan 可以了，感谢 |
|
7.16.2 什么时候出来？ |
|
|
@shi-yuan 您好，使用jdbc查询，不支持 group by filters，是我使用的版本有问题吗？ |
|
这里支持的，7.8.0.1还真不支持，升下级吧 |
|
elasticsearch-sql可以将mysql语句转成highlevelrestclient的api吗？ |
|
@test 一直报这个错 java.sql.SQLException: Error Caused by: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{K9EuakKZQm6PSSU9yLXa8w}{127.0.0.1}{127.0.0.1:9200}]] |
确定是默认的9200，而不是默认的9300？ |
|
@shi-yuan Properties properties = new Properties(); org.elasticsearch.transport.RemoteTransportException: [ssizsAe][127.0.0.1:9300][cluster:monitor/nodes/liveness] |
|
es开启了密码认证就要开启ssl。jdbc用的是tcp协议。要添加ssl认证 String usernamePassword = pool.getUsername() + ":" + pool.getPassword();
String xPackStr = "xpack.security.user=" + usernamePassword + ";" +
"xpack.security.transport.ssl.enabled=true;" +
"xpack.security.transport.ssl.keystore.path=" + keystorePath + ";" +
"xpack.security.transport.ssl.keystore.password=" + caPassword + ";" +
"xpack.security.transport.ssl.verification_mode=" + verification_mode;
properties.put(PROP_CONNECTIONPROPERTIES, xPackStr); |
|
谢谢，解决了，现在就是想融合mybatis去执行sql，报以下问题 |
|
请问能不能支持es 9200端口的访问呢 |
|
最新版8.5.3使用的elasticsearch-java客户端，以http的方式，9200端口 @Test
public void testJDBCWithParameter() throws Exception {
try (DruidDataSource dds = (DruidDataSource) ElasticSearchDruidDataSourceFactory.createDataSource(createElasticsearchClient());
Connection connection = dds.getConnection();
PreparedStatement ps = connection.prepareStatement("SELECT gender,lastname,age from " + TestsConstants.TEST_INDEX_ACCOUNT + " where lastname=?")) {
// set parameter
ps.setString(1, "Heath");
ResultSet resultSet = ps.executeQuery();
ResultSetMetaData metaData = resultSet.getMetaData();
assertThat(metaData.getColumnName(1), equalTo("gender"));
assertThat(metaData.getColumnName(2), equalTo("lastname"));
assertThat(metaData.getColumnName(3), equalTo("age"));
List<String> result = new ArrayList<>();
while (resultSet.next()) {
result.add(resultSet.getString("lastname") + "," + resultSet.getInt("age") + "," + resultSet.getString("gender"));
}
Assert.assertEquals(1, result.size());
Assert.assertEquals("Heath,39,F", result.get(0));
}
}
private Client createElasticsearchClient() throws Exception {
return new ElasticsearchRestClient(new ElasticsearchClient(getElasticsearchTransport(getRestClient())));
}
private RestClient getRestClient() throws Exception {
InetSocketAddress address = getTransportAddress().address();
String hostPort = String.format("http://%s:%s", address.getHostString(), address.getPort());
RestClientBuilder builder = RestClient.builder(HttpHost.create(hostPort));
builder.setHttpClientConfigCallback(clientBuilder -> {
RequestConfig.Builder requestConfigBuilder = RequestConfig.custom();
requestConfigBuilder.setConnectTimeout(10 * 1000);
int socketTimeout = 90 * 1000;
requestConfigBuilder.setSocketTimeout(socketTimeout);
requestConfigBuilder.setConnectionRequestTimeout(socketTimeout);
clientBuilder.setDefaultRequestConfig(requestConfigBuilder.build());
return clientBuilder;
});
return builder.build();
}
private TransportAddress getTransportAddress() throws Exception {
String host = System.getenv("ES_TEST_HOST");
String port = System.getenv("ES_TEST_PORT");
if (host == null) {
host = "localhost";
System.out.println("ES_TEST_HOST enviroment variable does not exist. choose default 'localhost'");
}
if (port == null) {
port = "9200";
System.out.println("ES_TEST_PORT enviroment variable does not exist. choose default '9200'");
}
System.out.println(String.format("Connection details: host: %s. port:%s.", host, port));
return new TransportAddress(InetAddress.getByName(host), Integer.parseInt(port));
}
private ElasticsearchTransport getElasticsearchTransport(RestClient restClient) {
TransportOptions.Builder transportOptionsBuilder = new RestClientOptions(RequestOptions.DEFAULT).toBuilder();
ContentType jsonContentType = Version.VERSION == null ? ContentType.APPLICATION_JSON
: ContentType.create("application/vnd.elasticsearch+json",
new BasicNameValuePair("compatible-with", String.valueOf(Version.VERSION.major())));
Consumer<String> setHeaderIfNotPresent = header -> {
if (transportOptionsBuilder.build().headers().stream().noneMatch((h) -> h.getKey().equalsIgnoreCase(header))) {
transportOptionsBuilder.addHeader(header, jsonContentType.toString());
}
};
setHeaderIfNotPresent.accept("Content-Type");
setHeaderIfNotPresent.accept("Accept");
TransportOptions transportOptionsWithHeader = transportOptionsBuilder.build();
return new RestClientTransport(restClient, new JacksonJsonpMapper(), transportOptionsWithHeader);
} |
Fix Issue: #28
Example
Unit test