Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIVE-28059: Iceberg REST Catalog #5606

Merged
merged 54 commits into from
Mar 10, 2025
Merged
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
26d22d0
HIVE-28059 : major rebase stage 1;
henrib Jan 10, 2025
a573191
Merge branch 'master' of https://github.com/apache/hive
henrib Jan 31, 2025
23a3c90
HIVE-28059 : major rebase and simpler code;
henrib Jan 31, 2025
e90e1d8
HIVE-28059 : rollback changes that are unnecessary;
henrib Feb 3, 2025
f7da91b
HIVE-28059 : rollback changes that are unnecessary;
henrib Feb 3, 2025
ef8b220
Update iceberg_stats.q.out
henrib Feb 3, 2025
4ee1720
HIVE-28059 : increase test server startup wait time;
henrib Feb 4, 2025
8a09245
HIVE-28059 : attempting a rename on artefactId to blindfix build issue;
henrib Feb 4, 2025
92b9883
HIVE-28059 : fixing src packaging;
henrib Feb 6, 2025
822ad44
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 6, 2025
bc36af2
HIVE-28059 : fixing PR comments;
henrib Feb 11, 2025
399e25b
HIVE-28059 : fixing more PR comments (conf, pom);
henrib Feb 11, 2025
cfe85d2
HIVE-28059 : refactored cache;
henrib Feb 12, 2025
81db7e4
HIVE-28059 : remove dead class & code;
henrib Feb 12, 2025
931eff5
HIVE-28059 : remove dead code;
henrib Feb 12, 2025
37ae8f3
HIVE-28059 : moving configuration variables to the common place;
henrib Feb 12, 2025
c3c71c0
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 12, 2025
1ab4d94
HIVE-28059 : default servlet path update (iceberg);
henrib Feb 13, 2025
7cb3ce2
HIVE-28059 : pom cleanup;
henrib Feb 13, 2025
33a9cff
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 13, 2025
cae7f2f
HIVE-28059 : renamed & commented test helper class
henrib Feb 13, 2025
a4d66fd
HIVE-28059 : removed useless synchronized;
henrib Feb 13, 2025
b83cfaf
HIVE-28059 : removed SecureServletCaller;
henrib Feb 14, 2025
e79c155
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 14, 2025
107e423
HIVE-28059 : nit, javadoc;
henrib Feb 15, 2025
4921457
HIVE-28059 : nit, javadoc (again!);
henrib Feb 15, 2025
9b7eb15
HIVE-28059 : javadoc (again!!);
henrib Feb 16, 2025
4c41e9c
HIVE-28059 : fix regression on PropertyServlet & test;
henrib Feb 16, 2025
507dd6e
HIVE-28059 : fix regression on PropertyServlet test;
henrib Feb 17, 2025
cc9f4fd
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 17, 2025
edd7148
HIVE-28059 : fix nits;
henrib Feb 18, 2025
0190228
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 19, 2025
0a6381c
HIVE-28059 : fixing more nits;
henrib Feb 19, 2025
2be9327
Javadoc nit;
henrib Feb 19, 2025
4750304
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 21, 2025
6c2b930
HIVE-28059 : create only one Jetty instance for HMS property & Iceber…
henrib Feb 21, 2025
11c9718
HIVE-28059 :
henrib Feb 21, 2025
0b1add3
HIVE-28059: Update src.xml
henrib Feb 21, 2025
670ab10
HIVE-28059 :
henrib Feb 21, 2025
e84c069
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 23, 2025
77fb867
HIVE-28059 : fixing property servlet tests;
henrib Feb 23, 2025
447f48d
HIVE-28059 : adding REST catalog to packaging;
henrib Feb 24, 2025
b99d91a
HIVE-28059 : addressing easiest pr comments;
henrib Feb 26, 2025
70b8c75
HIVE-28059 : fix packaging;
henrib Feb 27, 2025
e1675bb
Merge branch 'apache:master' into HMS-Catalog
henrib Feb 27, 2025
1efdf6f
Merge branch 'apache:master' into HMS-Catalog
henrib Mar 2, 2025
7f75025
HIVE-28059 : fix jar naming;
henrib Mar 2, 2025
4e09a68
HIVE-28059 : fixing nits & javadoc;
henrib Mar 2, 2025
370a0e0
HIVE-28059 : fixing javadoc;
henrib Mar 3, 2025
edcaea3
Merge branch 'apache:master' into HMS-Catalog
henrib Mar 6, 2025
d44bfe0
HIVE-28059 : latest nits (servlet names, etc);
henrib Mar 6, 2025
1e7a8da
HIVE-28059 : fixing thrift over http servlet init;
henrib Mar 7, 2025
81041bb
Merge branch 'apache:master' into HMS-Catalog
henrib Mar 7, 2025
f0977c3
Merge branch 'apache:master' into HMS-Catalog
henrib Mar 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
HIVE-28059 : fixing more nits;
- moving to metastore-iceberg-catalog;
- removing never thrown exceptions from signatures;
- fixing javadoc;
henrib committed Feb 19, 2025
commit 0a6381cfcde34d9206e54f86c3c7437712c1f5b9
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@
<version>4.1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>hive-standalone-metastore-icecat</artifactId>
<artifactId>hive-standalone-metastore-iceberg-catalog</artifactId>
<name>Hive Metastore Iceberg Catalog</name>
<properties>
<standalone.metastore.path.to.root>..</standalone.metastore.path.to.root>
@@ -59,6 +59,21 @@
<artifactId>iceberg-bundled-guava</artifactId>
<version>${iceberg.version}</version>
</dependency>
<!-- Intellij dislikes shaded Iceberg: uncomment the following 2 blocks helps when coding/debugging -->
<!--
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-api</artifactId>
<version>${iceberg.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
<version>${iceberg.version}</version>
<optional>true</optional>
</dependency>
-->
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-standalone-metastore-common</artifactId>
Original file line number Diff line number Diff line change
@@ -77,7 +77,7 @@
import org.apache.iceberg.util.PropertyUtil;

/**
* Original @ https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java
* Original @ https://github.com/apache/iceberg/blob/1.6.x/core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java
* Adaptor class to translate REST requests into {@link Catalog} API calls.
*/
public class HMSCatalogAdapter implements RESTClient {
@@ -696,10 +696,8 @@ public <T extends RESTResponse> T postForm(
}

@Override
public void close() throws IOException {
// The calling test is responsible for closing the underlying catalog backing this REST catalog
// so that the underlying backend catalog is not closed and reopened during the REST catalog's
// initialize method when fetching the server configuration.
public void close() {
// The caller is responsible for closing the underlying catalog backing this REST catalog.
}

private static class NamespaceNotSupported extends RuntimeException {
Original file line number Diff line number Diff line change
@@ -131,6 +131,14 @@ protected Server createServer() {
* @throws Exception if servlet initialization fails
*/
public static Server startServer(Configuration conf) throws Exception {
return new HMSCatalogServer(conf, null).startServer();
Server server = new HMSCatalogServer(conf, null).startServer();
if (server != null) {
if (!server.isStarted()) {
LOG.error("Unable to start property-maps servlet server on {}", server.getURI());
} else {
LOG.info("Started property-maps servlet server on {}", server.getURI());
}
}
return server;
}
}
Original file line number Diff line number Diff line change
@@ -29,7 +29,6 @@
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collectors;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@@ -45,7 +44,7 @@
import org.slf4j.LoggerFactory;

/**
* Original @ https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/rest/RESTCatalogServlet.java
* Original @ https://github.com/apache/iceberg/blob/1.6.x/core/src/test/java/org/apache/iceberg/rest/RESTCatalogServlet.java
* The RESTCatalogServlet provides a servlet implementation used in combination with a
* RESTCatalogAdaptor to proxy the REST Spec to any Catalog implementation.
*/
@@ -60,7 +59,7 @@ public HMSCatalogServlet(HMSCatalogAdapter restCatalogAdapter) {
}

@Override
protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
protected void service(HttpServletRequest request, HttpServletResponse response) {
try {
ServletRequestContext context = ServletRequestContext.from(request);
response.setStatus(HttpServletResponse.SC_OK);
Original file line number Diff line number Diff line change
@@ -37,6 +37,7 @@
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.StandardCharsets;
@@ -433,15 +434,7 @@ static <T> String serialize(T object) {
}
}

static <T> T deserialize(String s) {
try {
return MAPPER.readValue(s, new TypeReference<T>() {});
} catch (JsonProcessingException xany) {
throw new RuntimeException(xany);
}
}

static <T> T deserialize(BufferedReader s) {
static <T> T deserialize(Reader s) {
try {
return MAPPER.readValue(s, new TypeReference<T>() {});
} catch (IOException xany) {
Original file line number Diff line number Diff line change
@@ -336,7 +336,7 @@ protected int getServerPort() {
}

@Override
protected HttpServlet createServlet() throws IOException {
protected HttpServlet createServlet() {
ServletSecurity security = new ServletSecurity(configuration, PropertyServlet.isAuthJwt(configuration));
return security.proxy(new PropertyServlet(configuration));
}
Original file line number Diff line number Diff line change
@@ -53,7 +53,7 @@
* </p>
* A typical usage in a servlet is the following:
* <pre><code>
* SecureServletCaller security; // ...
* ServletSecurity security; // ...
* {@literal @}Override protected void doPost(HttpServletRequest request, HttpServletResponse response)
* throws ServletException, IOException {
* security.execute(request, response, this::runPost);
2 changes: 1 addition & 1 deletion standalone-metastore/pom.xml
Original file line number Diff line number Diff line change
@@ -29,7 +29,7 @@
<module>metastore-common</module>
<module>metastore-server</module>
<module>metastore-tools</module>
<module>metastore-catalog</module>
<module>metastore-iceberg-catalog</module>
</modules>
<properties>
<hive.version>4.1.0-SNAPSHOT</hive.version>