Skip to content

Commit

Permalink
[KYUUBI-SHADED #49] Create relocated package for spark-connect-rpc
Browse files Browse the repository at this point in the history
### _Why are the changes needed?_

This PR introduces a new module `kyuubi-relocated-spark-connect-rpc`, which forks [Spark Connect proto files](https://github.com/apache/spark/tree/5b2d2149b615acdd8730547a1f24c2b637222545/sql/connect/common/src/main/protobuf) and shifts the package name from `org.apache.spark.connect.proto` to `org.apache.kyuubi.shaded.spark.connect.proto`, this is a precondition for Kyuubi to support Spark Connect

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #49 from pan3793/spark-connect-rpc.

316d46f [Cheng Pan] fi
914e1ba [Cheng Pan] Add kyuubi-relocated-spark-connect-rpc

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
  • Loading branch information
pan3793 committed Aug 19, 2024
1 parent c850eee commit c58c858
Show file tree
Hide file tree
Showing 12 changed files with 3,928 additions and 0 deletions.
110 changes: 110 additions & 0 deletions kyuubi-relocated-spark-connect-rpc/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-parent</artifactId>
<version>0.5.0-SNAPSHOT</version>
</parent>

<artifactId>kyuubi-relocated-spark-connect-rpc</artifactId>
<description>Relocated Spark Connect RPC classes used by Kyuubi internally.</description>

<properties>
<grpc.version>1.65.1</grpc.version>
<protobuf.version>3.25.1</protobuf.version>
</properties>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-bom</artifactId>
<version>${grpc.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-bom</artifactId>
<version>${protobuf.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>

<build>
<plugins>
<!-- Add protobuf-maven-plugin and provide ScalaPB as a code generation plugin -->
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.6.1</version>
<configuration>
<protocArtifact>com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}</pluginArtifact>
<protoSourceRoot>src/main/protobuf</protoSourceRoot>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
<goal>test-compile</goal>
</goals>
<configuration>
<pluginParameter>@generated=omit</pluginParameter>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<!-- Used to resolve variables in the 'version' tag -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
proto files are copied from
https://github.com/apache/spark/tree/5b2d2149b615acdd8730547a1f24c2b637222545/sql/connect/common/src/main/protobuf

and with one additional change in each proto file
```patch
- option java_package = "org.apache.spark.connect.proto"
+ option java_package = "org.apache.kyuubi.shaded.spark.connect.proto"
```
Loading

0 comments on commit c58c858

Please sign in to comment.