Skip to content

Commit 6f69f1b

Browse files
committed
review comments
1 parent 0bdaf42 commit 6f69f1b

File tree

3 files changed

+18
-9
lines changed

3 files changed

+18
-9
lines changed

standalone-metastore/packaging/src/docker/README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,10 +133,16 @@ change the `POSTGRES_LOCAL_PATH` to the path of the downloaded jar.
133133

134134
- Metastore with S3 support
135135

136-
Download aws-java-sdk-bundle-xxx.jar and place it under the jars directory:
136+
Download AWS SDK bundle and place it under the jars directory:
137+
138+
Note: Hadoop 3.4.1 requires both AWS SDK V1 and V2 for its S3A connector:
137139
wget https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.12.770/aws-java-sdk-bundle-1.12.770.jar -P jars/
140+
wget https://repo1.maven.org/maven2/software/amazon/awssdk/bundle/2.26.19/bundle-2.26.19.jar -P jars/
138141

139-
Add the `fs.s3a.access.key` and `fs.s3a.secret.key` properties in `metastore-site.xml` under the conf directory.
142+
Then add the following configs to the `metastore-site.xml` under the conf directory:
143+
- fs.s3a.endpoint
144+
- fs.s3a.access.key
145+
- fs.s3a.secret.key
140146

141147
Then,
142148
```shell

standalone-metastore/packaging/src/docker/conf/metastore-site.xml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,10 @@
3434
</property>
3535

3636
<!--
37+
<property>
38+
<name>fs.s3a.endpoint</name>
39+
<value>s3.us-west-2.amazonaws.com</value>
40+
</property>
3741
<property>
3842
<name>fs.s3a.access.key</name>
3943
<value>YOUR_AWS_ACCESS_KEY</value>

standalone-metastore/packaging/src/docker/entrypoint.sh

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -25,16 +25,15 @@ set -x
2525
STAGING_DIR="/tmp/ext-jars"
2626

2727
TOOLS_LIB="${HADOOP_HOME}/share/hadoop/tools/lib"
28-
COMMON_LIB="${HADOOP_HOME}/share/hadoop/common/lib"
28+
HIVE_LIB="${HIVE_HOME}/lib/"
2929

3030
# Checks if /tmp/ext-jars is mounted (via Docker volume).
3131
if [ -d "$STAGING_DIR" ]; then
32-
# Check for aws-java-sdk-bundle (Wildcard handles versions)
33-
if ls "$STAGING_DIR"/aws-java-sdk-bundle-*.jar 1> /dev/null 2>&1; then
34-
echo "--> Installing AWS SDK Bundle..."
35-
cp "$STAGING_DIR"/aws-java-sdk-bundle-*.jar "$COMMON_LIB/"
36-
echo "--> activating hadoop-aws from tools..."
37-
cp "$TOOLS_LIB"/hadoop-aws-*.jar "$COMMON_LIB/"
32+
if ls "$STAGING_DIR"/*.jar 1> /dev/null 2>&1; then
33+
echo "--> Copying custom jars from volume to Hive..."
34+
cp -vf "$STAGING_DIR"/*.jar "$HIVE_LIB/"
35+
else
36+
echo "--> Volume mounted at $STAGING_DIR, but no jars found."
3837
fi
3938
fi
4039

0 commit comments

Comments
 (0)