Skip to content

chore(hbase): update images for 25.7.0 #1101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ All notable changes to this project will be documented in this file.
- nifi: Add [nifi-iceberg-bundle](https://github.com/stackabletech/nifi-iceberg-bundle) for NiFi `2.2.0` ([#1060], [#1106]).
- java: Add JDK 24 ([#1097]).
- ci: Add golang image to mirror workflow ([#1103]).
- hbase: add 2.6.2 and upgrade dependencies ([#1101])

### Changed

Expand Down Expand Up @@ -75,6 +76,7 @@ All notable changes to this project will be documented in this file.
- Remove ubi8-rust-builder image ([#1091]).
- opa: Remove `0.67.1` ([#1103]).
- opa: Remove legacy bundle-builder from container build ([#1103]).
- hbase: Remove 2.4.18 ([#1101])

[#1025]: https://github.com/stackabletech/docker-images/pull/1025
[#1027]: https://github.com/stackabletech/docker-images/pull/1027
Expand All @@ -100,6 +102,7 @@ All notable changes to this project will be documented in this file.
[#1093]: https://github.com/stackabletech/docker-images/pull/1093
[#1097]: https://github.com/stackabletech/docker-images/pull/1097
[#1098]: https://github.com/stackabletech/docker-images/pull/1098
[#1101]: https://github.com/stackabletech/docker-images/pull/1101
[#1102]: https://github.com/stackabletech/docker-images/pull/1102
[#1103]: https://github.com/stackabletech/docker-images/pull/1103
[#1106]: https://github.com/stackabletech/docker-images/pull/1106
Expand Down
24 changes: 6 additions & 18 deletions hbase/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,10 @@ FROM stackable/image/hadoop AS hadoop-builder
FROM stackable/image/java-devel AS hbase-builder

ARG PRODUCT
ARG HBASE_THIRDPARTY
ARG HBASE_OPERATOR_TOOLS
ARG ASYNC_PROFILER
ARG PHOENIX
ARG HBASE_PROFILE
ARG JMX_EXPORTER
ARG HADOOP
ARG TARGETARCH
ARG TARGETOS
Expand All @@ -28,7 +26,6 @@ WORKDIR /stackable

COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/patches/patchable.toml /stackable/src/hbase/stackable/patches/patchable.toml
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/patches/${PRODUCT} /stackable/src/hbase/stackable/patches/${PRODUCT}
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/jmx/config${JMX_EXPORTER} /stackable/jmx

# Cache mounts are owned by root by default
# We need to explicitly give the uid to use
Expand Down Expand Up @@ -59,15 +56,6 @@ mv hbase-assembly/target/bom.json /stackable/hbase-${PRODUCT}/hbase-${PRODUCT}.c

ln -s "/stackable/hbase-${PRODUCT}" /stackable/hbase

###
### JMX Prometheus Exporter/Agent
###
if [[ -n "${JMX_EXPORTER}" ]] ; then
curl --fail "https://repo.stackable.tech/repository/packages/jmx-exporter/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar" -o "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar"
chmod +x "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar"
ln -s "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar" /stackable/jmx/jmx_prometheus_javaagent.jar
fi

###
### Async Profiler
###
Expand Down Expand Up @@ -155,16 +143,13 @@ WORKDIR /stackable
RUN --mount=type=cache,id=maven-hbase-operator-tools-${HBASE_OPERATOR_TOOLS},uid=${STACKABLE_USER_UID},target=/stackable/.m2/repository <<EOF
cd "$(/stackable/patchable --images-repo-root=src checkout hbase-operator-tools ${HBASE_OPERATOR_TOOLS})"

# Remove the Git repo because `git-commit-id-maven-plugin` tries to get the latest commit if a Git repo is present,
# which fails due to a problem with worktrees, see https://github.com/git-commit-id/git-commit-id-maven-plugin/issues/215
rm .git

mvn \
--batch-mode \
--no-transfer-progress \
-Dhbase.version=${PRODUCT} \
-Dhbase-thirdparty.version=${HBASE_THIRDPARTY} \
-DskipTests \
-Dmaven.test.skip=true \
package assembly:single

# We need the "*" here as the directory won't be the same as the final tar file for SNAPSHOTs which we currently have to use for 2.6
Expand Down Expand Up @@ -208,6 +193,11 @@ USER ${STACKABLE_USER_UID}
WORKDIR /stackable

COPY --from=hadoop-builder --chown=${STACKABLE_USER_UID}:0 \
# The artifact name of the AWS bundle has changed between Haddop 3.3.6 and 3.4.1
# from aws-java-sdk-bundle-*.jar to bundle-*.jar.
# See: https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/aws_sdk_upgrade.html
# So we try to copy both and if one of them doesn't exist buildx will just ignore it :)
/stackable/hadoop/share/hadoop/tools/lib/bundle-*.jar \
/stackable/hadoop/share/hadoop/tools/lib/aws-java-sdk-bundle-*.jar \
/stackable/hadoop/share/hadoop/tools/lib/hadoop-aws-${HADOOP}.jar \
/stackable/hadoop/share/hadoop/tools/lib/
Expand Down Expand Up @@ -324,7 +314,6 @@ LABEL io.k8s.display-name="${NAME}"

COPY --chown=${STACKABLE_USER_UID}:0 --from=hbase-builder /stackable/hbase-${PRODUCT} /stackable/hbase-${PRODUCT}/
COPY --chown=${STACKABLE_USER_UID}:0 --from=hbase-builder /stackable/async-profiler /stackable/async-profiler/
COPY --chown=${STACKABLE_USER_UID}:0 --from=hbase-builder /stackable/jmx /stackable/jmx/

COPY --chown=${STACKABLE_USER_UID}:0 --from=hbase-operator-tools-builder /stackable/hbase-operator-tools-${HBASE_OPERATOR_TOOLS} /stackable/hbase-operator-tools-${HBASE_OPERATOR_TOOLS}/
COPY --chown=${STACKABLE_USER_UID}:0 --from=hbase-operator-tools-builder /stackable/bin/hbck2 /stackable/bin/hbck2
Expand Down Expand Up @@ -378,7 +367,6 @@ chmod g=u "/stackable/hbase/lib/phoenix-server-hbase-${HBASE_PROFILE}.jar"
# fix missing permissions
chmod g=u /stackable/async-profiler
chmod g=u /stackable/bin
chmod g=u /stackable/jmx
chmod g=u /stackable/phoenix
# the whole directory tree /stackable/hadoop/share/hadoop/tools/lib/ must be adapted
find /stackable/hadoop -type d -exec chmod g=u {} +
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
From f9e952767faf3032c0d4e729e51b5901fcf1a0e1 Mon Sep 17 00:00:00 2001
From: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
Date: Tue, 6 May 2025 17:58:34 +0200
Subject: Exclude hbase-testing-utils dependency from the build.

---
hbase-hbck2/pom.xml | 11 +++++++++--
hbase-tools/pom.xml | 5 +++--
2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/hbase-hbck2/pom.xml b/hbase-hbck2/pom.xml
index 1d57225..fd0018c 100644
--- a/hbase-hbck2/pom.xml
+++ b/hbase-hbck2/pom.xml
@@ -114,12 +114,19 @@
</exclusion>
</exclusions>
</dependency>
- <dependency>
+ <!-- Starting with Hbase 2.6.2, this brings in a dependency that is not publicly available:
+ org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2
+
+ Since we do not run the tests, we exclude this dependency.
+
+ To build successfully we also need to use -Dmaven.test.skip=true to skip building the tests.
+ -->
+ <!-- dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-testing-util</artifactId>
<version>${hbase.version}</version>
<scope>test</scope>
- </dependency>
+ </dependency -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
diff --git a/hbase-tools/pom.xml b/hbase-tools/pom.xml
index 55ef075..eeebe14 100644
--- a/hbase-tools/pom.xml
+++ b/hbase-tools/pom.xml
@@ -103,12 +103,13 @@
</exclusion>
</exclusions>
</dependency>
- <dependency>
+ <!-- See the hbase-hbck2/pom.xml for an explanation of why this is commented out -->
+ <!-- dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-testing-util</artifactId>
<version>${hbase.version}</version>
<scope>test</scope>
- </dependency>
+ </dependency -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
From 187db522edc569def2aea3f9453ecb1a674592fe Mon Sep 17 00:00:00 2001
From: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
Date: Wed, 7 May 2025 10:26:37 +0200
Subject: Configure git-commit-id-plugin to use native git

This fixes the problem that jgit cannot handle worktrees and the build
fails.

See
https://github.com/git-commit-id/git-commit-id-maven-plugin/issues/215
---
hbase-hbck2/pom.xml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hbase-hbck2/pom.xml b/hbase-hbck2/pom.xml
index fd0018c..f00bd18 100644
--- a/hbase-hbck2/pom.xml
+++ b/hbase-hbck2/pom.xml
@@ -215,8 +215,9 @@
<plugin>
<groupId>pl.project13.maven</groupId>
<artifactId>git-commit-id-plugin</artifactId>
- <version>2.2.5</version>
+ <version>4.9.10</version>
<configuration>
+ <useNativeGit>true</useNativeGit>
<dotGitDirectory>${project.basedir}/.git</dotGitDirectory>
<commitIdGenerationMode>flat</commitIdGenerationMode>
<failOnNoGitDirectory>false</failOnNoGitDirectory>
5 changes: 0 additions & 5 deletions hbase/stackable/jmx/config/README.md

This file was deleted.

3 changes: 0 additions & 3 deletions hbase/stackable/jmx/config1.2.0/master.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions hbase/stackable/jmx/config1.2.0/regionserver.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions hbase/stackable/jmx/config1.2.0/restserver.yaml

This file was deleted.

This file was deleted.

Loading