PIP-209: Removed C++/Python clients from main repo #17881

merlimat · 2022-09-29T01:37:26Z

Motivation

Removing C++ and Python client code from main repo

Documentation

doc-required
(Your PR needs to update docs and you will update later)
doc-not-needed
(Please explain why)
doc
(Your PR contains doc changes)
doc-complete
(Docs have been already added)

Matching PR in forked repository

PR in forked repository: merlimat#1

tisonkun · 2022-09-29T01:45:24Z

Do we have the corresponding new repo now?

Found:

Although I'd say the python client build system is currently tightly coupled with cpp library :)

merlimat · 2022-09-29T01:55:52Z

Yes, I just created the 2 repos with the current state of these libraries. The build system and CI of these repos will require some adjustments, though it's important the 2 changes are synchronized.

tisonkun

Generally looks good. One comment inline.

Also, comment for a cross-reference to the master issue #17724.

tisonkun · 2022-09-29T01:56:44Z

docker/pulsar/pom.xml

        <docker.buildArg.UBUNTU_MIRROR>${env.UBUNTU_MIRROR}</docker.buildArg.UBUNTU_MIRROR>
      </properties>
      <build>
        <plugins>
-          <!-- build Python client, copy the wheel file and then build docker image -->


tisonkun · 2022-09-29T01:59:02Z

pulsar-functions/instance/pom.xml

-                <mkdir dir="${basedir}/target/python-instance/pulsar"/>
-                <copydir src="${basedir}/../../pulsar-client-cpp/python/pulsar"
-                         dest="${basedir}/target/python-instance/pulsar"/>


Doesn't this removal cause file not found?

No, this is just to avoid copying the Pulsar python lib files directly in the Pulsar functions directory, which was not needed anyway.

tisonkun · 2022-09-29T02:00:33Z

src/stage-release.sh

-mkdir $DEST_PATH/RPMS
-cp -r $PULSAR_PATH/pulsar-client-cpp/pkg/rpm/RPMS/x86_64/* $DEST_PATH/RPMS
-
-mkdir $DEST_PATH/DEB
-cp -r $PULSAR_PATH/pulsar-client-cpp/pkg/deb/BUILD/DEB/* $DEST_PATH/DEB


Perhaps file a subtask under #17724 about correspondingly update release guide.

Yes, we'll remove the C++ release step from the main Pulsar release guide.

momo-jun · 2022-10-21T04:13:40Z

I changed the doc label from doc-required to doc-complete since the related doc PR has been merged. Feel free to correct me if I misunderstand it.

* PIP-209: Removed C++/Python clients from main repo * Removed python directory from Docekrfile * Fixed python client version argument scoping * Fixed handling of pulsar.functions.serde (cherry picked from commit f3c547b) Signed-off-by: Zixuan Liu <nodeces@gmail.com>

* [Dockerfile] Enable retries for apt-get when building Pulsar docker image (apache#14513) - also reduce default timeout to 30 seconds - prevents issues where apt repository doesn't respond (cherry picked from commit d3f6fe3) * PIP-155: Removed Python 2 support (apache#15376) * Remove Pulsar Client Build for Python 2.7 * Remove outdated homebrew files (source of truth is upstream homebrew) * Remove Python 2.7 build references; print error in some cases * Update python client tests to run with python client for python 3.5m * PIP-155: Removed Python 2 support * Fixed invocation in pulsar-build image * Fixed clang-format-10 indent differences * Fixed script invocation with wrong python * We don't need to rebuild the manylinux image each time * Fixed image name * Reverted back to use newer protobuf * Fixed image name * Fixed missing python3 in centos:7 image * Use python3 for gtest-parallel * Show bash commands in docker-tests.sh * Fixed gh action issue with git directory permissions * Fixed python to 3 * Fixed custom_logger_test.py * Fixed path in run_python_instance_tests.sh * Function runtime should use python3 * Fixed function runtime test python expectation * Fixed presto worker launcher * Fixed notes on how to format C++ code Co-authored-by: Michael Marshall <mmarshall@apache.org> (cherry picked from commit 2b2e0c5) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [improve][docker] Switch to Temurin JDK (apache#17129) Signed-off-by: Zixuan Liu <nodeces@gmail.com> (cherry picked from commit 4378856) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [refactor][ci] Build the docker image with docker-maven-plugin (apache#17148) (cherry picked from commit a68b58d) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [feat][build] Support ARM64-based docker images (apache#17733) (cherry picked from commit 9a2aeb2) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * PIP-209: Removed C++/Python clients from main repo (apache#17881) * PIP-209: Removed C++/Python clients from main repo * Removed python directory from Docekrfile * Fixed python client version argument scoping * Fixed handling of pulsar.functions.serde (cherry picked from commit f3c547b) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [improve][build] Avoid building image multiple times (apache#17208) Signed-off-by: Zixuan Liu <nodeces@gmail.com> (cherry picked from commit 79a97a9) * [improve] Allow to build and push multi-arch Docker images (apache#19432) Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com> Co-authored-by: Yong Zhang <zhangyong1025.zy@gmail.com> Co-authored-by: Zixuan Liu <nodeces@gmail.com> Co-authored-by: tison <wander4096@gmail.com> (cherry picked from commit 4190e40) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Fix publish image script (apache#20305) Signed-off-by: Zixuan Liu <nodeces@gmail.com> (cherry picked from commit 94c7bf3) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Fix the pulsar-all image may use the wrong upstream image (apache#20435) Signed-off-by: Zike Yang <zike@apache.org> Co-authored-by: Lari Hotari <lhotari@apache.org> (cherry picked from commit d7f3558) Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [feat][build] Adapt to Python client to be compatible with ARM arch Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Configure git-commit-id-plugin to skip git describe (apache#20550) (cherry picked from commit 05f7e62) * [improve][misc] Include native epoll library for Netty for arm64 From apache#22319 Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][misc] Rename all shaded Netty native libraries From apache#22415 Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [cleanup][build] Cleanup -Ddocker.nocache=true Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Fix ubuntu mirror Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Fix license Signed-off-by: Zixuan Liu <nodeces@gmail.com> * [fix][build] Downgrade docker-maven to 0.43.3 Signed-off-by: Zixuan Liu <nodeces@gmail.com> --------- Signed-off-by: Zixuan Liu <nodeces@gmail.com> Signed-off-by: Zike Yang <zike@apache.org> Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com> Co-authored-by: Matteo Merli <mmerli@apache.org> Co-authored-by: Michael Marshall <mmarshall@apache.org> Co-authored-by: tison <wander4096@gmail.com> Co-authored-by: Yong Zhang <zhangyong1025.zy@gmail.com> Co-authored-by: Zike Yang <zike@apache.org> Co-authored-by: Lari Hotari <lhotari@apache.org>

merlimat added component/client-c++ labels Sep 29, 2022

merlimat added this to the 2.12.0 milestone Sep 29, 2022

merlimat requested review from RobertIndie and BewareMyPower September 29, 2022 01:37

merlimat self-assigned this Sep 29, 2022

github-actions bot added the doc-required Your PR changes impact docs and you will update later. label Sep 29, 2022

PIP-209: Removed C++/Python clients from main repo

4865e46

merlimat force-pushed the pip-209-remove-c++-client branch from e954852 to 4865e46 Compare September 29, 2022 01:39

tisonkun reviewed Sep 29, 2022

View reviewed changes

merlimat added 3 commits September 28, 2022 21:08

Removed python directory from Docekrfile

8535f97

Fixed python client version argument scoping

3baf5c2

Fixed handling of pulsar.functions.serde

98621a6

merlimat added the ready-to-test label Sep 29, 2022

tisonkun approved these changes Sep 29, 2022

View reviewed changes

merlimat closed this Sep 29, 2022

merlimat reopened this Sep 29, 2022

RobertIndie approved these changes Sep 30, 2022

View reviewed changes

merlimat merged commit f3c547b into apache:master Sep 30, 2022

lhotari mentioned this pull request Sep 30, 2022

[fix][CI] Don't run pulsar-ci-flaky workflow for PRs which contain only document changes #17906

Merged

4 tasks

nicoloboschi mentioned this pull request Oct 4, 2022

[fix][ci] CI: fix required jobs check #17926

Merged

1 task

This was referenced Oct 4, 2022

[fix][ci] succeed only if all needs pass #17923

Closed

[fix][testclient] Fix proxyProtocol parsing when value is empty, fix invalid property name #17930

Merged

momo-jun added doc-complete Your PR changes impact docs and the related docs have been already added. and removed doc-required Your PR changes impact docs and you will update later. labels Oct 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PIP-209: Removed C++/Python clients from main repo #17881

PIP-209: Removed C++/Python clients from main repo #17881

merlimat commented Sep 29, 2022

tisonkun commented Sep 29, 2022 •

edited

Loading

merlimat commented Sep 29, 2022

tisonkun left a comment

tisonkun Sep 29, 2022

tisonkun Sep 29, 2022

merlimat Sep 29, 2022

tisonkun Sep 29, 2022

merlimat Sep 29, 2022

momo-jun commented Oct 21, 2022

PIP-209: Removed C++/Python clients from main repo #17881

PIP-209: Removed C++/Python clients from main repo #17881

Conversation

merlimat commented Sep 29, 2022

Motivation

Documentation

Matching PR in forked repository

tisonkun commented Sep 29, 2022 • edited Loading

merlimat commented Sep 29, 2022

tisonkun left a comment

Choose a reason for hiding this comment

tisonkun Sep 29, 2022

Choose a reason for hiding this comment

tisonkun Sep 29, 2022

Choose a reason for hiding this comment

merlimat Sep 29, 2022

Choose a reason for hiding this comment

tisonkun Sep 29, 2022

Choose a reason for hiding this comment

merlimat Sep 29, 2022

Choose a reason for hiding this comment

momo-jun commented Oct 21, 2022

tisonkun commented Sep 29, 2022 •

edited

Loading