Skip to content

Comments

Add changes for CI builds and publishing artifacts to bintray.#2

Merged
jonlee2 merged 2 commits intolinkedin:2.0-lifrom
jonlee2:bintray-publish-ci
Mar 8, 2019
Merged

Add changes for CI builds and publishing artifacts to bintray.#2
jonlee2 merged 2 commits intolinkedin:2.0-lifrom
jonlee2:bintray-publish-ci

Conversation

@jonlee2
Copy link
Collaborator

@jonlee2 jonlee2 commented Mar 8, 2019

Travis will kick off a build and publish artifacts to bintray upon creating a tag in the "x.y.z.w" format.

Removed system test related settings.

@jonlee2 jonlee2 requested a review from radai-rosenblatt March 8, 2019 02:11
@jonlee2 jonlee2 force-pushed the bintray-publish-ci branch from c26bf78 to 87005c6 Compare March 8, 2019 02:28
Travis will kick off a build and publish artifacts to bintray upon creating a tag in the "x.y.z.w" format.
@jonlee2 jonlee2 force-pushed the bintray-publish-ci branch from 87005c6 to 8ded9d5 Compare March 8, 2019 02:30
# Excluded integration tests for now because Travis will hit build timeout (50 minutes) when including them.
# Also excluded streams unitTest because they often fail with "pure virtual method called" error (KAFKA-3502).
script:
- ./gradlew clean compileJava compileScala compileTestJava compileTestScala checkstyleMain checkstyleTest findbugsMain unitTest -x :streams:unitTest rat --no-daemon -PxmlFindBugsReport=true -PtestLoggingEvents=started,passed,skipped,failed

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least add a todo to re-enable streams unit tests

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jonlee2 jonlee2 merged commit fe46e4d into linkedin:2.0-li Mar 8, 2019
@jonlee2 jonlee2 deleted the bintray-publish-ci branch March 8, 2019 18:44
jonlee2 pushed a commit that referenced this pull request Mar 8, 2019
…he#6115)

This fix is aiming for #2 issue pointed out within https://issues.apache.org/jira/browse/KAFKA-7672
In the current setup, we do offset checkpoint file write when EOS is turned on during #suspend, which introduces the potential race condition during StateManager #closeSuspend call. To mitigate the problem, we attempt to always write checkpoint file in #suspend call.

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Matthias J. Sax <mjsax@apache.org>,  John Roesler <john@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
xiowu0 pushed a commit that referenced this pull request Jun 11, 2019
…he#6115)

This fix is aiming for #2 issue pointed out within https://issues.apache.org/jira/browse/KAFKA-7672
In the current setup, we do offset checkpoint file write when EOS is turned on during #suspend, which introduces the potential race condition during StateManager #closeSuspend call. To mitigate the problem, we attempt to always write checkpoint file in #suspend call.

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Matthias J. Sax <mjsax@apache.org>,  John Roesler <john@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
xiowu0 pushed a commit that referenced this pull request Jun 11, 2019
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["describe exit criteria"]
xiowu0 pushed a commit that referenced this pull request Jun 11, 2019
…tray. (#2)

TICKET =
LI_DESCRIPTION =

Travis will kick off a build and publish artifacts to bintray upon creating a tag in the "x.y.z.w" format.

EXIT_CRITERIA = MANUAL ["describe exit criteria"]
xiowu0 pushed a commit that referenced this pull request Jul 16, 2019
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
xiowu0 pushed a commit that referenced this pull request Jul 17, 2019
TICKET =
LI_DESCRIPTION =

Add build changes to publish github LinkedIn Kafka artifacts to bintray (#1)
[NOTE] This is a temporary measure to publish artifacts until CI is properly set up to do the job automatically. Users are not expected to run this themselves.

Add changes for CI builds and publishing artifacts to bintray. (#2)
Travis will kick off a build and publish artifacts to bintray upon creating a tag in the "x.y.z.w" format.

Try different encrypted bintray-related env variables for Travis (#4)
Travis couldn't access one of the initially encrypted variables for some reason.

Set Bintray-related env variables via repository setting instead of in .travis.yml (#5)

Use the maven repo under the LinkedIn Bintray account to publish artifacts (#25)

EXIT_CRITERIA = MANUAL [""]
xiowu0 pushed a commit that referenced this pull request Aug 22, 2019
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
gitlw pushed a commit to gitlw/kafka that referenced this pull request May 5, 2020
…fter receiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
gitlw pushed a commit that referenced this pull request Jun 13, 2020
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
gitlw pushed a commit to gitlw/kafka that referenced this pull request Oct 5, 2021
…fter receiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 referenced this pull request in lmr3796/kafka Oct 22, 2021
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Oct 22, 2021
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Oct 27, 2021
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Oct 28, 2021
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Nov 19, 2021
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Nov 19, 2021
TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- ~~Avoid appending to the time index during shutdown if the time index has not yet be initialized~~
  This is covered in apache#8346 and apache#10960

EXIT_CRITERIA = TICKET [KAFKA-8667, KAFKA-8668]

The patch is a sqaush of the 2 commits:

== This is the 1st commit [155b4f8] ==

[LI-HOTFIX] Reduce lock retention and improve broker shutdown time:

TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- Avoid appending to the time index during shutdown if the time index has not yet be initialized

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee

== This is the commit #2 [c53fffd] ==

[LI-HOTFIX] Update fetcher thread idle flag in addPartitions

TICKET = KAFKA-8667
LI_DESCRIPTION =

This patch fixes in bug introduced by “[LI-HOTFIX] Reduce lock retention and improve broker shutdown time” HOTFIX where the fetcher thread idle flag is not set in addPartitions, which can cause idle fetcher thread not shutdown in time.

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee
lmr3796 pushed a commit that referenced this pull request Jan 1, 2022
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Jan 4, 2022
TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- ~~Avoid appending to the time index during shutdown if the time index has not yet be initialized~~
  This is covered in apache#8346 and apache#10960

EXIT_CRITERIA = TICKET [KAFKA-8667, KAFKA-8668]

The patch is a sqaush of the 2 commits:

== This is the 1st commit [155b4f8] ==

[LI-HOTFIX] Reduce lock retention and improve broker shutdown time:

TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- Avoid appending to the time index during shutdown if the time index has not yet be initialized

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee

== This is the commit #2 [c53fffd] ==

[LI-HOTFIX] Update fetcher thread idle flag in addPartitions

TICKET = KAFKA-8667
LI_DESCRIPTION =

This patch fixes in bug introduced by “[LI-HOTFIX] Reduce lock retention and improve broker shutdown time” HOTFIX where the fetcher thread idle flag is not set in addPartitions, which can cause idle fetcher thread not shutdown in time.

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee
lmr3796 pushed a commit that referenced this pull request Feb 9, 2022
TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- ~~Avoid appending to the time index during shutdown if the time index has not yet be initialized~~
  This is covered in apache#8346 and apache#10960

EXIT_CRITERIA = TICKET [KAFKA-8667, KAFKA-8668]

The patch is a sqaush of the 2 commits:

== This is the 1st commit [155b4f8] ==

[LI-HOTFIX] Reduce lock retention and improve broker shutdown time:

TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- Avoid appending to the time index during shutdown if the time index has not yet be initialized

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee

== This is the commit #2 [c53fffd] ==

[LI-HOTFIX] Update fetcher thread idle flag in addPartitions

TICKET = KAFKA-8667
LI_DESCRIPTION =

This patch fixes in bug introduced by “[LI-HOTFIX] Reduce lock retention and improve broker shutdown time” HOTFIX where the fetcher thread idle flag is not set in addPartitions, which can cause idle fetcher thread not shutdown in time.

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee
lmr3796 pushed a commit that referenced this pull request Mar 25, 2022
…eiving offset commit exception.

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Mar 25, 2022
TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- ~~Avoid appending to the time index during shutdown if the time index has not yet be initialized~~
  This is covered in apache#8346 and apache#10960

EXIT_CRITERIA = TICKET [KAFKA-8667, KAFKA-8668]

The patch is a sqaush of the 2 commits:

== This is the 1st commit [155b4f8] ==

[LI-HOTFIX] Reduce lock retention and improve broker shutdown time:

TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- Avoid appending to the time index during shutdown if the time index has not yet be initialized

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee

== This is the commit #2 [c53fffd] ==

[LI-HOTFIX] Update fetcher thread idle flag in addPartitions

TICKET = KAFKA-8667
LI_DESCRIPTION =

This patch fixes in bug introduced by “[LI-HOTFIX] Reduce lock retention and improve broker shutdown time” HOTFIX where the fetcher thread idle flag is not set in addPartitions, which can cause idle fetcher thread not shutdown in time.

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee
lmr3796 pushed a commit that referenced this pull request Jun 2, 2022
…eiving offset commit exception

TICKET =
LI_DESCRIPTION =

EXIT_CRITERIA = MANUAL ["needed until we make a long-term fix"]
lmr3796 pushed a commit that referenced this pull request Jun 2, 2022
TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- ~~Avoid appending to the time index during shutdown if the time index has not yet be initialized~~
  This is covered in apache#8346 and apache#10960

EXIT_CRITERIA = TICKET [KAFKA-8667, KAFKA-8668]

The patch is a sqaush of the 2 commits:

== This is the 1st commit [155b4f8] ==

[LI-HOTFIX] Reduce lock retention and improve broker shutdown time:

TICKET = [KAFKA-8667, KAFKA-8668]
LI_DESCRIPTION =
- Avoid acquiring partitionMap lock in shutdownIdleFetcherThread
- Avoid appending to the time index during shutdown if the time index has not yet be initialized

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee

== This is the commit #2 [c53fffd] ==

[LI-HOTFIX] Update fetcher thread idle flag in addPartitions

TICKET = KAFKA-8667
LI_DESCRIPTION =

This patch fixes in bug introduced by “[LI-HOTFIX] Reduce lock retention and improve broker shutdown time” HOTFIX where the fetcher thread idle flag is not set in addPartitions, which can cause idle fetcher thread not shutdown in time.

RB=1431408
BUG=LIKAFKA-19361
G=Kafka-Code-Reviews
R=jkoshy,jonlee
A=jkoshy,jonlee
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants