Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-16389: Bump Apache Avro to 1.9.0 #1007

Open
wants to merge 1 commit into
base: branch-3
Choose a base branch
from
Open

HADOOP-16389: Bump Apache Avro to 1.9.0 #1007

wants to merge 1 commit into from
+2 −2

Conversation

Fokko
Copy link

@Fokko Fokko commented Jun 22, 2019

Apache Avro 1.9.0 brings a lot of new features:

  • Deprecate Joda-Time in favor of Java8 JSR310 and setting it as default
  • Remove support for Hadoop 1.x
  • Move from Jackson 1.x to 2.9
  • Add ZStandard Codec
  • Lots of updates on the dependencies to fix CVE's
  • Remove Jackson classes from public API
  • Apache Avro is built by default with Java 8
  • Apache Avro is compiled and tested with Java 11 to guarantee compatibility
  • Apache Avro MapReduce is compiled and tested with Hadoop 3
  • Apache Avro is now leaner, multiple dependencies were removed: guava, paranamer, commons-codec, and commons-logging

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 30 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1034 trunk passed
+1 compile 18 trunk passed
+1 mvnsite 21 trunk passed
+1 shadedclient 1736 branch has no errors when building and testing our client artifacts.
+1 javadoc 15 trunk passed
_ Patch Compile Tests _
+1 mvninstall 12 the patch passed
+1 compile 10 the patch passed
+1 javac 10 the patch passed
+1 mvnsite 13 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 1 The patch has no ill-formed XML file.
+1 shadedclient 711 patch has no errors when building and testing our client artifacts.
+1 javadoc 17 the patch passed
_ Other Tests _
+1 unit 16 hadoop-project in the patch passed.
+1 asflicense 31 The patch does not generate ASF License warnings.
2723
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/1/artifact/out/Dockerfile
GITHUB PR #1007
JIRA Issue HADOOP-16389
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname Linux 24aeb4387d31 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 645d67b
Default Java 1.8.0_212
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/1/testReport/
Max. process+thread count 447 (vs. ulimit of 5500)
modules C: hadoop-project U: hadoop-project
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/1/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

jiwq
jiwq approved these changes Jun 23, 2019
View reviewed changes
Copy link
Member

@jiwq jiwq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@jojochuang
Copy link
Contributor

Probably make sense for Hadoop 3. Not so sure about Hadoop 2.x

@steveloughran
Copy link
Contributor

this is potentially really traumatic, which is why we've put it off in the past (HADOOP-13386) until satisfied.

-1 to branch-2 right now, branch-3 trunk would be the place to start. Please join the HADOOP-13386 discussion and lets be happy there before any merge.

@Fokko Fokko mentioned this pull request Jul 14, 2019
Open
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 514 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
0 mvndep 71 Maven dependency ordering for branch
+1 mvninstall 1089 trunk passed
+1 compile 1041 trunk passed
+1 mvnsite 871 trunk passed
+1 shadedclient 3751 branch has no errors when building and testing our client artifacts.
+1 javadoc 348 trunk passed
_ Patch Compile Tests _
0 mvndep 25 Maven dependency ordering for patch
+1 mvninstall 1084 the patch passed
+1 compile 984 the patch passed
+1 javac 984 the patch passed
+1 mvnsite 850 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 xml 3 The patch has no ill-formed XML file.
+1 shadedclient 677 patch has no errors when building and testing our client artifacts.
+1 javadoc 338 the patch passed
_ Other Tests _
-1 unit 940 root in the patch failed.
+1 asflicense 48 The patch does not generate ASF License warnings.
9723
Reason Tests
Failed junit tests hadoop.io.TestEnumSetWritable
Subsystem Report/Notes
Docker Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/2/artifact/out/Dockerfile
GITHUB PR #1007
JIRA Issue HADOOP-16389
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname Linux 67df33a58511 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 0976f6f
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/2/artifact/out/patch-unit-root.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/2/testReport/
Max. process+thread count 1449 (vs. ulimit of 5500)
modules C: hadoop-project . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/2/console
versions git=2.7.4 maven=3.3.9
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Fokko
Copy link
Author

Fokko commented Jul 17, 2019

It now compiles on my machine. I have some failing tests due to localization, but apart from that it looks good so far.

@Fokko Fokko closed this Jul 17, 2019
@Fokko Fokko reopened this Jul 17, 2019
@Fokko
Copy link
Author

Fokko commented Jul 17, 2019

My mistake, rebased onto master instead of trunk

@Fokko Fokko changed the base branch from trunk to branch-3 July 17, 2019 14:04
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 0 Docker mode activated.
-1 patch 156 #1007 does not apply to branch-3. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
Subsystem Report/Notes
JIRA Issue HADOOP-16389
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1007/4/console
versions git=2.7.4
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@aajisaka aajisaka changed the base branch from branch-3 to trunk July 26, 2019 06:08
@aajisaka aajisaka changed the base branch from trunk to branch-3 July 26, 2019 06:09
@aajisaka
Copy link
Member

Hi @Fokko, would you rebase for trunk instead of branch-3? In Apache Hadoop, trunk is regarded as master and branch-3 is unused.

shanthoosh added a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
@shanthoosh
SAMZA-2173: Fix NullPointerException in SetConfig MessageFormat when …
130ece5 
…configurations are deleted. (apache#1007)

* Fix NullPointerException in SetConfig message when configurations are deleted by encapsulating the isDelete check in the CoordinatorStreamMessage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiwq jiwq jiwq approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Fokko @hadoop-yetus @jojochuang @steveloughran @aajisaka @jiwq