HADOOP-16748. Migrate to Python 3 and upgrade Yetus to 0.13.0 in branch-2.10

[aajisaka]

• Upgrade Yetus to 0.13.0 to support Python 3 for the release scripts.
• Removed determine-flaky-tests-hadoop.py.
• Temporarily disabled shelldocs check due to YETUS-1099.

Reviewed-by: Inigo Goiri inigoiri@apache.org
Reviewed-by: Mingliang Liu liuml07@apache.org
(cherry picked from commit b9b49ed)

Conflicts:
dev-support/Jenkinsfile
dev-support/bin/yetus-wrapper
dev-support/docker/Dockerfile
dev-support/docker/Dockerfile_aarch64

Description of PR

Backport HADOOP-16748 to branch-2.10.

How was this patch tested?

Not tested. The jenkins job will do.

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• n/a Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• n/a If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• n/a If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[aajisaka]

After the check passed, I'll merge this PR and then backport HADOOP-17570.

[asf-cloudbees-jenkins-ci-hadoop]

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3832/1/console in case of problems.

[aajisaka]

pylint fails because it is not compatible with Python 3.5.x, which is provided by Ubuntu Xenial. I'll backport HADOOP-16054 (Upgrade to Bionic) first.

[asf-cloudbees-jenkins-ci-hadoop]

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3832/2/console in case of problems.

[aajisaka]

Backported HADOOP-16054 and rebased.

[aajisaka]

OK: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3832/2/artifact/out/console.txt

+1 overall

| Vote |      Subsystem |  Runtime   | Comment
============================================================================
|   0  |        reexec  |   8m 22s   | Docker mode activated. 
+---------------------------------------------------------------------------
|      |                |            | Prechecks 
+---------------------------------------------------------------------------
|  +1  |       dupname  |   0m  0s   | No case conflicting files found. 
|   0  |     codespell  |   0m  0s   | codespell was not available. 
|   0  |     shelldocs  |   0m  0s   | Shelldocs was not available. 
|  +1  |       @author  |   0m  0s   | The patch does not contain any @author 
|      |                |            | tags.
+---------------------------------------------------------------------------
|      |                |            | branch-2.10 Compile Tests 
+---------------------------------------------------------------------------
|   0  |        mvndep  |   2m 22s   | Maven dependency ordering for branch 
+---------------------------------------------------------------------------
|      |                |            | Patch Compile Tests 
+---------------------------------------------------------------------------
|   0  |        mvndep  |   0m 15s   | Maven dependency ordering for patch 
|  +1  |        blanks  |   0m  0s   | The patch has no blanks issues. 
|  +1  |      hadolint  |   0m  3s   | No new issues. 
|  +1  |        pylint  |   0m 12s   | The patch generated 0 new + 175 
|      |                |            | unchanged - 196 fixed = 175 total (was
|      |                |            | 371)
|  +1  |    shellcheck  |   0m  0s   | No new issues. 
+---------------------------------------------------------------------------
|      |                |            | Other Tests 
+---------------------------------------------------------------------------
|  +1  |    asflicense  |   0m 27s   | The patch does not generate ASF License 
|      |                |            | warnings.
|      |                |  13m 57s   |