Skip to content

[SPARK-14867][BUILD] Remove --force option in build/mvn #12631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dongjoon-hyun wants to merge 8 commits into from
Closed

[SPARK-14867][BUILD] Remove --force option in build/mvn #12631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
cb137ad
[SPARK-14867][BUILD] Make `build/mvn` to use the downloaded maven if …
dongjoon-hyun Apr 14, 2016
27605ba
Choose Maven based on versions.
dongjoon-hyun Apr 23, 2016
b1cfc12
Change to local variables.
dongjoon-hyun Apr 23, 2016
6fdf92d
Add a code for backward compatability.
dongjoon-hyun Apr 23, 2016
1f4b786
Determine the maven version from `pom.xml`.
dongjoon-hyun Apr 24, 2016
d1356b8
Add a deprecation message for option --force
dongjoon-hyun Apr 24, 2016
9c4add2
Update test-dependecies.sh.
dongjoon-hyun Apr 24, 2016
20a520a
Use awk command according to the comments.
dongjoon-hyun Apr 25, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 22 additions & 21 deletions build/mvn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,17 +67,26 @@ install_app() {
fi
}

# Install maven under the build/ folder
# Determine the Maven version from the root pom.xml file and
# install maven under the build/ folder if needed.
install_mvn() {
local MVN_VERSION="3.3.9"
local APACHE_MIRROR=${APACHE_MIRROR:-'https://www.apache.org/dyn/closer.lua?action=download&filename='}
local MVN_VERSION=`grep "<maven.version>" "${_DIR}/../pom.xml" | head -n1 | awk -F '[<>]' '{print $3}'`
MVN_BIN="$(command -v mvn)"
if [ "$MVN_BIN" ]; then
local MVN_DETECTED_VERSION="$(mvn --version | head -n1 | awk '{print $3}')"
fi
# See simple version normalization: http://stackoverflow.com/questions/16989598/bash-comparing-version-numbers
function version { echo "$@" | awk -F. '{ printf("%03d%03d%03d\n", $1,$2,$3); }'; }
if [ $(version $MVN_DETECTED_VERSION) -lt $(version $MVN_VERSION) ]; then
Copy link
Member

@srowen srowen Apr 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's no local version, then this always returns "false" as if the detected version is 0?

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Apr 24, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When there is no local version, it becomes '000000000' and the condition returns true.

local APACHE_MIRROR=${APACHE_MIRROR:-'https://www.apache.org/dyn/closer.lua?action=download&filename='}

install_app \
"${APACHE_MIRROR}/maven/maven-3/${MVN_VERSION}/binaries" \
"apache-maven-${MVN_VERSION}-bin.tar.gz" \
"apache-maven-${MVN_VERSION}/bin/mvn"
install_app \
"${APACHE_MIRROR}/maven/maven-3/${MVN_VERSION}/binaries" \
"apache-maven-${MVN_VERSION}-bin.tar.gz" \
"apache-maven-${MVN_VERSION}/bin/mvn"

MVN_BIN="${_DIR}/apache-maven-${MVN_VERSION}/bin/mvn"
MVN_BIN="${_DIR}/apache-maven-${MVN_VERSION}/bin/mvn"
fi
}

# Install zinc under the build/ folder
Expand All @@ -98,8 +107,7 @@ install_zinc() {
# the build/ folder
install_scala() {
# determine the Scala version used in Spark
local scala_version=`grep "scala.version" "${_DIR}/../pom.xml" | \
head -1 | cut -f2 -d'>' | cut -f1 -d'<'`
local scala_version=`grep "scala.version" "${_DIR}/../pom.xml" | head -n1 | awk -F '[<>]' '{print $3}'`
local scala_bin="${_DIR}/scala-${scala_version}/bin/scala"
local TYPESAFE_MIRROR=${TYPESAFE_MIRROR:-https://downloads.typesafe.com}

Expand All @@ -116,23 +124,16 @@ install_scala() {
# the environment
ZINC_PORT=${ZINC_PORT:-"3030"}

# Check for the `--force` flag dictating that `mvn` should be downloaded
# regardless of whether the system already has a `mvn` install
# Remove `--force` for backward compatibility.
if [ "$1" == "--force" ]; then
FORCE_MVN=1
echo "WARNING: '--force' is deprecated and ignored."
shift
fi

# Install Maven if necessary
MVN_BIN="$(command -v mvn)"

if [ ! "$MVN_BIN" -o -n "$FORCE_MVN" ]; then
install_mvn
fi

# Install the proper version of Scala and Zinc for the build
# Install the proper version of Scala, Zinc and Maven for the build
install_zinc
install_scala
install_mvn

# Reset the current working directory
cd "${_CALLING_DIR}"
Expand Down
2 changes: 1 addition & 1 deletion dev/test-dependencies.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ export LC_ALL=C

# NOTE: These should match those in the release publishing script
HADOOP2_MODULE_PROFILES="-Phive-thriftserver -Pyarn -Phive"
MVN="build/mvn --force"
MVN="build/mvn"
HADOOP_PROFILES=(
hadoop-2.2
hadoop-2.3
Expand Down