Skip to content

Commit

Permalink
Add .x resolution and --resolve command
Browse files Browse the repository at this point in the history 
* New persistent file, `~/.gimme/versions/known-versions.txt`, which is
  kept in sorted unique order
* Add `--resolve` for handling version specifiers in a
  normally-lightweight manner, where stdout gets the resolved version.
* Handle `NN.x` (and `NN.MM.x`) as a version specifier, against the
  known versions
* Have the `--known` output be cached to disk; there's a cache bypass
  mechanism (flag), and a cache age control environment variable.
* Rework some cache handling stuff to be in cleaner functional
  abstractions
* Adds a `_version_sort` filter
* Bug-fix for stable age using access-time not mod-time on darwin|*bsd

See the README updates for notes on interaction between `--resolve` and
git tag resolution, and how to avoid that.

Fixes #129
Resolves #110
  • Loading branch information
@philpennock
philpennock committed Feb 17, 2018
1 parent 88af448 commit 54c6bee
Show file tree
Hide file tree
Showing 2 changed files with 180 additions and 28 deletions.
39 changes: 38 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,8 @@ versions and point Gimme at that instead.
Invoke `gimme -k` or `gimme --known` to have Gimme report the versions which
can be installed; invoking `gimme stable` installs the version which the Go
Maintainers have declared to be stable. Both of these involve making
non-cached network requests to retrieve this information.
network requests to retrieve this information, although the `--known` output
is cached. (Use `--force-update` to ignore the cache).

The `stable` request retrieves <https://golang.org/VERSION?m=text> and reports
that.
Expand All @@ -175,3 +176,39 @@ retrieved from, thus it's possible for `known` to know about more or fewer
versions than are actually available. We proceed on the basis that the
documented releases are suitable and undocumented releases no longer are.

This `known` list also includes any versions locally known.

### Asking Gimme what a version is

Gimme now supports the concept of `.x`, as a version suffix; eg, `1.10.x`
might be `1.10` before the release of `1.10.1` but become `1.10.1` once that's
available.

To make this easier, and reduce duplicate invocations, Gimme now supports a
"query" which, instead of producing normal output, just prints the resolution
of a version specifier. This is the `--resolve` option. It handles the `.x`
suffix and the `stable` string; all other inputs are passed through unchanged,
although unknown names will be accompanied by an error message and an exit
code of 2.

Thus given a list of versions to invoke against, tooling might do a first pass
to use `--resolve` on each and de-duplicate, so that if an alias and a
hard-coded version map to the same version, then only one invocation needs to
happen.

Gimme only supports `.x` at the end of a version specifier.
The `--resolve` option must be given a version on the command-line afterwards,
not by any other means.
The `--resolve` option and mechanism ignores any installed versions and relies
solely upon upstream-exposed lists of available versions and resolvable tags.
A git tag named ending `.x` will never be found.
Use of `.x` will not find release candidates, alphas, betas or other
non-release versions: it's only for finding the last stable release.
Use of `${GIMME_TYPE}` to override `auto` and prevent `git` will affect
`--resolve` by inhibiting use of git tags as valid names. This is a feature.

Note that because Gimme supports version identifiers which are git tags,
`--resolve` defaults to handling this too. This means that `--resolve` can be
heavy-weight: without the Go repo cloned, first the entire Go repo must be
cloned. We default to "correct". To avoid this, export `GIMME_TYPE=binary`
and disable the git resolution mechanism.
169 changes: 142 additions & 27 deletions gimme
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
#+ -f --force force - remove the existing go installation if present prior to install
#+ -l --list list - list installed go versions and exit
#+ -k --known known - list known go versions and exit
#+ --force-known-update - when used with --known, ignores the cache and updates
#+ -r --resolve resolve - resolve a version specifier to a version, show that and exit
#+ -
#+ Influential env vars:
#+ -
Expand All @@ -41,6 +43,7 @@
#+ GIMME_CC_FOR_TARGET - cross compiler for cgo support
#+ GIMME_DOWNLOAD_BASE - override base URL dir for download (default '${GIMME_DOWNLOAD_BASE}')
#+ GIMME_LIST_KNOWN - override base URL for known go versions (default '${GIMME_LIST_KNOWN}')
#+ GIMME_KNOWN_CACHE_MAX - seconds the cache for --known is valid for (default '${GIMME_KNOWN_CACHE_MAX}')
#+ -
#
set -e
Expand All @@ -51,13 +54,21 @@ set -o pipefail

[[ ${GIMME_DEBUG} ]] && set -x

GIMME_VERSION="v1.3.0"
GIMME_COPYRIGHT="Copyright (c) 2015-2018 gimme contributors"
GIMME_LICENSE_URL="https://raw.githubusercontent.com/travis-ci/gimme/v1.3.0/LICENSE"
readonly GIMME_VERSION="v1.3.0"
readonly GIMME_COPYRIGHT="Copyright (c) 2015-2018 gimme contributors"
readonly GIMME_LICENSE_URL="https://raw.githubusercontent.com/travis-ci/gimme/${GIMME_VERSION}/LICENSE"
export GIMME_VERSION
export GIMME_COPYRIGHT
export GIMME_LICENSE_URL

program_name="$(basename "$0")"
# shellcheck disable=SC1117
warn() { printf >&2 "%s: %s\n" "${program_name}" "${*}"; }
die() {
warn "$@"
exit 1
}

# _do_curl "url" "file"
_do_curl() {
mkdir -p "$(dirname "${2}")"
Expand Down Expand Up @@ -92,6 +103,21 @@ _sha256sum() {
fi
}

# sort versions, handling 1.10 after 1.9, not before 1.2
# FreeBSD sort has --version-sort, none of the others do
# Looks like --general-numeric-sort is the safest; checked macOS 10.12.6, FreeBSD 10.3, Ubuntu Trusty
if sort --version-sort &>/dev/null; then
_version_sort() { sort --version-sort; }
else
_version_sort() {
# If we go to four-digit minor or patch versions, then extend the padding here
# (but in such a world, perhaps --version-sort will have become standard by then?)
sed -E 's/\.([0-9](\.|$))/.00\1/g; s/\.([0-9][0-9](\.|$))/.0\1/g' |
sort --general-numeric-sort |
sed 's/\.00*/./g'
}
fi

# _do_curls "file" "url" ["url"...]
_do_curls() {
f="${1}"
Expand Down Expand Up @@ -430,25 +456,94 @@ _list_versions() {
done
}

_list_known() {
_update_remote_known_list_if_needed() {
# shellcheck disable=SC1117
local exp="go([[:alnum:]\.]*)\.src.*" # :alnum: catches beta versions too
local list="${GIMME_TMP}/known-versions"
local list="${GIMME_VERSION_PREFIX}/known-versions.txt"
local dlfile="${GIMME_TMP}/known-dl"

local known
known="$(_list_versions 2>/dev/null)"
if [[ -e "${list}" ]] &&
! ((force_known_update)) &&
! _file_older_than_secs "${list}" "${GIMME_KNOWN_CACHE_MAX}"; then
echo "${list}"
return 0
fi

_do_curl "${GIMME_LIST_KNOWN}" "${list}"
_do_curl "${GIMME_LIST_KNOWN}" "${dlfile}"

while read -r line; do
if [[ "${line}" =~ ${exp} ]]; then
# shellcheck disable=SC1117
known="$known\n${BASH_REMATCH[1]}"
echo "${BASH_REMATCH[1]}"
fi
done <"${list}"

done <"${dlfile}" | _version_sort | uniq >"${list}.new"
rm -f "${list}" &>/dev/null
echo -e "${known}" | grep . | sort -n -r | uniq
mv "${list}.new" "${list}"

rm -f "${dlfile}"
echo "${list}"
return 0
}

_list_known() {
local knownfile
knownfile="$(_update_remote_known_list_if_needed)"

(
_list_versions 2>/dev/null
cat -- "${knownfile}"
) | grep . | _version_sort | uniq
}

# For the "invoked on commandline" case, we want to always pass unknown
# strings through, so that we can be a uniqueness filter, but for unknown
# names we want to exit with a value other than 1, so we document that
# we'll exit 2. For use by other functions, 2 is as good as 1.
_resolve_version() {
case "${1}" in
stable)
_get_curr_stable
return 0
;;
tip)
echo "tip"
return 0
;;
*.x)
true
;;
*)
echo "${1}"
local GIMME_GO_VERSION="$1"
local ASSERT_ABORT='return'
if _assert_version_given 2>/dev/null; then
return 0
fi
warn "version specifier '${1}' unknown"
return 2
;;
esac
# We have a .x suffix
local base="${1%.x}"
local ver last='' known
known="$(_update_remote_known_list_if_needed)" # will be version-sorted
# avoid regexp attacks
while read -r ver; do
case "${ver}" in
${base})
last="${ver}"
;;
${base}.*)
last="${ver}"
;;
esac
done <"$known"
if [[ -n "${last}" ]]; then
echo "${last}"
return 0
fi
echo "${1}"
warn "given '${1}' but no release for '${base}' found"
return 2
}

_realpath() {
Expand All @@ -458,14 +553,8 @@ _realpath() {

_get_curr_stable() {
local stable="${GIMME_VERSION_PREFIX}/stable"
local now_secs
now_secs="$(date +%s)"
local stable_age
stable_age="$(_stat_unix "${stable}" 2>/dev/null || echo 0)"
local age
age=$((now_secs - stable_age))

if [[ "${age}" -gt 86400 ]]; then

if _file_older_than_secs "${stable}" 86400; then
_update_stable "${stable}"
fi

Expand All @@ -478,20 +567,27 @@ _update_stable() {

_do_curl "${url}" "${stable}"
sed -i.old -e 's/^go\(.*\)/\1/' "${stable}"
rm -f "${stable}.old"
}

_stat_unix() {
_last_mod_timestamp() {
local filename="${1}"
case "${GIMME_HOSTOS}" in
darwin | *bsd)
stat -f %a "${filename}"
stat -f %m "${filename}"
;;
linux)
stat -c %Y "${filename}"
;;
esac
}

_file_older_than_secs() {
local file="${1}"
local age_secs="${2}"
((($(date +%s) - $(_last_mod_timestamp "${file}")) > age_secs))
}

_assert_version_given() {
# By the time we're called, aliases such as "stable" must have been resolved
# but we could be a reference in git.
Expand All @@ -504,7 +600,14 @@ _assert_version_given() {
echo >&2 'error: no GIMME_GO_VERSION supplied'
echo >&2 " ex: GIMME_GO_VERSION=1.4.1 ${0} ${*}"
echo >&2 " ex: ${0} 1.4.1 ${*}"
exit 1
${ASSERT_ABORT:-exit} 1
fi

# Note: _resolve_version calls back to us (_assert_version_given), but
# only for cases where the version does not end with .x, so this should
# be safe.
if [[ "${GIMME_GO_VERSION}" == *.x ]]; then
GIMME_GO_VERSION="$(_resolve_version "${GIMME_GO_VERSION}")" || ${ASSERT_ABORT:-exit} 1
fi

if [[ "${GIMME_GO_VERSION}" == +([[:digit:]]).+([[:digit:]])* ]]; then
Expand All @@ -518,7 +621,7 @@ _assert_version_given() {

echo >&2 'error: GIMME_GO_VERSION not recognized as valid'
echo >&2 " got: ${GIMME_GO_VERSION}"
exit 1
${ASSERT_ABORT:-exit} 1
}

_exclude_from_backups() {
Expand Down Expand Up @@ -558,6 +661,7 @@ _to_goarch() {
: "${GIMME_BINARY_OSX:=osx10.8}"
: "${GIMME_DOWNLOAD_BASE:=https://storage.googleapis.com/golang}"
: "${GIMME_LIST_KNOWN:=https://golang.org/dl}"
: "${GIMME_KNOWN_CACHE_MAX:=10800}"

# The version prefix must be an absolute path
case "${GIMME_VERSION_PREFIX}" in
Expand All @@ -580,6 +684,9 @@ if [[ "${GIMME_OS}" == mingw* ]]; then
fi
fi

force_install=0
force_known_update=0

while [[ $# -gt 0 ]]; do
case "${1}" in
-h | --help | help | wat)
Expand All @@ -599,6 +706,11 @@ while [[ $# -gt 0 ]]; do
echo "${GIMME_VERSION}"
exit 0
;;
-r | --resolve | resolve)
[[ $# -eq 2 ]] || die "resolve must be given an option afterwards (and nothing else)"
_resolve_version "${2}"
exit $?
;;
-l | --list | list)
_list_versions
exit 0
Expand All @@ -608,7 +720,10 @@ while [[ $# -gt 0 ]]; do
exit 0
;;
-f | --force | force)
force=1
force_install=1
;;
--force-known-update | force-known-update)
force_known_update=1
;;
-i | install)
true # ignore a dummy argument
Expand Down Expand Up @@ -656,7 +771,7 @@ fi

_assert_version_given "$@"

[ ${force} ] && _wipe_version "${GIMME_GO_VERSION}"
((force_install)) && _wipe_version "${GIMME_GO_VERSION}"

unset GOARCH
unset GOBIN
Expand Down

0 comments on commit 54c6bee

Please sign in to comment.