Skip to content

Assert and stack traces for Bash scripts #12142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 26, 2021
Merged

Assert and stack traces for Bash scripts #12142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9aad8d6
common.sh: printStackTrace() and assertFail()
cameel Oct 14, 2021
b85172b
common.sh: Print the source code from locations indicated in the stac…
cameel Oct 14, 2021
3b1b9a0
common.sh: Adjust formatting in msg_on_error() and add stack trace
cameel Oct 14, 2021
0280c8d
Use fail and assertFail where appropriate in command-line tests
cameel Oct 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 71 additions & 7 deletions scripts/common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@
# (c) 2016-2019 solidity contributors.
# ------------------------------------------------------------------------------

# The fail() function defined below requires set -e to be enabled.
set -e

# Save the initial working directory so that printStackTrace() can access it even if the sourcing
# changes directory. The paths returned by `caller` are relative to it.
_initial_work_dir=$(pwd)

if [ "$CIRCLECI" ]
then
export TERM="${TERM:-xterm}"
Expand All @@ -33,12 +40,63 @@ else
function printLog() { echo "$(tput setaf 3)$1$(tput sgr0)"; }
fi

function printStackTrace
{
printWarning ""
printWarning "Stack trace:"

local frame=1
while caller "$frame" > /dev/null
do
local lineNumber line file function

# `caller` returns something that could already be printed as a stacktrace but we can make
# it more readable by rearranging the components.
# NOTE: This assumes that paths do not contain spaces.
lineNumber=$(caller "$frame" | cut --delimiter " " --field 1)
function=$(caller "$frame" | cut --delimiter " " --field 2)
file=$(caller "$frame" | cut --delimiter " " --field 3)

# Paths in the output from `caller` can be relative or absolute (depends on how the path
# with which the script was invoked) and if they're relative, they're not necessarily
# relative to the current working dir. This is a heuristic that will work if they're absolute,
# relative to current dir, or relative to the dir that was current when the script started.
# If neither works, it gives up.
line=$(
{
tail "--lines=+${lineNumber}" "$file" ||
tail "--lines=+${lineNumber}" "${_initial_work_dir}/${file}"
} 2> /dev/null |
head --lines=1 |
sed -e 's/^[[:space:]]*//'
) || line="<failed to find source line>"

>&2 printf " %s:%d in function %s()\n" "$file" "$lineNumber" "$function"
>&2 printf " %s\n" "$line"

((frame++))
done
}

function fail()
{
printError "$@"

# Using return rather than exit lets the invoking code handle the failure by suppressing the exit code.
return 1
}

function assertFail()
{
printError ""
(( $# == 0 )) && printError "Assertion failed."
(( $# == 1 )) && printError "Assertion failed: $1"
printStackTrace

# Intentionally using exit here because assertion failures are not supposed to be handled.
exit 2
}

function msg_on_error()
{
local error_message
Expand Down Expand Up @@ -67,7 +125,7 @@ function msg_on_error()
shift
;;
*)
fail "Invalid option for msg_on_error: $1"
assertFail "Invalid option for msg_on_error: $1"
;;
esac
done
Expand All @@ -85,24 +143,30 @@ function msg_on_error()
rm "$stdout_file" "$stderr_file"
return 0
else
printError "Command failed: $SOLC ${command[*]}"
printError ""
printError "Command failed: ${error_message}"
printError " command: $SOLC ${command[*]}"
if [[ -s "$stdout_file" ]]
then
printError "stdout:"
printError "--- stdout ---"
printError "-----------"
>&2 cat "$stdout_file"
printError "--------------"
else
printError "stdout: <EMPTY>"
printError " stdout: <EMPTY>"
fi
if [[ -s "$stderr_file" ]]
then
printError "stderr:"
printError "--- stderr ---"
>&2 cat "$stderr_file"
printError "--------------"
else
printError "stderr: <EMPTY>"
printError " stderr: <EMPTY>"
fi

printError "$error_message"
rm "$stdout_file" "$stderr_file"

printStackTrace
return 1
fi
}
Expand Down
37 changes: 20 additions & 17 deletions test/cmdlineTests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ echo "Using solc binary at ${SOLC}"
INTERACTIVE=true
if ! tty -s || [ "$CI" ]
then
INTERACTIVE=""
INTERACTIVE=false
fi

# extend stack size in case we run via ASAN
Expand Down Expand Up @@ -123,7 +123,7 @@ function update_expectation {

function ask_expectation_update
{
if [[ $INTERACTIVE != "" ]]
if [[ $INTERACTIVE == true ]]
then
local newExpectation="${1}"
local expectationFile="${2}"
Expand All @@ -142,12 +142,13 @@ function ask_expectation_update
e*) "$editor" "$expectationFile"; break;;
u*) update_expectation "$newExpectation" "$expectationFile"; break;;
s*) return;;
q*) exit 1;;
q*) fail;;
esac
done
fi
else
exit 1
[[ $INTERACTIVE == false ]] || assertFail
Copy link
Collaborator Author

@cameel cameel Oct 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I implemented a full assert that you would invoke like this:

assert "[[ $INTERACTIVE == false ]]"

It could print the condition if you did not provide a message, which was nice.

The problem is that it required eval to check the condition and also that quoting was tricky - you had to quote the whole condition with "" (otherwise the variables would be evaluated inside the assert rather than in the current function) and then use '' over any variable used inside that could evaluate to an empty string.

In the end I thought that just a simple assertFail that prints a stack trace is enough. It still clearly indicates that the condition you check is an assertion and condition evaluation has no quirks (at least not more than usually in Bash). Also, the condition itself can be seen in the stack trace.

fail
fi
}

Expand Down Expand Up @@ -252,7 +253,7 @@ EOF
printError "Incorrect exit code. Expected $exit_code_expected but got $exitCode."

[[ $exit_code_expectation_file != "" ]] && ask_expectation_update "$exitCode" "$exit_code_expectation_file"
[[ $exit_code_expectation_file == "" ]] && exit 1
[[ $exit_code_expectation_file == "" ]] && fail
fi

if [[ "$(cat "$stdout_path")" != "${stdout_expected}" ]]
Expand All @@ -266,7 +267,7 @@ EOF
printError "When running $solc_command"

[[ $stdout_expectation_file != "" ]] && ask_expectation_update "$(cat "$stdout_path")" "$stdout_expectation_file"
[[ $stdout_expectation_file == "" ]] && exit 1
[[ $stdout_expectation_file == "" ]] && fail
fi

if [[ "$(cat "$stderr_path")" != "${stderr_expected}" ]]
Expand All @@ -280,7 +281,7 @@ EOF
printError "When running $solc_command"

[[ $stderr_expectation_file != "" ]] && ask_expectation_update "$(cat "$stderr_path")" "$stderr_expectation_file"
[[ $stderr_expectation_file == "" ]] && exit 1
[[ $stderr_expectation_file == "" ]] && fail
fi

rm "$stdout_path" "$stderr_path"
Expand All @@ -300,10 +301,10 @@ function test_solc_assembly_output()
if [ -z "$empty" ]
then
printError "Incorrect assembly output. Expected: "
echo -e "${expected}"
>&2 echo -e "${expected}"
printError "with arguments ${solc_args[*]}, but got:"
echo "${output}"
exit 1
>&2 echo "${output}"
fail
fi
}

Expand Down Expand Up @@ -373,7 +374,7 @@ printTask "Running general commandline tests..."
then
printError "Ambiguous input. Found input files in multiple formats:"
echo -e "${inputFiles}"
exit 1
fail
fi

# Use printf to get rid of the trailing newline
Expand Down Expand Up @@ -475,7 +476,8 @@ echo "Done."

printTask "Testing library checksum..."
echo '' | msg_on_error --no-stdout "$SOLC" - --link --libraries a=0x90f20564390eAe531E810af625A22f51385Cd222
echo '' | "$SOLC" - --link --libraries a=0x80f20564390eAe531E810af625A22f51385Cd222 &>/dev/null && exit 1
echo '' | "$SOLC" - --link --libraries a=0x80f20564390eAe531E810af625A22f51385Cd222 &>/dev/null && \
fail "solc --link did not reject a library address with an invalid checksum."

printTask "Testing long library names..."
echo '' | msg_on_error --no-stdout "$SOLC" - --link --libraries aveeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeerylonglibraryname=0x90f20564390eAe531E810af625A22f51385Cd222
Expand Down Expand Up @@ -503,7 +505,8 @@ SOLTMPDIR=$(mktemp -d)
# First time it works
echo 'contract C {}' | msg_on_error --no-stderr "$SOLC" - --bin -o "$SOLTMPDIR/non-existing-stuff-to-create"
# Second time it fails
echo 'contract C {}' | "$SOLC" - --bin -o "$SOLTMPDIR/non-existing-stuff-to-create" 2>/dev/null && exit 1
echo 'contract C {}' | "$SOLC" - --bin -o "$SOLTMPDIR/non-existing-stuff-to-create" 2>/dev/null && \
fail "solc did not refuse to overwrite $SOLTMPDIR/non-existing-stuff-to-create."
# Unless we force
echo 'contract C {}' | msg_on_error --no-stderr "$SOLC" - --overwrite --bin -o "$SOLTMPDIR/non-existing-stuff-to-create"
)
Expand All @@ -517,8 +520,8 @@ printTask "Testing assemble, yul, strict-assembly and optimize..."

# Test options above in conjunction with --optimize.
# Using both, --assemble and --optimize should fail.
echo '{}' | "$SOLC" - --assemble --optimize &>/dev/null && exit 1
echo '{}' | "$SOLC" - --yul --optimize &>/dev/null && exit 1
echo '{}' | "$SOLC" - --assemble --optimize &>/dev/null && fail "solc --assemble --optimize did not fail as expected."
echo '{}' | "$SOLC" - --yul --optimize &>/dev/null && fail "solc --yul --optimize did not fail as expected."

# Test yul and strict assembly output
# Non-empty code results in non-empty binary representation with optimizations turned off,
Expand Down Expand Up @@ -563,8 +566,8 @@ SOLTMPDIR=$(mktemp -d)
cd "$SOLTMPDIR"
if ! "$REPO_ROOT/scripts/ASTImportTest.sh"
then
rm -rf "$SOLTMPDIR"
exit 1
rm -r "$SOLTMPDIR"
fail
fi
)
rm -r "$SOLTMPDIR"
Expand Down