Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-116622: Android test script improvements #124012

Merged
merged 4 commits into from
Sep 13, 2024
Merged

gh-116622: Android test script improvements #124012

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
acb3419
Set Android test script stdout to line-buffered
mhsmith Sep 12, 2024
8e4bebe
Print warning logcat messages on stderr
mhsmith Sep 12, 2024
ae499f9
Add a -vv option to display high-volume messages which are rarely useful
mhsmith Sep 12, 2024
537e149
Documentation and comment improvements
mhsmith Sep 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions Android/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,12 @@ approachable user experience:

## Prerequisites

Export the `ANDROID_HOME` environment variable to point at your Android SDK. If
you don't already have the SDK, here's how to install it:
First, make sure you have all the usual tools and libraries needed to build
Python for your development machine.

Second, you'll need an Android SDK. If you already have the SDK installed,
export the `ANDROID_HOME` environment variable to point at its location.
Otherwise, here's how to install it:

* Download the "Command line tools" from <https://developer.android.com/studio>.
* Create a directory `android-sdk/cmdline-tools`, and unzip the command line
Expand All @@ -37,11 +41,6 @@ development tools, which currently means Linux or macOS. This involves doing a
cross-build where you use a "build" Python (for your development machine) to
help produce a "host" Python for Android.

First, make sure you have all the usual tools and libraries needed to build
Python for your development machine. The only Android tool you need to install
is the command line tools package above: the build script will download the
rest.

The easiest way to do a build is to use the `android.py` script. You can either
have it perform the entire build process from start to finish in one step, or
you can do it in discrete steps that mirror running `configure` and `make` for
Expand Down Expand Up @@ -80,12 +79,15 @@ call. For example, if you want a pydebug build that also caches the results from

## Testing

The tests can be run on Linux, macOS, or Windows, although on Windows you'll
have to build the `cross-build/HOST` subdirectory on one of the other platforms
and copy it over.
The test suite can be run on Linux, macOS, or Windows:

* On Linux, the emulator needs access to the KVM virtualization interface, and
a DISPLAY environment variable pointing at an X server.
* On Windows, you won't be able to do the build on the same machine, so you'll
have to copy the `cross-build/HOST` directory from somewhere else.

The test suite can usually be run on a device with 2 GB of RAM, though for some
configurations or test orders you may need to increase this. As of Android
The test suite can usually be run on a device with 2 GB of RAM, but this is
borderline, so you may need to increase it to 4 GB. As of Android
Studio Koala, 2 GB is the default for all emulators, although the user interface
may indicate otherwise. The effective setting is `hw.ramSize` in
~/.android/avd/*.avd/hardware-qemu.ini, whereas Android Studio displays the
Expand Down
25 changes: 20 additions & 5 deletions Android/android.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,8 @@ def setup_testbed():
f"{temp_dir}/{outer_jar}", "gradle-wrapper.jar"])


# run_testbed will build the app automatically, but it hides the Gradle output
# by default, so it's useful to have this as a separate command for the buildbot.
# run_testbed will build the app automatically, but it's useful to have this as
# a separate command to allow running the app outside of this script.
def build_testbed(context):
setup_sdk()
setup_testbed()
Expand Down Expand Up @@ -376,6 +376,8 @@ async def find_pid(serial):
shown_error = False
while True:
try:
# `pidof` requires API level 24 or higher. The level 23 emulator
# includes it, but it doesn't work (it returns all processes).
pid = (await async_check_output(
adb, "-s", serial, "shell", "pidof", "-s", APP_ID
)).strip()
Expand Down Expand Up @@ -407,6 +409,7 @@ async def logcat_task(context, initial_devices):
serial = await wait_for(find_device(context, initial_devices), startup_timeout)
pid = await wait_for(find_pid(serial), startup_timeout)

# `--pid` requires API level 24 or higher.
args = [adb, "-s", serial, "logcat", "--pid", pid, "--format", "tag"]
hidden_output = []
async with async_process(
Expand All @@ -421,11 +424,15 @@ async def logcat_task(context, initial_devices):
# such messages, but other components might.
level, message = None, line

# Exclude high-volume messages which are rarely useful.
if context.verbose < 2 and "from python test_syslog" in message:
continue

# Put high-level messages on stderr so they're highlighted in the
# buildbot logs. This will include Python's own stderr.
stream = (
sys.stderr
if level in ["E", "F"] # ERROR and FATAL (aka ASSERT)
if level in ["W", "E", "F"] # WARNING, ERROR, FATAL (aka ASSERT)
else sys.stdout
)

Expand Down Expand Up @@ -573,8 +580,9 @@ def parse_args():
test = subcommands.add_parser(
"test", help="Run the test suite")
test.add_argument(
"-v", "--verbose", action="store_true",
help="Show Gradle output, and non-Python logcat messages")
"-v", "--verbose", action="count", default=0,
help="Show Gradle output, and non-Python logcat messages. "
"Use twice to include high-volume messages which are rarely useful.")
Comment on lines +583 to +585
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this is merged, I'll switch the the buildbot over to -v mode so it will capture all non-Python log messages except the high-volume ones from test_syslog. These messages have been critical for understanding test failures in the past, including SELinux errors, low memory warnings, and signals being delivered to the wrong thread.

Since -v also displays the Gradle build log, there will no longer be any need for the buildbot to have a separate build_testbed step, hence the change to its comment above.

device_group = test.add_mutually_exclusive_group(required=True)
device_group.add_argument(
"--connected", metavar="SERIAL", help="Run on a connected device. "
Expand All @@ -591,6 +599,13 @@ def parse_args():

def main():
install_signal_handler()

# Under the buildbot, stdout is not a TTY, but we must still flush after
# every line to make sure our output appears in the correct order relative
# to the output of our subprocesses.
for stream in [sys.stdout, sys.stderr]:
stream.reconfigure(line_buffering=True)

context = parse_args()
dispatch = {"configure-build": configure_build_python,
"make-build": make_build_python,
Expand Down
Loading