Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add binary package automated testing #1995

Open
derekbruening opened this issue Aug 29, 2016 · 0 comments
Open

add binary package automated testing #1995

derekbruening opened this issue Aug 29, 2016 · 0 comments

Comments

@derekbruening
Copy link
Contributor

derekbruening commented Aug 29, 2016

The testing we perform prior to a release (see
https://github.com/DynamoRIO/dynamorio/wiki/New-Release) consists mainly of
sanity checks running all of the samples and polished tools on both small
console apps but also on graphical apps like notepad. It would be nice to
automate that.

Xref adding graphical app testing to the pre-commit suite: #120.

This issue is about adding binary package automated testing, as opposed to
the source code building and testing of our regular test suite. The goal
here is to have a script that can be run on VM's that do not necessarily
have a development environment set up, for sanity testing on a wide range
of platforms.

Step 1 is to avoid Windows popups by making SHOW_RESULTS a runtime option
(separately but with the same name in each sample I suppose). Even if the
rest of the automation isn't finished, that alone would make testing
easier.

Step 2 is to take the current sequences of commands and turn them into a
more formal script with some automated checking of results.

Step 3 is to set up cron jobs to run in virtual machines periodically.
Perhaps when each "nightly build" is created: #1967?

We do not expect to be able to easily automate some pieces:

  • .msi install and uninstall (for Dr. Memory)
  • Visual Studio external tool testing (for Dr. Memory)
  • admin -syswide_on testing
  • cmd interactive output testing (less of an issue on recent Windows)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant