Testing: implement automated tests + write manual functional test instructions

[helen]

Popular plugin that is supposed to help you with site privacy with an array of options - very important we test all situations and ensure they continue to work as expected through changes in the plugin and WordPress core, both in an automated fashion and manually during release prep.

I am not sure that automated tests can be thorough without actually bootstrapping WordPress, i.e. I'm not sure WP_Mock will be good enough to catch things like #53 proactively. If it is, that's great! But my sense is that integration tests will be just as, if not more, important as unit tests, because if core changes the way redirects or query var filling works, we can't mock that.

Here's what needs to be tested (we can turn this into a proper matrix at some point), please feel free to note anything I've neglected:

RSA technical features

• Redirect to login
• Redirect to URL
• Redirect to page (see page types below)
• Show a message
• IP whitelisting (option and config-set)
• Network-enforced settings (multisite)
• WP CLI integration
  • List
  • Add (existing, new, invalid)
  • Remove (existing, non-existing, config-set, invalid)
  • Various flags

WordPress features

• Single site
• Multisite
• User with no role on site (multisite)
• Different page types, to and from (page on front, page for posts, BuddyPress-style virtual pages, regular pages)

RSA UI features

• Settings
  • IP add/remove
  • Config-set IPs
  • Page selection
  • Radio selection
  • Multisite
• Simple message display (especially with different HTML tags)

[petenelson]

I can take a stab at this. I've been doing a lot more unit testing recently.

[petenelson]

While working on this, I found a bug in the code that checks to see if we're on the existing page. If there isn't a permalink structure set, it gets caught in a redirect loop. I'll address this as part of the pull request.

// Are we already on the selected page?
// There's a separate unpleasant conditional to match the page on front because of the way query vars are (not) filled at this point
if (
	( isset( $wp->query_vars['pagename'] ) && $wp->query_vars['pagename'] === $page->post_name )
	||
	( empty ( $wp->query_vars ) && 'page' === get_option( 'show_on_front' ) && (int) self::$rsa_options['page'] === (int) get_option( 'page_on_front' ) )
	) {
	return;
}

[petenelson]

I have about 48% code coverage so far and automated tests running here: https://travis-ci.org/petenelson/restricted-site-access

This branch has all of the new unit tests code and is still a work-in-progress. I'll get a PR opened for this once it's closer to full code coverage.

[helen]

Punting this out of 7.1.0 because it's not really tied to a milestone and I want to continue using it as an umbrella for implementing WP Acceptance tests and hopefully reducing the fragile HTML-based tests.

[helen]

Also a note that Gruntfile.js references a phpunit-boostrap.php that doesn't seem to exist anymore.