Mocking the GitHub communications for functional tests

[karlcow]

When we are testing the UI with intern, we want to make sure that the UI reacts accordingly to what we want of the user experience. Right now, the tests are dependent of Github communication but it's not necessary most of the time for the suer experience. Communications issues in between webcompat and github have to be dealt with elsewhere.

1. Mock all response from Github to WebCompat in intern (when possible)

There is two features which are currently directly talking to Github.

• search everywhere (in progress)
• load search results from URI bar

decided in Paris september 2015 at webcompat meeting with @miketaylr

[karlcow]

In issue #663 we wanted to make a canonical test repo. But if we mock the correct issues, we kind of make #663 irrelevant. Let's close #663

[miketaylr]

I spent a lot of time today looking at mocking modules and servers, and too many of them make the assumption that we want to mock out python unit tests. So instead I came up with the following idea:

We have a @api_mock('file.json') decorator for API endpoints that checks if we're in TESTING mode, and if so it serves that file instead of requesting it from GitHub. It serves them from tests/fixtures/.

Right now this is a READ-only solution, so it won't work for all tests or endpoints. But it's a start.

[miketaylr]

Also need to figure out how to mock logins, and how to decide when to send some kind of fixture-auth.js and fixture-non-auth.js, etc.

[miketaylr]

Also figure out why this breaks nose:

..E......................
======================================================================
ERROR: API access to labels without auth returns JSON 401.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/miket/dev/compat/webcompat.com/tests/test_api_urls.py", line 68, in test_api_labels_without_auth
    json_body = json.loads(rv.data)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

----------------------------------------------------------------------
Ran 25 tests in 1.128s

[miketaylr]

Also need to figure out how... to decide when to send some kind of fixture-auth.js and fixture-non-auth.js, etc.

One ugly idea is to extend the decorator to take 2 args, and then decide which file to serve based on g.user:

@api_mock(auth_fixture='file-auth.json', non_auth_fixture='file-non-auth.json')

[miketaylr]

Also figure out why this breaks nose

That's likely breaking because we're just serving the same file to authed and non-authed. The suggestion in my previous comment would fix it.

[miketaylr]

OK, just updated this so it's like @api_mock('file-auth.json', 'file-non-auth.json'). None can be used for either value.

[miketaylr]

Hmm, not super happy with this. It sort of falls apart with routes like @api.route('/issues/category/<issue_category>'). Thinking thinking. 💭

[miketaylr]

OK, so the way around that is to do something like

webcompat.com/webcompat/api/endpoints.py

Lines 141 to 144 in 6dcfa73

	@api.route('/issues/category/new')
	@api_mock(None, 'category-new.json')
	def get_new_issues():
	return get_issue_category('new')

. Let's do a few more endpoints and see how it goes.

[miketaylr]

Another way to approach this is instead of passing in a string that refers to a file is to pass in the API route/request URL. That way we can handle fixtures for multiple issues, for example -- the func tests look for 69, 200, 100, 101, etc. We could use a little magick™ to detect auth state, and if not authed look for a -non-auth.json resource.

Everybody loves magical APIs. ✨

[miketaylr]

Or maybe I'm overthinking this. endpoints.py needs to worry about auth vs. non-auth. Let's just send a single fixture resource.