Service tests (#937)

- Eliminate manual testing which is error-prone and time consuming, and must be repeated many times through the PR review process
- Make contributing more fun. For many, fixing bugs and making new badges is faster and more satisfying with automated tests than with manual testing.
- Push out the work of testing new badges to a much broader net. The PR originator could write tests, but so could any other contributor who wants to push review along.
- Detect badge failures resulting from changes in vendor contracts without waiting for user reports.
- Detect and prevent regressions in the code.
- Be runnable, readable, writable, and editable by as many developers as possible, including those who may not be familiar with JavaScript test tools.

-- @paulmelnikow, @niccokunzmann, @Daniel15

.istanbul.yml

@@ -0,0 +1,2 @@
+instrumentation:
+  excludes: ['lib/suggest.js', 'vendor/**/*.js']

.travis.yml

@@ -5,6 +5,8 @@ node_js:
 script:
   - npm run lint
   - npm run test:js
+  - if [ "$TRAVIS_EVENT_TYPE" == cron ]; then npm run test:services; fi
+  - if [ "$TRAVIS_EVENT_TYPE" == pull_request ]; then npm run test:services -- --pr; fi
 
 branches:
   except:
@@ -14,6 +16,12 @@ notifications:
   email:
     - thaddee.tyl@gmail.com
   irc: "irc.freenode.org#shields"
+  slack:
+    rooms:
+      # One builds on badges/shields, the other for paulmelnikow/shields
+      secure: VyTRZfcRAIrYTgfYG5RNJrKd0boDYNBfWEhzm20aLJvDeI2TgwAB2bSClkjjBe9KD3vlEjW6WbOdXfc/pBEdiIYgxl0vjg1qv3qJLJ4a6oacJg3UukJim28SucxZL2yhyseahHt12HyuPdUtwy+QzhJRp6Jo7T7M7ClKh0/qI4M=
+      secure: Z494jqFmkiIrBEmhTW7zKTlewskUo/lFCs/IWgEA/zUUEPHIDYNJjjE47Lkd7WlKoD3k2Cpb5/HJ8vwXAGmOdMG7JAxzlL0etJYLgrE/MJ0+TDDAj9jCyS6G5OUzxAbcThSadQgC8umtQmZ+IzEyo2rBPh5VknfL/SN4QS8L83f8mdo/JKnfYR81Z05g+CXfBWQALDJzlRT0nzgSSWUjrIJKMCNlvpNnL/rrAeHhc1eYernUsZpiOVJVlkMWxuBmQHwntWp7aV9/yCuzz5w+tlYlHwYJySE2JP3LUb9z0vuueYL1YEXQMdcBcvb3SLBg1ZUPRKk+OwEI4U1U002BOZnr/uIqYGsAWL90W6AfjVA2mNwf0wMgD981SZLuqovZadvQ3+hxjl0oyb8eVc4uDXnKyrCuwI4gqc/T6RomfCs1bXlHsQrGp03l0IT5ADkzfTDS0VFHYYB5Og3cJodC0SF/m2s9ZnEhMtWnTdz4sOhRFV1GccODI+5lz6F/XVdaY2zon9zkRjpoZKXsjCpPMc/OTXKVwV90Y0vots3vlyaO3eJvlH/qkFkZyavHeSSl9vo50QUfcVGY5AgoNuaKwz4RwzklGTsW06TV4jHpw8SMBnapNDVziMUzSQ4nUvQYtJofmpxT2K/2JImtIyfvO489uQZjgr5WkXaPMUaGWdc=
+    on_success: change
 
 git:
   depth: 10

doc/TUTORIAL.md

@@ -217,6 +217,24 @@ Edit [try.html][tryhtml] in the right section (Build, Downloads, ...) and add yo
 
 Save, restart and you can see it [locally][try].
 
+## (4.4) Write Tests
+
+When creating a badge for a new service or changing a badge's behavior, tests
+should be included. They serve several purposes:
+
+1. They speed up future contributors when they are debugging or improving a
+   badge.
+2. If a contributors like to change your badge, chances are, they forget
+   edge cases and break your code.
+   Tests may give hints in such cases.
+3. The contributor and reviewer can easily verify the code works as
+   intended.
+4. When a badge stops working on the live server, maintainers can find out
+   right away.
+
+There is a dedicated [tutorial for tests in the service-tests folder][tests-tutorial].
+Please follow it to include tests on your pull-request.
+
 ## (5) Create a Pull Request
 
 You have implemented changes in `server.js`, `try.html` and `index.html`.
@@ -249,4 +267,5 @@ These files can also be of help for creating your own badge.
 [new-badge]: https://github.com/badges/shields/pulls?q=is%3Apr+label%3Anew-badge
 [docker-example]: https://github.com/badges/shields/blob/bf373d11cd522835f198b50b4e1719027a0a2184/server.js#L5014
 [travis-example]: https://github.com/badges/shields/blob/bf373d11cd522835f198b50b4e1719027a0a2184/server.js#L431
-[regex]: https://www.w3schools.com/jsref/jsref_obj_regexp.asp
+[regex]: https://www.w3schools.com/jsref/jsref_obj_regexp.asp
+[tests-tutorial]: ../service-tests/#readme

package.json

@@ -36,8 +36,15 @@
     "xml2js": "~0.4.16"
   },
   "scripts": {
+    "coverage:test:js": "istanbul cover _mocha 'test/**/*.spec.js' --dir coverage/js",
+    "coverage:test:services": "istanbul cover _mocha --delay service-tests/runner/cli.js --dir coverage/services",
+    "coverage:test": "npm run coverage:test:js && npm run coverage:test:services",
+    "coverage:report": "istanbul report",
+    "coverage:report:reopen": "opn coverage/lcov-report/index.html",
+    "coverage:report:open": "npm run coverage:report && npm run coverage:report:reopen",
     "lint": "eslint '**/*.js'",
     "test:js": "mocha 'test/**/*.spec.js'",
+    "test:services": "mocha --delay service-tests/runner/cli.js",
     "test": "npm run lint && npm run test:js"
   },
   "bin": {
@@ -56,9 +63,17 @@
   ],
   "devDependencies": {
     "eslint": "^3.18.0",
-    "is-png": "^1.0.0",
+    "glob": "^7.1.1",
+    "icedfrisby": "^1.1.0",
+    "icedfrisby-nock": "^0.3.0",
+    "is-png": "^1.1.0",
     "is-svg": "^2.1.0",
+    "istanbul": "^0.4.5",
+    "lodash.difference": "^4.5.0",
     "mocha": "^3.2.0",
+    "nock": "^9.0.13",
+    "node-fetch": "^1.6.3",
+    "opn-cli": "^3.1.0",
     "sinon": "^2.1.0"
   }
 }