build(codecov): switch from coveralls to codecov

Coveralls.io cannot be easily used by GitHub Actions because the secret token is not passed to PRs which are opened by third-party forks (#392). Furthermore, Coveralls.io's GitHub Action only supports LCOV files (coverallsapp/github-action#30). But, coverage.py generates .coverage files. 

Since they are roughly equivalent in functionality for our purpose, we switch from coveralls.io to CodeCov.

.codecov.yaml

@@ -0,0 +1,12 @@
+coverage:
+  status:
+    project:
+      default:
+        target: 85%
+        threshold: 1%
+    patch:
+      default:
+        threshold: 2%
+
+comment:
+  layout: "reach, diff, flags, files"

.coveragerc

@@ -1,3 +1,28 @@
 [run]
+branch = True
 concurrency = multiprocessing
 source = fonduer
+
+[report]
+# Regexes for lines to exclude from consideration
+exclude_lines =
+    # Have to re-enable the standard pragma
+    pragma: no cover
+
+    # Don't complain about missing debug-only code:
+    def __repr__
+    if self\.debug
+
+    # Don't complain if tests don't hit defensive assertion code:
+    raise AssertionError
+    raise NotImplementedError
+
+    # Don't complain if non-runnable code isn't run:
+    if 0:
+    if __name__ == .__main__.:
+
+ignore_errors = True
+
+omit =
+    # Omit anything in the test directory
+    tests/*

.github/workflows/pythonpackage.yml

@@ -53,7 +53,7 @@ jobs:
     - name: Install Fonduer
       run: |
         make dev_extra
-        pip install -q coveralls
+        pip install -q pytest-cov
     - name: Run preliminary checks
       run: |
         make check
@@ -70,20 +70,20 @@ jobs:
         PGPASSWORD: postgres
     - name: Test with pytest
       run: |
-        coverage run -m pytest tests
+        pytest --cov=./ --cov-report=xml
       env:
         PGPASSWORD: postgres
         PGUSER: postgres
         POSTGRES_PORT: 5432
         # Use 127.0.0.1 instead of localhost (#351)
         POSTGRES_HOST: 127.0.0.1
-    - name: Upload to coveralls.io
-      if: github.event_name == 'push' && github.repository == 'HazyResearch/fonduer'
-      run: |
-          coverage combine
-          coveralls
-      env:
-        COVERALLS_REPO_TOKEN: ${{ secrets.coveralls_repo_token }}
+    - name: Upload to codecov.io
+      uses: codecov/codecov-action@v1
+      with:
+        file: ./coverage.xml
+        flags: unittests
+        name: codecov-umbrella
+        fail_ci_if_error: true
   macos-test:
     runs-on: macos-latest
     strategy:
@@ -146,7 +146,7 @@ jobs:
     - name: Install Fonduer
       run: |
         make dev_extra
-        pip install coveralls
+        pip install -q pytest-cov
     - name: Run preliminary checks
       run: |
         make check
@@ -156,7 +156,7 @@ jobs:
     - name: Test with pytest
       run: |
         python -m spacy download en
-        coverage run -m pytest tests
+        pytest --cov=./ --cov-report=xml
       env:
         PGPASSWORD: postgres
         PGUSER: postgres
@@ -166,13 +166,13 @@ jobs:
         # https://stackoverflow.com/questions/53014306/error-15-initializing-
         # libiomp5-dylib-but-found-libiomp5-dylib-already-initial
         KMP_DUPLICATE_LIB_OK: true
-    - name: Upload to coveralls.io
-      if: github.event_name == 'push' && github.repository == 'HazyResearch/fonduer'
-      run: |
-          coverage combine
-          coveralls
-      env:
-        COVERALLS_REPO_TOKEN: ${{ secrets.coveralls_repo_token }}
+    - name: Upload to codecov.io
+      uses: codecov/codecov-action@v1
+      with:
+        file: ./coverage.xml
+        flags: unittests
+        name: codecov-umbrella
+        fail_ci_if_error: true
   build-and-publish:
     name: Build and Publish Fonduer to PyPI
     runs-on: ubuntu-latest

README.rst

@@ -1,7 +1,7 @@
 |Fonduer|
 =========
 
-|CI-CD| |CodeClimate| |Coveralls| |ReadTheDocs| |PyPI| |PyPIVersion| |GitHubStars| |License| |CodeStyle|
+|CI-CD| |CodeClimate| |Codecov| |ReadTheDocs| |PyPI| |PyPIVersion| |GitHubStars| |License| |CodeStyle|
 
 Fonduer is a Python package and framework for building knowledge base
 construction (KBC) applications from **richly formatted data**.
@@ -50,8 +50,8 @@ Fonduer leverages the work of Emmental_ and Snorkel_.
    :target: https://github.com/HazyResearch/fonduer
 .. |CI-CD| image:: https://img.shields.io/github/workflow/status/HazyResearch/fonduer/ci.svg
    :target: https://github.com/HazyResearch/fonduer/actions
-.. |Coveralls| image:: https://img.shields.io/coveralls/github/HazyResearch/fonduer.svg
-   :target: https://coveralls.io/github/HazyResearch/fonduer
+.. |Codecov| image:: https://img.shields.io/codecov/c/github/HazyResearch/fonduer
+   :target: https://codecov.io/gh/HazyResearch/fonduer
 .. |ReadTheDocs| image:: https://img.shields.io/readthedocs/fonduer.svg
    :target: https://fonduer.readthedocs.io/
 .. |PyPI| image:: https://img.shields.io/pypi/v/fonduer.svg