Add test coverage collecting #128

Jamim · 2019-09-07T02:37:58Z

Hi everyone,

I believe that we should be able to see the test coverage.

These changes:

make tests use pytest
add displaying of the test coverage
add sending of collected coverage data to https://codecov.io/
~~fix skipping tests for pypy3.5~~ (fixed in Fix skipping tests for PyPy #133)

Best regards!

codecov-io · 2019-09-07T02:45:49Z

Codecov Report

❗ No coverage uploaded for pull request base (master@6d1cd1f). Click here to learn what that means.
The diff coverage is 52.17%.

@@           Coverage Diff            @@
##             master    #128   +/-   ##
========================================
  Coverage          ?   85.5%           
========================================
  Files             ?      33           
  Lines             ?    1608           
  Branches          ?     181           
========================================
  Hits              ?    1375           
  Misses            ?     186           
  Partials          ?      47

Impacted Files	Coverage Δ
...try-api/src/opentelemetry/context/async_context.py	`52.17% <52.17%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d1cd1f...89c4f01. Read the comment docs.

Jamim · 2019-09-07T02:58:34Z

There are some links which you may use to see how it will look like:
https://codecov.io/gh/open-telemetry/opentelemetry-python
https://codecov.io/gh/open-telemetry/opentelemetry-python/tree/de8f21ecf59322ddbd71c145c74326824a16bd30

reyang

LGTM.

toumorokoshi

Just want to call out the pytest change. If we have a consensus there I approve.

toumorokoshi · 2019-09-09T03:50:46Z

pytest.ini

@@ -0,0 +1,8 @@
+[pytest]


I know @c24t had some reservations about using pytest. We were supposed to discuss this after the alpha release.

FWIW I really like pytest and wholeheartedly support using pytest and pytest functionality.

I know @c24t had some reservations about using pytest.

Hello @c24t, can you please clarify what kind of reservations you had?
Thank you!

We were supposed to discuss this after the alpha release.

Hi @toumorokoshi,
I think nothing prevents us from discussing it a bit earlier.

Also, I found a related issue: #28.

My biggest complaints about pytest have to do with fixtures. Fixtures with side effects, autouse fixtures, and the monkeypatch fixture in particular have the potential to turn any test suite into a horrifying mess. Using fixtures as arguments in tests makes working with tests difficult outside of pytest, and adds a lot of magic. I don't mean to say it's not possible to write nice tests with pytest, just that the library encourages some pretty bad (IMHO) behavior.

When we considered adding it before it wasn't clear that there were any features of pytest that we couldn't get just as easily from the built in unittest library. It may be that the extra complexity is justified if pytest makes writing and running tests easier. I still don't see a strong reason for using pytest in this PR, but I may be missing something here. What makes pytest --cov better than coverage run?

Hello @toumorokoshi and @c24t,

On the SIG meeting 2019-09-12 we discussed migration to pytest and, if I recall correctly, we found that most developers are fine with it or have no opinion. Unfortunately, we didn't write out a corresponding decision in the OT Python SIG Notes.
So my question is, can we resolve this thread now and move forward with pytest or is it necessary to have an additional discussion?

Thank you!

I think I've pinged the appropriate opinionated people regarding the use of pytest, so I'm going to remove my "changes required". I'll let others who are against pytest argue their point.

@toumorokoshi To actually dismiss your review you have to expand the "Changes requested" box at the bottom of the PR, and then click "..." -> "Dismiss" next to your review there. Re-requesting your review won't actually dismiss it (I ran into that too once, it's not the most intuitive UI).

toumorokoshi · 2019-09-09T03:51:59Z

tox.ini

@@ -38,7 +39,10 @@ commands =
 mypy: mypy --namespace-packages opentelemetry-api/src/opentelemetry/
 ; For test code, we don't want to enforce the full mypy strictness
 mypy: mypy --namespace-packages --config-file=mypy-relaxed.ini opentelemetry-api/tests/
- test: python -m unittest discover
+ test: pytest --cov {envsitepackagesdir}/opentelemetry


there's a few unit tests for opentelemetry-api as well. I feel like we should report coverage on those.

Hello @toumorokoshi,

Coverage on API is being reported for every *-test-* due to settings in pytest.ini.
Three lines those follows this one are for additional reporting.

You can find the full build log for Python 3.7 here:
https://travis-ci.org/open-telemetry/opentelemetry-python/jobs/581936178

Also, please let me know if I'm missing something or there is any misunderstanding on my side.

Thanks!

So the combined coverage report (e.g. considering that the ext-wsgi tests will also increase coverage of the API code) is available under https://codecov.io/github/open-telemetry/opentelemetry-python/commit/de8f21ecf59322ddbd71c145c74326824a16bd30? The API package's trace module seems to be missing in that report.

So the combined coverage report (e.g. considering that the ext-wsgi tests will also increase coverage of the API code) is available under codecov.io/github/open-telemetry/opentelemetry-python/commit/de8f21ecf59322ddbd71c145c74326824a16bd30?

Yes, it is.

The API package's trace module seems to be missing in that report.

I've tried to investigate and fix this, but it seems like this is a bug on the Codecov side.

The reason you need {envsitepackagesdir} here is because we're installing to site-packages after #122 right? This makes for some pretty ugly paths in the coverage report vs. something like:

(cd opentelemetry-sdk/tests; pytest --cov ../src)

I've described this bug here.

@Jamim can you post the text for people like me that can't log into that slack?

The issue is already resolved in the current version of changes, but here that text is:

Hi there,
I found that some files are not visible on Codecov.io for some reason.
I tried to investigate this but I didn't find a reason why some files are shown and other files are not.

PR: #128
Build: https://travis-ci.org/open-telemetry/opentelemetry-python/builds/581936174
Codecov: https://codecov.io/gh/open-telemetry/opentelemetry-python/tree/de8f21ecf59322ddbd71c145c74326824a16bd30

On https://codecov.io/gh/open-telemetry/opentelemetry-python/tree/de8f21ecf59322ddbd71c145c74326824a16bd30/opentelemetry-api/src/opentelemetry page I'm expecting to see

context/ distributedcontext/ logs/ metrics/ resources/ trace/ loader.py types.py version.py

but I can see only

context/ distributedcontext/ loader.py types.py

And corresponding reports contain stats for missing files, e.g.
https://codecov.io/codecov/v4/raw/2019-09-07/E6CBE819EACB252EDCB65A9B1EE32DAB/de8f21ecf59322ddbd71c145c74326824a16bd30/675278e2-de86-49ca-b141-f260fb05b38c.txt contains stats for API's metrics/__init__.py.

Oberon00

Nice work so far, I think I spotted some dead code through the report too 😃 👍

tox.ini

Oberon00 · 2019-09-09T11:18:35Z

tox.ini

@@ -15,6 +15,7 @@ python =
 [testenv]
 deps =
 mypy: mypy~=0.711
+ test: pytest-cov


Please make the coverage a different environment: I want to run unit tests without coverage (assuming that coverage significantly slows down the unit tests, otherwise I don't really care). Maybe https://tox.readthedocs.io/en/latest/config.html#complex-factor-conditions can help, or maybe we can call the environment "TESTENVNAME-cov" and set the --cov option in a setenv to minimize code duplication in tox.ini.

The setup is quite complex already.
I think we shouldn't increase the complexity until we really see some performance issues.

Hello @Oberon00,
Eventually, I've switched to the approach that you suggested in order to use the editable mode.
Could you please re-review this PR?
Thank you!

Why is editable mode required? Is that to more quickly iterate on unit tests?

I think it would be great to also benchmark the performance of the unit tests with coverage. In my experience it's not a huge overhead.

Why is editable mode required?

Answered in #128 (comment).

Is that to more quickly iterate on unit tests?

I didn't check, but it's not a reason anyway.

Oberon00 · 2019-09-09T11:22:24Z

tox.ini

@@ -38,7 +39,10 @@ commands =
 mypy: mypy --namespace-packages opentelemetry-api/src/opentelemetry/
 ; For test code, we don't want to enforce the full mypy strictness
 mypy: mypy --namespace-packages --config-file=mypy-relaxed.ini opentelemetry-api/tests/
- test: python -m unittest discover
+ test: pytest --cov {envsitepackagesdir}/opentelemetry


So the combined coverage report (e.g. considering that the ext-wsgi tests will also increase coverage of the API code) is available under https://codecov.io/github/open-telemetry/opentelemetry-python/commit/de8f21ecf59322ddbd71c145c74326824a16bd30? The API package's trace module seems to be missing in that report.

c24t

The codecov.io integration looks great. Did you have to configure anything outside this repo to get it working? If there are project-specific settings there, who has access to them?

c24t · 2019-09-10T23:53:01Z

tox.ini

@@ -38,7 +39,10 @@ commands =
 mypy: mypy --namespace-packages opentelemetry-api/src/opentelemetry/
 ; For test code, we don't want to enforce the full mypy strictness
 mypy: mypy --namespace-packages --config-file=mypy-relaxed.ini opentelemetry-api/tests/
- test: python -m unittest discover
+ test: pytest --cov {envsitepackagesdir}/opentelemetry


The reason you need {envsitepackagesdir} here is because we're installing to site-packages after #122 right? This makes for some pretty ugly paths in the coverage report vs. something like:

(cd opentelemetry-sdk/tests; pytest --cov ../src)

c24t · 2019-09-11T00:06:25Z

pytest.ini

@@ -0,0 +1,8 @@
+[pytest]
+addopts =
+ --cov-append


Why put these here instead of tox.ini? -rsv seems sane, but I'd be pretty surprised if I appended to the coverage report instead of regenerating it by running pytest --cov, e.g. after switching branches.

Fair enough. There is only -rs -v in addopts now.

c24t · 2019-09-11T02:59:03Z

tox.ini

@@ -38,7 +39,10 @@ commands =
 mypy: mypy --namespace-packages opentelemetry-api/src/opentelemetry/
 ; For test code, we don't want to enforce the full mypy strictness
 mypy: mypy --namespace-packages --config-file=mypy-relaxed.ini opentelemetry-api/tests/
- test: python -m unittest discover
+ test: pytest --cov {envsitepackagesdir}/opentelemetry
+ test-sdk: coverage report --include *opentelemetry/sdk/*


Why filter the results after running coverage instead of passing the path to pytest? E.g.

pytest --cov-report xml --cov ../src/

It was necessary in purpose to collect coverage not only for a current one package, but for api/sdk as well, but now those extra reports are gone.

c24t · 2019-09-11T03:24:40Z

pytest.ini

@@ -0,0 +1,8 @@
+[pytest]


My biggest complaints about pytest have to do with fixtures. Fixtures with side effects, autouse fixtures, and the monkeypatch fixture in particular have the potential to turn any test suite into a horrifying mess. Using fixtures as arguments in tests makes working with tests difficult outside of pytest, and adds a lot of magic. I don't mean to say it's not possible to write nice tests with pytest, just that the library encourages some pretty bad (IMHO) behavior.

When we considered adding it before it wasn't clear that there were any features of pytest that we couldn't get just as easily from the built in unittest library. It may be that the extra complexity is justified if pytest makes writing and running tests easier. I still don't see a strong reason for using pytest in this PR, but I may be missing something here. What makes pytest --cov better than coverage run?

c24t · 2019-09-11T03:26:10Z

.codecov.yml

@@ -0,0 +1,5 @@
+fixes:
+ - "^.*/site-packages/opentelemetry/sdk/::opentelemetry-sdk/src/opentelemetry/sdk/"


We may want to stop testing against site-packages to avoid this ugly hack and others like it.

Jamim · 2019-10-09T02:01:53Z

Hello everyone!

First of all, I apologize for the huge delay.

I've implemented a new approach which is based on thoughts those we discussed at the SIG meeting.
Now coverage is collected separately from the main test flow using the editable mode to install OT packages.
Could you please re-review this PR?

Thank you!

Oberon00 · 2019-10-09T08:52:17Z

@Jamim, @hectorhdzg: So now we have both this PR and #208 active for coverage?

Jamim · 2019-10-09T11:23:47Z

So now we have both this PR and #208 active for coverage?

Yes, and they are quite different in terms of the implementation.
Looks like the team should choose one of them.

I personally vote for the current one 😅

hectorhdzg · 2019-10-09T17:18:43Z

@Oberon00 and @Jamim I closed mine in favor of this one to avoid confusion

Jamim · 2019-10-09T17:51:15Z

Hello @hectorhdzg,

I understand that you had good intentions, so I have to say Thank you for pinging me and submitting that PR!

opentelemetry-api/src/opentelemetry/context/async_context.py

toumorokoshi · 2019-10-11T02:53:04Z

pytest.ini

@@ -0,0 +1,2 @@
+[pytest]
+addopts = -rs -v


duplicate addopts?

Do you mean that there is the same line inpytest-cov.ini?
That file is a separate config file used in the coverage testenv only.
It's required in order to add python_files = *.py setting which, I assume, should not be used in the test testenv.

toumorokoshi · 2019-10-11T02:54:53Z

tox.ini

@@ -4,6 +4,11 @@ skip_missing_interpreters = True
 envlist =
 py3{4,5,6,7,8}-test-{api,sdk,example-app,ext-wsgi,ext-http-requests,ext-jaeger}
 pypy3-test-{api,sdk,example-app,ext-wsgi,ext-http-requests,ext-jaeger}
+ py3{4,5,6,7,8}-coverage


will this double execute all tests? Since under the hood it's calling pytest --cov, which IIUC will run all the test again.

If it is, I think we should eliminate duplicate tests before merging.

will this double execute all tests?

Yes, it will.

If it is, I think we should eliminate duplicate tests before merging.

I don't think so currently.

There are reasons why:

As far as I know, we don't want to move back to using editable mode for the main test flow
Please see Fix setup for ext packages. #122 and #208/files#r332781829.

Non-editable mode forces us to provide fixes for Codecov.

There was (and probably is) a missing files issue on the Codecov side when using fixes, so it looks reasonable to use editable mode to avoid the issue.
You can find some details in this thread.

Pretty ugly paths in the coverage report for non-editable mode.

It was suggested to make the coverage a different environment.

cc @Oberon00, @c24t

I worry that this will double the build times in CI and locally for those running tox.

I don't want to block this PR if this is the direction people want to go, but I think we should go back and find a way to not double-execute tests. Can you file a ticket once this is merged so we remember we should go back and hopefully resolve this?

Sure, I'll submit a corresponding issue once this PR is merged.

toumorokoshi · 2019-10-11T02:58:12Z

tox.ini

@@ -15,6 +15,7 @@ python =
 [testenv]
 deps =
 mypy: mypy~=0.711
+ test: pytest-cov


Why is editable mode required? Is that to more quickly iterate on unit tests?

I think it would be great to also benchmark the performance of the unit tests with coverage. In my experience it's not a huge overhead.

toumorokoshi

I think most of my feedback is not a blocker, but I feel strongly about not double-executing tests in the CI (and hopefully not by default when I'm running tox locally).

Oberon00 · 2019-10-11T08:57:09Z

opentelemetry-api/src/opentelemetry/context/async_context.py

@@ -12,32 +12,34 @@
 # See the License for the specific language governing permissions and


I think this file was changed by mistake? Could you please remove these changes from the PR?

Not, it wasn't.
This file was changed in purpose to make it "importable" for Python <3.7.
Please see a corresponding build. That build failed with:

==================================== ERRORS ==================================== _ ERROR collecting opentelemetry-api/src/opentelemetry/context/async_context.py _ ImportError while importing test module '/home/travis/build/Jamim/opentelemetry-python/opentelemetry-api/src/opentelemetry/context/async_context.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: opentelemetry-api/src/opentelemetry/context/async_context.py:16: in <module> from contextvars import ContextVar E ModuleNotFoundError: No module named 'contextvars'

But doesn't this break the logic of the importing __init__.py? You are not supposed to import that file directly anyway.

But doesn't this break the logic of the importing __init__.py?

No, it doesn't.

Python 3.7

$ python Python 3.7.3 (default, Apr 3 2019, 19:16:38) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from opentelemetry.context import Context >>> Context AsyncRuntimeContext({})

Python 3.6

$ python Python 3.6.8 (default, Oct 7 2019, 12:59:55) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from opentelemetry.context import Context >>> Context ThreadLocalRuntimeContext({})

You are not supposed to import that file directly anyway.

Having non-importable files in the project is not good in general.

toumorokoshi · 2019-10-24T15:15:04Z

coverage.sh

@@ -0,0 +1,29 @@
+#!/bin/bash


can this be moved to a scripts/ directory? I think we should avoid littering the root directory with a bunch of utility scripts.

I'm also putting stuff into a scripts directory: https://github.com/open-telemetry/opentelemetry-python/pull/228/files

It's done. Thank you!

toumorokoshi

I have a couple changes I'd love to see (specifically putting scripts into a subdirectory), but there's a lot of good improvements here and we can fix issues incrementally rather than block this pr. Thanks!

As discussed in the SIG meeting: This PR is a good improvement even now, imperfect details can be fixed later.

reyang · 2019-10-30T04:58:52Z

@c24t should we merge it now or wait after alpha 0.2 release?

I believe that we should be able to see the test coverage. These changes: - make tests use pytest - add displaying of the test coverage - add sending of collected coverage data to https://codecov.io

c24t · 2019-10-31T18:58:30Z

ext/opentelemetry-ext-jaeger/src/opentelemetry/ext/jaeger/gen/agent/Agent.py

@@ -124,7 +124,7 @@ class emitZipkinBatch_args(object):

 thrift_spec = (
 None, # 0
- (1, TType.LIST, 'spans', (TType.STRUCT, (zipkincore.ttypes.Span, zipkincore.ttypes.Span.thrift_spec), False), None, ), # 1
+ (1, TType.LIST, 'spans', (TType.STRUCT, (zipkincore_ttypes.Span, zipkincore_ttypes.Span.thrift_spec), False), None, ), # 1


Why these changes to the generated files?

c24t · 2019-10-31T19:18:34Z

@Jamim FYI it makes it easier to review if you push new commits instead of amending and force-pushing a single commit. Two reasons for this: comments can get orphaned if they refer to commits that aren't on the feature branch anymore (although that doesn't seem to have happened here), and without the commit history it's hard to tell which changes were made in response to which comments.

For example, I don't know when the generated jaeger thrift files were changed, and don't know whether other reviewers chose not to comment on these changes or if they didn't exist at review time.

c24t · 2019-11-05T21:31:32Z

AFAICT we needed the change to the generated jaeger files because of test name collisions, which happened because we were trying to import files from all packages on each coverage run. But this seems like the wrong behavior to me: should a line in the API package be reported as "covered" if it's only ever run downstream of e.g. a ext-http-requests test?

I changed the test coverage script in cdc64f5 to collect coverage stats separately for each package, but still combine them into a single report. This fixes some false-positive coverage, and means we don't have to change the generated files to work around the test import problem.

@Jamim please let me know if you have any comments. @toumorokoshi feel free to merge if this LGTY.

c24t

Stamping my own changes.

Jamim · 2019-11-05T22:29:59Z

Hello @c24t,

Thank you for working on this PR!

I changed the test coverage script in cdc64f5 to collect coverage stats separately for each package, but still combine them into a single report. This fixes some false-positive coverage, and means we don't have to change the generated files to work around the test import problem.

Unfortunately, now we have an issue with some files missing from the report again.
But we can try to fix it later in a separate PR.

Also, we don't need pytest-cov.ini anymore since it's equal to pytest.ini now.
Should I remove it?

c24t · 2019-11-06T01:27:52Z

Unfortunately, now we have an issue with some files missing from the report again. But we can try to fix it later in a separate PR.

The missing files are the ones that have 0% coverage, right? I agree that's a big problem, but the better solution may be to restructure our tests so we don't have this collection problem. If you're ok with solving this in another PR that sounds good to me.

Also, we don't need pytest-cov.ini anymore since it's equal to pytest.ini now. Should I remove it?

Sure, good call.

Jamim · 2019-11-06T14:06:57Z

The missing files are the ones that have 0% coverage, right?

Right.

Jamim · 2019-11-06T14:08:25Z

Also, we don't need pytest-cov.ini anymore since it's equal to pytest.ini now.
Should I remove it?

Sure, good call.

It's done.

Change CONTRIBUTING.md Resolves open-telemetry#117 Signed-off-by: Olivier Albertini <olivier.albertini@montreal.ca>

Jamim requested review from a-feld, c24t, carlosalberto, lzchen, Oberon00, reyang and toumorokoshi as code owners September 7, 2019 02:37

reyang approved these changes Sep 7, 2019

View reviewed changes

toumorokoshi suggested changes Sep 9, 2019

View reviewed changes

Oberon00 reviewed Sep 9, 2019

View reviewed changes

Jamim changed the title ~~Add test coverage collecting~~ [WIP] Add test coverage collecting Sep 11, 2019

c24t reviewed Sep 11, 2019

View reviewed changes

hectorhdzg mentioned this pull request Oct 8, 2019

Add code coverage in tests #208

Closed

Jamim force-pushed the feature/coverage branch 2 times, most recently from 7ffb269 to b7dea94 Compare October 9, 2019 01:52

Jamim changed the title ~~[WIP] Add test coverage collecting~~ Add test coverage collecting Oct 9, 2019

Jamim force-pushed the feature/coverage branch 3 times, most recently from 4999fa4 to 294ea63 Compare October 9, 2019 04:26

Jamim requested review from toumorokoshi and Oberon00 October 9, 2019 16:46

mauriciovasquezbernal mentioned this pull request Oct 10, 2019

Add Travis CI #6

Closed

3 tasks

toumorokoshi reviewed Oct 11, 2019

View reviewed changes

toumorokoshi suggested changes Oct 11, 2019

View reviewed changes

Oberon00 previously requested changes Oct 11, 2019

View reviewed changes

c24t mentioned this pull request Oct 21, 2019

Add sampler API, use in SDK tracer #225

Merged

Jamim force-pushed the feature/coverage branch from 294ea63 to ccb3a32 Compare October 24, 2019 14:08

toumorokoshi reviewed Oct 24, 2019

View reviewed changes

toumorokoshi approved these changes Oct 24, 2019

View reviewed changes

Jamim force-pushed the feature/coverage branch from ccb3a32 to 086237b Compare October 24, 2019 15:24

Add test coverage collecting

c9ddd65

I believe that we should be able to see the test coverage. These changes: - make tests use pytest - add displaying of the test coverage - add sending of collected coverage data to https://codecov.io

Jamim force-pushed the feature/coverage branch from 086237b to c9ddd65 Compare October 30, 2019 09:52

c24t reviewed Oct 31, 2019

View reviewed changes

toumorokoshi mentioned this pull request Nov 4, 2019

Metrics Console Exporter + Add last_updated_timestamp to metrics #192

Merged

c24t added 3 commits November 5, 2019 12:06

Revert changes to generated jaeger modules

c012914

Only report coverage for package being tested

cdc64f5

Ignore generated files in coverage

3da192d

c24t approved these changes Nov 5, 2019

View reviewed changes

Remove pytest-cov.ini

89c4f01

c24t merged commit 72a331c into open-telemetry:master Nov 7, 2019

mauriciovasquezbernal mentioned this pull request Nov 7, 2019

test cases are executed twice #273

Closed

Jamim deleted the feature/coverage branch January 16, 2020 22:37

srikanthccv pushed a commit to srikanthccv/opentelemetry-python that referenced this pull request Nov 1, 2020

chore: add commitlint and husky (open-telemetry#128)

44214ac

Change CONTRIBUTING.md Resolves open-telemetry#117 Signed-off-by: Olivier Albertini <olivier.albertini@montreal.ca>

		@@ -0,0 +1,5 @@
		fixes:
		- "^.*/site-packages/opentelemetry/sdk/::opentelemetry-sdk/src/opentelemetry/sdk/"

		@@ -12,32 +12,34 @@
		# See the License for the specific language governing permissions and

Add test coverage collecting #128

Add test coverage collecting #128

Conversation

Jamim commented Sep 7, 2019 • edited Loading

codecov-io commented Sep 7, 2019 • edited Loading

Codecov Report

Jamim commented Sep 7, 2019

reyang left a comment

Choose a reason for hiding this comment

toumorokoshi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim Sep 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toumorokoshi Oct 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim Sep 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim Oct 9, 2019 • edited Loading

Choose a reason for hiding this comment

Oberon00 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c24t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim commented Oct 9, 2019

Oberon00 commented Oct 9, 2019

Jamim commented Oct 9, 2019

hectorhdzg commented Oct 9, 2019

Jamim commented Oct 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim Oct 11, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toumorokoshi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jamim Oct 24, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toumorokoshi left a comment

Choose a reason for hiding this comment

reyang commented Oct 30, 2019

Choose a reason for hiding this comment

c24t commented Oct 31, 2019

c24t commented Nov 5, 2019 • edited Loading

c24t left a comment

Choose a reason for hiding this comment

Jamim commented Nov 5, 2019

c24t commented Nov 6, 2019

Jamim commented Nov 6, 2019

Jamim commented Sep 7, 2019 •

edited

Loading

codecov-io commented Sep 7, 2019 •

edited

Loading

Jamim Sep 10, 2019 •

edited

Loading

toumorokoshi Oct 10, 2019 •

edited

Loading

Jamim Sep 10, 2019 •

edited

Loading

Jamim Oct 9, 2019 •

edited

Loading

Jamim Oct 11, 2019 •

edited

Loading

Jamim Oct 24, 2019 •

edited

Loading

c24t commented Nov 5, 2019 •

edited

Loading