Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(#735): add warning when agent but no prediction/annotation is provided #987

Merged
merged 5 commits into from
Jan 19, 2022

Conversation

dcfidalgo
Copy link
Contributor

Closes #735
This PR adds a warning when a *_agent is provided but no corresponding annotation/prediction.
Also, some refactoring is done, introducing a base class for the records in which common checks are performed.

@dcfidalgo dcfidalgo self-assigned this Jan 17, 2022
@codecov
Copy link

codecov bot commented Jan 17, 2022

Codecov Report

Merging #987 (1910688) into master (d3cbaea) will decrease coverage by 0.04%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #987      +/-   ##
==========================================
- Coverage   95.51%   95.46%   -0.05%     
==========================================
  Files         109      109              
  Lines        4212     4216       +4     
==========================================
+ Hits         4023     4025       +2     
- Misses        189      191       +2     
Flag Coverage Δ
pytest 95.46% <100.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rubrix/__init__.py 95.00% <100.00%> (ø)
src/rubrix/client/models.py 98.59% <100.00%> (-0.25%) ⬇️
...rver/tasks/token_classification/service/service.py 100.00% <100.00%> (ø)
src/rubrix/server/commons/es_wrapper.py 92.79% <0.00%> (-1.81%) ⬇️
...ubrix/labeling/text_classification/label_models.py 100.00% <0.00%> (ø)
src/rubrix/server/tasks/commons/dao/dao.py 97.27% <0.00%> (+0.10%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3cbaea...1910688. Read the comment docs.

@dcfidalgo dcfidalgo requested a review from frascuchon January 18, 2022 15:40
@dcfidalgo dcfidalgo requested a review from frascuchon January 18, 2022 22:17
Copy link
Member

@frascuchon frascuchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@dcfidalgo dcfidalgo merged commit 974ecb2 into master Jan 19, 2022
@dcfidalgo dcfidalgo deleted the feat/add_warning_only_agent branch January 19, 2022 09:47
dvsrepo added a commit that referenced this pull request Jan 25, 2022
* 'master' of https://github.com/recognai/rubrix: (28 commits)
  chore(#994): simplify server start (#1014)
  refactor(#945): elasticsearch index configuration per task (#1018)
  docs(#980): Add majorityvote example to weak supervision guide (#1004)
  fix(#1015): manage emojis in Token Classification records (#1016)
  fix(#1010): fix WeakLabels when not providing rules (#1011)
  fix(Breadcrumbs): Copy text function (#1003)
  fix(statics): handle 404 errors for static files (#1006)
  refactor: Rename 'exclude_missing_annotation' argument for WeakLabels.annotation (#1005)
  docs: update video for spacy ner tutorial (#1000)
  fix(#977) Remove redirection when accessing login (#996)
  feat(#951): new uncovered_by_rules records filter (#991)
  feat(#735): add warning when agent but no prediction/annotation is provided (#987)
  fix(#800): limit number of metadata fields (#993)
  docs: improve annotate records section in web app reference (#985)
  fix(#844) Keep header layout when errors occurred in dataset view (#992)
  fix(#974): display the dropdown in the last record of the scroll (#986)
  refactor: compute common aggregations one by one (#990)
  feat(#953): add additional metrics to `LabelModel.score` method (#979)
  refactor(rules): using labeling rules management with vuex store (#878)
  feat(#955): add default for `rules` in WeakLabels (#976)
  ...
frascuchon pushed a commit that referenced this pull request Jan 31, 2022
…ovided (#987)

* feat: add warning when only agent is provided

* refactor: avoid global import

* docs: remove not important members from docs

* test: add test

* refactor: use root_validators instead of init

(cherry picked from commit 974ecb2)
frascuchon pushed a commit that referenced this pull request Jan 31, 2022
…ovided (#987)

* feat: add warning when only agent is provided

* refactor: avoid global import

* docs: remove not important members from docs

* test: add test

* refactor: use root_validators instead of init

(cherry picked from commit 974ecb2)
frascuchon pushed a commit that referenced this pull request Jan 31, 2022
…ovided (#987)

* feat: add warning when only agent is provided

* refactor: avoid global import

* docs: remove not important members from docs

* test: add test

* refactor: use root_validators instead of init

(cherry picked from commit 974ecb2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Log] Provided prediction agent without prediction is not registered
2 participants