Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: expose allow overlapping property #4697

Merged
merged 13 commits into from
Apr 8, 2024
Merged

feat: expose allow overlapping property #4697

merged 13 commits into from
Apr 8, 2024

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Apr 3, 2024 •
edited
Loading

Description

This PR adds the allow_overlapping field for span questions to configure span questions with overlapping support. The default value is set to False to align the previous version of the span question.

It must be merged after argilla-io/argilla-server#89

span_question = rg.SpanQuestion(name="spans", .., allow_overlapping=True)
...

Closes #4694

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

  • New feature (non-breaking change which adds functionality)
  • Refactor (change restructuring the codebase without changing functionality)
  • Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

(Please describe the tests that you ran to verify your changes. And ideally, reference tests)

  • Test A
  • Test B

Checklist

  • I added relevant documentation
  • I followed the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I filled out the contributor form (see text above)
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints language: python Pull requests or issues that update Python code severity: minor Indicates that the issue isn't urgent or blocking team: backend Indicates that the issue or pull request is owned by the backend team type: improvement Indicates updates or improvements on existing features labels Apr 3, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 3, 2024

The URL of the deployed environment for this PR is https://argilla-quickstart-pr-4697-ki24f765kq-no.a.run.app

burtenshaw
burtenshaw approved these changes Apr 3, 2024
View reviewed changes
Copy link
Contributor

@burtenshaw burtenshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionality looks good.

I left some comments on testing. We could expand testing for all use cases, but I'm not sure how you want the suite to flow.

CHANGELOG.md Outdated Show resolved Hide resolved
src/argilla/client/feedback/schemas/questions.py Outdated Show resolved Hide resolved
tests/integration/client/feedback/dataset/local/test_dataset.py Outdated
@@ -72,15 +72,17 @@ def test_create_dataset_with_span_questions(argilla_user: "ServerUser") -> None:

ds = FeedbackDataset(
fields=[TextField(name="text")],
questions=[SpanQuestion(name="spans", field="text", labels=["label1", "label2"])],
questions=[SpanQuestion(name="spans", field="text", labels=["label1", "label2"], allow_overlapping=True)],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should we separate these tests so that we have with/without overlap?
  • Should we add records to both that have overlaps or not?
  • Should we assert raises of the overlap error code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I've created a new separate test checking possible values for the allow_overlapping flag.

For related records questions, I will include some integration tests. Validations for responses and suggestions are not covered on this SDK, we will cover them in the new SDK

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Apr 3, 2024
@codecov Codecov
Copy link

codecov bot commented Apr 3, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 45.84%. Comparing base (ffeb2cd) to head (c0c527d).
Report is 32 commits behind head on feat/overlapped-span-questions.

❗ Current head c0c527d differs from pull request most recent head 99de15c. Consider uploading reports for the commit 99de15c to get more accurate results

Additional details and impacted files 
@@                        Coverage Diff                         @@
##           feat/overlapped-span-questions    #4697      +/-   ##
==================================================================
- Coverage                           45.85%   45.84%   -0.01%     
==================================================================
  Files                                 193      193              
  Lines                               11810    11812       +2     
==================================================================
  Hits                                 5415     5415              
- Misses                               6395     6397       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Apr 6, 2024
@frascuchon frascuchon merged commit 1f8aa08 into feat/overlapped-span-questions Apr 8, 2024
13 of 14 checks passed
@frascuchon frascuchon deleted the feat/expose-allow-overlapping-property branch April 8, 2024 08:18
@louisguitton louisguitton mentioned this pull request Jun 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@burtenshaw burtenshaw burtenshaw approved these changes

@jfcalvo jfcalvo Awaiting requested review from jfcalvo

@nataliaElv nataliaElv Awaiting requested review from nataliaElv

Assignees
No one assigned
Labels
area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints language: python Pull requests or issues that update Python code lgtm This PR has been approved by a maintainer severity: minor Indicates that the issue isn't urgent or blocking size:L This PR changes 100-499 lines, ignoring generated files. team: backend Indicates that the issue or pull request is owned by the backend team type: improvement Indicates updates or improvements on existing features
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@frascuchon @burtenshaw