Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SampleRowProvider -- Enable Crop Mode #125

Closed
Tracked by #124
nicolay-r opened this issue Nov 22, 2023 · 0 comments
Closed
Tracked by #124

SampleRowProvider -- Enable Crop Mode #125

nicolay-r opened this issue Nov 22, 2023 · 0 comments
Assignees
@nicolay-r
Labels
critical enhancement New feature or request

Comments

@nicolay-r
Copy link
Owner

nicolay-r commented Nov 22, 2023 •
edited
Loading

Reason: In some cases we may end up dealing with the very long sentences and texts without punkt signs. Once saved into samples, these sentences may quickly occupy the whole disk space 💾

To enable this option, there is a need to switch onto cropping modes here:

ARElight/arelight/samplers/bert.py

Lines 50 to 51 in 75c3395

return BaseSampleRowProvider(label_provider=MultipleLabelProvider(label_scaler),
text_provider=text_provider)

@nicolay-r nicolay-r added enhancement New feature or request critical labels Nov 22, 2023
@nicolay-r nicolay-r self-assigned this Nov 22, 2023
This was referenced Nov 22, 2023
Closed
Closed
nicolay-r added a commit that referenced this issue Nov 23, 2023
@nicolay-r
#125 switched to cropping mode.
03d61f6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nicolay-r nicolay-r

Labels
critical enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nicolay-r