filter on loading rather than increase loading limit

[dafnapension]

per @elronbandel comment in recent PR:
Perhaps employing the HuggingFace filter upfront can ease load on the system, better than increase loading_limit only to allow the following pre-processing step that filters out.

[codecov]

Codecov Report

Attention: Patch coverage is 88.88889% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 88.59%. Comparing base (3923a46) to head (3818172).

Files	Patch %	Lines
src/unitxt/loaders.py	75.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #584      +/-   ##
==========================================
+ Coverage   88.55%   88.59%   +0.03%     
==========================================
  Files          87       87              
  Lines        7858     7876      +18     
==========================================
+ Hits         6959     6978      +19     
+ Misses        899      898       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[elronbandel]

I like the idea. Could we do it with expression rather then lambda? What sould be with other loaders which are not HFLoader? Lastly maybe we should consider doing it all through an independent operator?

[dafnapension]

My understanding of the HF api, is that a function returning True or False, when fed with an insrance, is needed there. We need it to be artifact-able, so can be cataloged. Lambda seemed most apropriate.
Another, safer, solution can be laid on the shoulders of recipe: if it sees a filter by condition, or by expression, operator among the steps, the recipe should position any stream refiner by number - following the filter. perhaps always have the refiners -- the last in the pipeline?
The loader cache will still have to suffer, but no more than necessary, and the solution is universal.

I see that currently, the load limits sneak into the loader, and become effective as it actually loads. Is it advisable to change this too?