Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow flushing collectors to only emit when complete. #332

Merged

Conversation

blackwinter
Copy link
Member

@blackwinter blackwinter commented Sep 11, 2020

Primarily needed for EqualsFilter but may also be useful as a generic option for flushing collectors:

  • EqualsFilter: Use case described below.
  • Combine, Entity: No actual use case yet.
  • All: Already built-in (as an internal condition in emit()).
  • Choose, Concat, Group, Range, Square, Tuples: Inherently "incomplete" (isComplete() always false).

When transforming Alma publishing data, we need to select fields based on some other field's subfield value:

<combine name="date" value="${value}" sameEntity="true">
  <if>
    <equalsFilter name="" value="" flushWith="POC  .a" reset="true">
      <data name="" source="@portfolio" />
      <data name="" source="POC  .a" />
    </equalsFilter>
  </if>
  <data name="value" source="POC  .b" />
</combine>

But this fires (emit()s) on every flush, even if the target value (@portfolio) wasn't received (it's only populated at a later point in the stream). Having the option to restrict firing to only when the collector isComplete(), fixes the issue:

<combine name="date" value="${value}" sameEntity="true">
  <if>
    <equalsFilter name="" value="" flushWith="POC  .a" flushIncomplete="false" reset="true">
      <data name="" source="@portfolio" />
      <data name="" source="POC  .a" />
    </equalsFilter>
  </if>
  <data name="value" source="POC  .b" />
</combine>

NOTE: This option is only applicable in combination with flushWith. Maybe the name should reflect that? (flushIncomplete with inverted default?) [EDIT: Changed attribute from complete="true" to flushIncomplete="false".]

@blackwinter
Copy link
Member Author

In hindsight, I think I'd prefer flushIncomplete=false (default: true). Let me know if you disagree.

@dr0i
Copy link
Member

dr0i commented Sep 14, 2020

@blackwinter your proposal makes it clearer - so go on!

Without this option, a flushing collector (e.g. `EqualsFilter`) always
emits its value when being flushed, regardless of whether it's already
complete.

Only applies to a subset of all flushing collectors since most of them
are never "complete"; `All`, on the other hand, has this condition
already built-in.
@blackwinter blackwinter force-pushed the flushing-collectors-emit-if-complete branch from cea1634 to 8bf5bcb Compare September 14, 2020 14:14
@blackwinter
Copy link
Member Author

Done.

fsteeg added a commit that referenced this pull request Sep 15, 2020
@dr0i dr0i merged commit 8bf5bcb into metafacture:master Sep 15, 2020
@dr0i dr0i mentioned this pull request Sep 15, 2020
@blackwinter blackwinter deleted the flushing-collectors-emit-if-complete branch September 15, 2020 14:09
@blackwinter
Copy link
Member Author

Thank you!

@dr0i
Copy link
Member

dr0i commented Sep 15, 2020

Thx a lot. I think this is a very good enhancement which would probably saved me some headaches (and stupid workarounds, working with "@ variables" to postpone the execution of rules ).
I added your description to https://github.com/metafacture/metafacture-core/wiki/Metamorph-Cookbook.

blackwinter pushed a commit that referenced this pull request Dec 13, 2024
Add hint to README regarding problems when build on Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants