Skip to content
This repository has been archived by the owner on Sep 8, 2023. It is now read-only.

[Improvement] Add options to write a dummy placeholder for documents that failed processor step #33

Open
hkkenneth opened this issue Jan 11, 2017 · 1 comment
Assignees

Comments

@hkkenneth
Copy link
Collaborator

When this option is on, even when a non-optional processor fails, it will trigger a record to be written to elastic search.

However, note that we possibly cannot catch failure in reader and writer in the same way...

@hkkenneth hkkenneth self-assigned this Jan 11, 2017
@hkkenneth
Copy link
Collaborator Author

Further explanation:

Why cannot do this in reader: if reading an item fails, we know nothing about this item, nothing about document ID, patient ID etc.

Why cannot do this in writer: if writing to ElasticSearch has problem, writing the dummy placeholder to ES will not succeed...

hkkenneth added a commit to KHP-Informatics/cogstack that referenced this issue Apr 21, 2017
hkkenneth added a commit to KHP-Informatics/cogstack that referenced this issue Apr 23, 2017
hkkenneth added a commit to KHP-Informatics/cogstack that referenced this issue May 9, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant