records: refactor file handling #297
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zannkukai
approved these changes
Sep 9, 2020
| # If extract fulltext is disabled or file is not a PDF | ||
| if not current_app.config.get( | ||
| 'SONAR_DOCUMENTS_EXTRACT_FULLTEXT_ON_IMPORT' | ||
| ) or file.mimetype != 'application/pdf': |
There was a problem hiding this comment.
could be interesting (in next PR/US) to extract fulltext from other mimetype (rtf, odt, ...)
In our repository we also have some 'application/pdf+a' mimeType.
Maybe interesting to have a general config parameter to list all mimeTypes available for fulltext extraction.
There was a problem hiding this comment.
Yes, for sure it could be interesting. I'll will notify that to Miguel.
This commit aims to refactor the way files are handled when added to a record, to be more generalistic and have specific treatment for documents without affecting others records types. * Moves specific methods `create_thumbnail` and `create_fulltext_file` in `DocumentRecord` * Tests file existence to determine if file have to be processed. * Handles files metadata in a more generalistic way. * Connects to `file_uploaded` and `file_deleted` signals with to different functions, to know if the file is deleted when processing. * Removes `size` property when harvesting from RERODOC, as this property is calculated when the file is created. Co-Authored-by: Sébastien Délèze <sebastien.deleze@rero.ch>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit aims to refactor the way files are handled when added to a record, to be more generalistic and have specific treatment for documents without affecting others records types.
create_thumbnailandcreate_fulltext_fileinDocumentRecordfile_uploadedandfile_deletedsignals with to different functions, to know if the file is deleted when processing.sizeproperty when harvesting from RERODOC, as this property is calculated when the file is created.Co-Authored-by: Sébastien Délèze sebastien.deleze@rero.ch