Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validate metadata as early as possible #235

Closed
benoit74 opened this issue Apr 18, 2024 · 2 comments · Fixed by #266
Closed

Validate metadata as early as possible #235

benoit74 opened this issue Apr 18, 2024 · 2 comments · Fixed by #266
Assignees
@benoit74
Labels
enhancement New feature or request
Milestone
2.0.0

Comments

@benoit74
Copy link
Collaborator

Scraper must validate metadata and fail as early as possible (especially when called without WARCs to validate inputs).
@benoit74 benoit74 added the enhancement New feature or request label Apr 18, 2024
@benoit74
Copy link
Collaborator Author

Ideally we should call https://github.com/openzim/python-scraperlib/blob/7d498319baadba715316c15cf9857ff2f6974a00/src/zimscraperlib/zim/creator.py#L172

But since we probably do not have all metadata early, we will have to resort to calling only some individual methods early.

@benoit74 benoit74 mentioned this issue Apr 19, 2024
Closed
@benoit74 benoit74 added this to the 2.0.0 milestone May 2, 2024
@benoit74 benoit74 self-assigned this May 21, 2024
@benoit74 benoit74 mentioned this issue May 21, 2024
Merged
@benoit74
Copy link
Collaborator Author

Fixed by #266

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benoit74 benoit74

Labels
enhancement New feature or request
Projects
None yet
Milestone
2.0.0
Development

Successfully merging a pull request may close this issue.

Validate ZIM metadata early openzim/warc2zim
1 participant
@benoit74