Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in author parsing in TheNamibian #619

Merged
merged 2 commits into from
Oct 3, 2024

Conversation

addie9800
Copy link
Collaborator

The author parsing crashed, if it encounters an article with more than author. This is caused that the element element with path Person can be of type list. in which case the get() function that is called within get_value_by_key_path() causes an Error, since it is not implemented for a list.

@MaxDall
Copy link
Collaborator

MaxDall commented Sep 30, 2024

@addie9800 Thanks for catching this! Could you, by any chance, provide the URL leading to the parser crash? I would love to take a look at this and see if it makes sense to rewrite some base functionalities.

@addie9800
Copy link
Collaborator Author

addie9800 commented Sep 30, 2024

@addie9800 Thanks for catching this! Could you, by any chance, provide the URL leading to the parser crash? I would love to take a look at this and see if it makes sense to rewrite some base functionalities.

Unfortunately not, since it seems to have been a mistake. Atleast I can't find any article anymore that would cause that crash. But as far as I remember I think the problematic JSON looked something like this:

{"@type":"Person","@id":"https://www.namibian.com.na/#/schema/person/71d23d353c0a85623bc0847105ce9089","name":"Another Author","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https://www.namibian.com.na/#/schema/person/image/4979e099ca67b0b40e0680c425b7afaa","url":"https://secure.gravatar.com/avatar/7e3425284010a188970bae2a44e8dd26?s=96&d=mm&r=g","contentUrl":"https://secure.gravatar.com/avatar/7e3425284010a188970bae2a44e8dd26?s=96&d=mm&r=g","caption":"Envaalde Matheus"},"url":"https://www.namibian.com.na/author/envaalde-matheus/"},{"@type":"Person","@id":"https://www.namibian.com.na/#/schema/person/71d23d353c0a85623bc0847105ce9089","name":"Envaalde Matheus","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https://www.namibian.com.na/#/schema/person/image/4979e099ca67b0b40e0680c425b7afaa","url":"https://secure.gravatar.com/avatar/7e3425284010a188970bae2a44e8dd26?s=96&d=mm&r=g","contentUrl":"https://secure.gravatar.com/avatar/7e3425284010a188970bae2a44e8dd26?s=96&d=mm&r=g","caption":"Envaalde Matheus"},"url":"https://www.namibian.com.na/author/envaalde-matheus/"}]}

@addie9800
Copy link
Collaborator Author

@MaxDall conincidentally, I just found another article with the same issue: https://www.namibian.com.na/churches-want-corruption-free-leader/

@MaxDall
Copy link
Collaborator

MaxDall commented Oct 3, 2024

@addie9800 I fixed this using xpath_search. Seeing this I decided to deprecate get_value_by_key_path. I will open another PR to do so. Thanks for catching this 👍

@addie9800 addie9800 merged commit 9dd965d into master Oct 3, 2024
5 checks passed
@addie9800 addie9800 deleted the fix-bug-in-get-value-by-path branch October 3, 2024 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants