Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wikidata v0 with author description and infobox #9130

Merged
merged 7 commits into from
May 2, 2024
Merged

Conversation

RayBB
Copy link
Collaborator

@RayBB RayBB commented Apr 21, 2024

Wikidata Project Google Doc

In short, this PR enables us to start fetching info from wikidata and showing it on the author pages.

How it works:

  • For patrons, we only fetch missing items (and bust cache) when they visit the edit page.
    • This has the unfortunate side effect that if they add a wikidata id, it won't be loaded until the next time the edit page is visited. That is okay for this v0 because in v1 we will bulk upload the wikidata and then remove this restriction. We have this restriction now so we don't overload wikidata with requests at go live.
    • The date has also moved from under the name to the infobox for patrons.
  • For librarians, we fetch missing items on the view/edit pages but only bust_cache on the edit page.
    • Cache is 30 days
    • Librarians still see the date under the name and on the infobox.

About this first step

This PR is v0 of the wikidata project:

  • (v0) 1. Way to fetch live data to handling cache_miss case
    • Also gives patrons way to reload the cached record if they have made an edit to wikidata
  • (v0) 2. Render data on authors (view+edit) page
  • (v0) 3. Database where we ‘cache’ wikidata entities
    • Note: Can’t use memcache since it doesn’t persist between restarts + since memcache is in memory might be too large to store there

Technical

This PR supersedes: #8949 and #8236 - much discussion here

Testing

Drini has tested thoroughly (see comments in this PR) but we also have nice unit tests for caching now.

Screenshot

Before After
Screenshot 2024-05-02 at 00-12-27 Cory Doctorow Open Library Screenshot 2024-05-02 at 00-12-08 Cory Doctorow Open Library

Stakeholders

@cdrini

@RayBB RayBB changed the title infobox with wikidata show wikidata description in infobox Apr 21, 2024
@RayBB RayBB changed the title show wikidata description in infobox Wikidata v0 with description in infobox May 1, 2024
@RayBB RayBB marked this pull request as ready for review May 1, 2024 15:20
Copy link
Collaborator

@cdrini cdrini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm! Deploying to testing and will give a whirl there!

Copy link
Collaborator

@cdrini cdrini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested:

⚠ Modifying the short description on wikidata and reloading the author page as a non-librarian showed the new description! This suggests it re-fetched.

One bug to look into otherwise the testing went ok! Although I realized I was kind of assuming it fetched/didn't fetch the wikidata data.

@RayBB RayBB requested a review from cdrini May 1, 2024 21:53
@RayBB
Copy link
Collaborator Author

RayBB commented May 1, 2024

@cdrini I fixed it :')
fd0d267#diff-e97d8adfed291d93588f5da99576884753fdaa96ca162b04bc449948d627c18eR82

I also write a ton of tests along the way so you can feel confident that that part of the code is working.
For these few weeks until we do the bulk import and simplify it...

@RayBB RayBB mentioned this pull request May 1, 2024
@RayBB RayBB changed the title Wikidata v0 with description in infobox Wikidata v0 with author description in infobox May 1, 2024
@RayBB RayBB changed the title Wikidata v0 with author description in infobox Wikidata v0 with author description and infobox May 1, 2024
@cdrini cdrini force-pushed the infobox-with-wikidata branch from c03d1c4 to 4710814 Compare May 2, 2024 16:04
@cdrini cdrini force-pushed the infobox-with-wikidata branch from 478104f to aafdc27 Compare May 2, 2024 16:11
Copy link
Collaborator

@cdrini cdrini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm! Did a lighter round of testing and things appear to roughly work. Great catch on that bug!!! Super subtle.

Note: We should chat about at least on the edit page displaying that this data is coming from wikidata. But that can be a future change.

@cdrini cdrini merged commit 1426c38 into master May 2, 2024
4 checks passed
@cdrini cdrini deleted the infobox-with-wikidata branch May 2, 2024 16:27
@cdrini cdrini self-assigned this May 23, 2024
@tfmorris
Copy link
Contributor

tfmorris commented Nov 6, 2024

Is there a reason that this isn't linked to the issue #710 that it is intended to address? The side effect is that anyone who's interested in Wikidata integration and following that issue won't have been pinged to review this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants