Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOAB Fetcher now fetches ISBN and imprint fields where possible #9229

Merged
merged 3 commits into from
Oct 12, 2022

Conversation

shafinkamal
Copy link
Contributor

@shafinkamal shafinkamal commented Oct 10, 2022

Fixes #8708.

Previously, the DOAB Fetcher could not obtain the ISBN of a DOAB entry. This PR modifies the URL for DOAB query to include BITSTREAM data (where ISBN information can be found).

Below is an example of the previous and updated URL's for DOAB queries (note the ",bitstreams" suffix)

PREVIOUS: https://directory.doabooks.org/rest/search?query=handle:%2220.500.12854/26303%22&expand=metadata

UPDATED: https://directory.doabooks.org/rest/search?query=handle:%2220.500.12854/26303%22&expand=metadata,bitstreams

The fetcher can then obtain the ISBN and add the field to the entry accordingly. I am creating this Draft PR to discuss with maintainers that it seems like many DOAB entries do not have ISBN metadata or BITSTREAM objects and so the ISBN may perhaps be unobtainable in these instances. So I am wondering if there is anything more can be done or if (under the assumption that this PR is accepted firstly) #8708 should be closed?

  • Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
  • Tests created for changes (if applicable)
  • Manually tested changed features in running JabRef (always required)
  • Screenshots added in PR description (for UI changes)
  • Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
  • Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

This commit modifies the URL for DOAB query to include `BITSTREAM` data. The fetcher can then obtain the ISBN and add the field to the entry accordingly. fixes JabRef#8708.
@ThiloteE
Copy link
Member

ThiloteE commented Oct 10, 2022

When DOAB entries lack ISBN, then obviously JabRef can't fetch them. That's fine then. No more work needed to be done. 👍

As long as your changes do not break anything and JabRef now can fetch the ISBN and the imprint fields, issue 8708 can indeed be closed.

@shafinkamal
Copy link
Contributor Author

Thanks for clarifying, @ThiloteE.

To follow up, which StandardField should the imprint field be associated with? The only related StandardField I see is PUBLISHER which is already associated with is publisher.name.

Thanks again :)

@ThiloteE
Copy link
Member

You are correct. Nice catch. imprint is indeed not a bibtex or biblatex standard field. Using https://en.wikipedia.org/wiki/Imprint_(trade_name) as basis and comparing with bibtex and biblatex standards, the bibtex / biblatex Publisher field indeed seems to be the closest to it.

So, I would introduce a "If the publisher field is empty, then map imprint to publisher" condition

I found an entry for testing:

https://directory.doabooks.org/handle/20.500.12854/70984 has both imprint and publisher.

I have not found an entry without publisher yet.

@shafinkamal
Copy link
Contributor Author

shafinkamal commented Oct 11, 2022

Hello, I am seeking advice/recommendations on how to resolve the failing checks. The 'Fetcher Tests' check seems to fail at DBLPFetcherTest which from my understanding, is not related to the commits I have made.

Also unsure on how to resolve the check: Deployment / Create installer and portable version for macOS.

Any help is greatly appreciated.

Thanks!

@ThiloteE
Copy link
Member

You can ignore those two. The fetcher error is a http error code 500, which points to serverside error and to compile the macOS version of JabRef there has to be done something related to certificates and/or injecting secrets, which usually is only done by maintainers for a proper release of JabRef.

@shafinkamal shafinkamal marked this pull request as ready for review October 11, 2022 13:25
@shafinkamal shafinkamal changed the title Draft: DOAB Fetcher now fetches ISBN field where possible DOAB Fetcher now fetches ISBN field where possible Oct 11, 2022
@shafinkamal
Copy link
Contributor Author

Thanks @ThiloteE. Marking the PR for review.

@shafinkamal shafinkamal changed the title DOAB Fetcher now fetches ISBN field where possible DOAB Fetcher now fetches ISBN and imprint fields where possible Oct 12, 2022
@Siedlerchr Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Oct 12, 2022
Copy link
Member

@Siedlerchr Siedlerchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! Overall looks good to me! Just a tiny improvement

@shafinkamal
Copy link
Contributor Author

Thanks @Siedlerchr. Committed + pushed the changes.

@calixtus calixtus merged commit d3ad678 into JabRef:main Oct 12, 2022
@calixtus
Copy link
Member

Thanks @shafinkamal for you contribution. Just merged it! Looking forward to see more 👍

@shafinkamal shafinkamal deleted the fix-for-issue-8708 branch October 13, 2022 00:28
Siedlerchr added a commit that referenced this pull request Oct 13, 2022
* upstream/main: (42 commits)
  Place subgroups w.r.t. alphabetical ordering (#9228)
  DOAB Fetcher now fetches ISBN and imprint fields where possible (#9229)
  ISSUE-9145: implement isbn fetcher (#9205)
  Remove explicit javafx jmod stuff (#9245)
  New Crowdin updates (#9239)
  Support Arabic language (#9243)
  Process CLI arguments before starting gui (#9217)
  Update README Decision Records link (#9241)
  Opening a file removes duplicate keywords and triggers lib changed (#9216)
  New Crowdin updates (#9237)
  Add Ukrainian as language (#9236)
  Add ADR-0026 on using Swing's FileChooser (#9235)
  Log exception (#9227)
  Fix archunit (#9234)
  Bump tika-core from 2.4.1 to 2.5.0 (#9232)
  Check group type before showing dialog in edit group (#8909)
  Fix unexpected closing of preferences dialog (#9225)
  Fix typo in connection error message (#9226)
  Improve VSCode config (#9218)
  Searching for an entry on the web shouldn't take forever (#9199)
  ...

# Conflicts:
#	build.gradle
#	lib/afterburner.fx.jar
#	src/main/java/org/jabref/cli/Launcher.java
#	src/main/java/org/jabref/gui/EntryTypeView.java
#	src/main/java/org/jabref/gui/auximport/FromAuxDialog.java
#	src/main/java/org/jabref/gui/bibtexextractor/ExtractBibtexDialog.java
#	src/main/java/org/jabref/gui/collab/ExternalChangesResolverDialog.java
#	src/main/java/org/jabref/gui/commonfxcontrols/CitationKeyPatternPanel.java
#	src/main/java/org/jabref/gui/contentselector/ContentSelectorDialogView.java
#	src/main/java/org/jabref/gui/customentrytypes/CustomizeEntryTypeDialogView.java
#	src/main/java/org/jabref/gui/documentviewer/DocumentViewerView.java
#	src/main/java/org/jabref/gui/edit/ManageKeywordsDialog.java
#	src/main/java/org/jabref/gui/entryeditor/EntryEditor.java
#	src/main/java/org/jabref/gui/entryeditor/fileannotationtab/FileAnnotationTabView.java
#	src/main/java/org/jabref/gui/errorconsole/ErrorConsoleView.java
#	src/main/java/org/jabref/gui/exporter/CreateModifyExporterDialogView.java
#	src/main/java/org/jabref/gui/externalfiles/UnlinkedFilesDialogView.java
#	src/main/java/org/jabref/gui/help/AboutDialogView.java
#	src/main/java/org/jabref/gui/importer/ImportCustomEntryTypesDialog.java
#	src/main/java/org/jabref/gui/importer/ImportEntriesDialog.java
#	src/main/java/org/jabref/gui/integrity/IntegrityCheckDialog.java
#	src/main/java/org/jabref/gui/libraryproperties/AbstractPropertiesTabView.java
#	src/main/java/org/jabref/gui/libraryproperties/LibraryPropertiesView.java
#	src/main/java/org/jabref/gui/libraryproperties/constants/ConstantsPropertiesView.java
#	src/main/java/org/jabref/gui/libraryproperties/general/GeneralPropertiesView.java
#	src/main/java/org/jabref/gui/libraryproperties/keypattern/KeyPatternPropertiesView.java
#	src/main/java/org/jabref/gui/libraryproperties/saving/SavingPropertiesView.java
#	src/main/java/org/jabref/gui/linkedfile/LinkedFileEditDialogView.java
#	src/main/java/org/jabref/gui/openoffice/ManageCitationsDialogView.java
#	src/main/java/org/jabref/gui/openoffice/StyleSelectDialogView.java
#	src/main/java/org/jabref/gui/preferences/AbstractPreferenceTabView.java
#	src/main/java/org/jabref/gui/preferences/PreferencesDialogView.java
#	src/main/java/org/jabref/gui/preferences/customexporter/CustomExporterTab.java
#	src/main/java/org/jabref/gui/preferences/externalfiletypes/EditExternalFileTypeEntryDialog.java
#	src/main/java/org/jabref/gui/preferences/journals/JournalAbbreviationsTab.java
#	src/main/java/org/jabref/gui/preferences/keybindings/KeyBindingsTab.java
#	src/main/java/org/jabref/gui/preferences/preview/PreviewTab.java
#	src/main/java/org/jabref/gui/preferences/protectedterms/ProtectedTermsTab.java
#	src/main/java/org/jabref/gui/search/GlobalSearchResultDialog.java
#	src/main/java/org/jabref/gui/shared/SharedDatabaseLoginDialogView.java
#	src/main/java/org/jabref/gui/slr/ManageStudyDefinitionView.java
#	src/main/java/org/jabref/gui/texparser/ParseLatexDialogView.java
#	src/main/java/org/jabref/gui/texparser/ParseLatexResultView.java
Siedlerchr added a commit that referenced this pull request Oct 17, 2022
* upstream/main:
  Make autosave tick in shared database opening dialog active (#9258)
  Bump controlsfx from 11.1.1 to 11.1.2 (#9261)
  Bump actions/configure-pages from 1 to 2 (#9262)
  Bump hmarr/auto-approve-action from 2.4.0 to 3.0.0 (#9259)
  Bump gittools/actions from 0.9.13 to 0.9.14 (#9260)
  Fix for resizing cleanup entries dialog (#9240)
  Refresh example styles
  Squashed 'buildres/csl/csl-locales/' changes from cb98d36691..e243665390
  Squashed 'buildres/csl/csl-styles/' changes from 7bde3e4..4eee79a
  Fix casing
  Update contributing.md
  Remove obsolete Jekyll howto
  Updates Just-the-Docs from 0.3.3 to 0.4.0.rc3 (#9249)
  Place subgroups w.r.t. alphabetical ordering (#9228)
  DOAB Fetcher now fetches ISBN and imprint fields where possible (#9229)
  ISSUE-9145: implement isbn fetcher (#9205)
  Remove explicit javafx jmod stuff (#9245)
  New Crowdin updates (#9239)
Siedlerchr added a commit that referenced this pull request Oct 21, 2022
* upstream/main: (28 commits)
  Minor code improvements in library properties dialog (#9265)
  Adjust and make tests for BibTex/Biblatex/CSL mapping more accurate by adding Apa 7th edition (#9255)
  Removed swing from default file dir detection (#9222)
  Make autosave tick in shared database opening dialog active (#9258)
  Bump controlsfx from 11.1.1 to 11.1.2 (#9261)
  Bump actions/configure-pages from 1 to 2 (#9262)
  Bump hmarr/auto-approve-action from 2.4.0 to 3.0.0 (#9259)
  Bump gittools/actions from 0.9.13 to 0.9.14 (#9260)
  Fix for resizing cleanup entries dialog (#9240)
  Refresh example styles
  Squashed 'buildres/csl/csl-locales/' changes from cb98d36691..e243665390
  Squashed 'buildres/csl/csl-styles/' changes from 7bde3e4..4eee79a
  Fix casing
  Update contributing.md
  Remove obsolete Jekyll howto
  Updates Just-the-Docs from 0.3.3 to 0.4.0.rc3 (#9249)
  Place subgroups w.r.t. alphabetical ordering (#9228)
  DOAB Fetcher now fetches ISBN and imprint fields where possible (#9229)
  ISSUE-9145: implement isbn fetcher (#9205)
  Remove explicit javafx jmod stuff (#9245)
  ...
Siedlerchr added a commit to TheGor1lla/jabref that referenced this pull request Oct 21, 2022
* upstream/main:
  [WIP] Add GitHub action: Greetings.yml - Automates advice to JabRefs first time code contributors (JabRef#9272)
  Minor code improvements in library properties dialog (JabRef#9265)
  Adjust and make tests for BibTex/Biblatex/CSL mapping more accurate by adding Apa 7th edition (JabRef#9255)
  Removed swing from default file dir detection (JabRef#9222)
  Make autosave tick in shared database opening dialog active (JabRef#9258)
  Bump controlsfx from 11.1.1 to 11.1.2 (JabRef#9261)
  Bump actions/configure-pages from 1 to 2 (JabRef#9262)
  Bump hmarr/auto-approve-action from 2.4.0 to 3.0.0 (JabRef#9259)
  Bump gittools/actions from 0.9.13 to 0.9.14 (JabRef#9260)
  Fix for resizing cleanup entries dialog (JabRef#9240)
  Refresh example styles
  Squashed 'buildres/csl/csl-locales/' changes from cb98d36691..e243665390
  Squashed 'buildres/csl/csl-styles/' changes from 7bde3e4..4eee79a
  Fix casing
  Update contributing.md
  Remove obsolete Jekyll howto
  Updates Just-the-Docs from 0.3.3 to 0.4.0.rc3 (JabRef#9249)
  Place subgroups w.r.t. alphabetical ordering (JabRef#9228)
  DOAB Fetcher now fetches ISBN and imprint fields where possible (JabRef#9229)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fetcher status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DOAB fetcher does not fetch field ISBN
4 participants