Fix Google Scholar fetcher for downloading a single entry

[koppor]

GoogleScholar changed their page (IMHO)

First page:

Second "page"

Click on cited loads the content of that thing:

There, BibTeX can be downloaded as usual.

Example URL: https://scholar.google.ch/scholar?q=info:RExzBa3OlkQJ:scholar.google.com/&output=cite&scirp=0&hl=en

Third page

Example URL: https://scholar.googleusercontent.com/scholar.bib?q=info:RExzBa3OlkQJ:scholar.google.com/&output=citation&scisdr=CgVYZoPbEOvJ014vy3E:AAGBfm0AAAAAX6Mq03ED_BBuflXyRuQujflFTqExM8uU&scisig=AAGBfm0AAAAAX6Mq0_wSs1k5gywcNDtaUBn0PeTKsRGQ&scisf=4&ct=citation&cd=-1&hl=en

Block notice:

Summary

While programming, I came to the last step. The issue is that after 10 tries, I am banned and cannot continue.

• Change in CHANGELOG.md described (if applicable)
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Screenshots added in PR description (for UI changes)
• Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

[koppor]

• I think, the underlying http connection should be reused in order to send the cookies at the subsequent requests
• Try out with proxy so that one is not blocked

[Siedlerchr]

You can also just start JabRef with -debug as program argument.

[Siedlerchr]

If yu want to reuse the connection you should use unirest or jsoup

[koppor]

Refs #6369

[koppor]

Liebes Tagebuch,

bei Google Scholar ist die Reihenfolge der Wörter entscheidend.

Und jetzt hau ich "In" nach vorne:

[Siedlerchr]

You can either search by author or by title, for title you need to put in quotes: I don't understand your search query above.
https://scholar.google.com/intl/de/scholar/help.html#searching

[koppor]

Google Scholar also works when not using quotes. #convenience. Not sure whether our Google Scholar implementaiton should behave differently than when using their web page.

[tobiasdiez]

There is nothing wrong with the set... methods. The with... methods were added to quickly add one or two field values, mostly in lambda expressions, e.g. map(entry -> entry.withField(...))

[Siedlerchr]

You need to listen for the web engine ready event, see the preview Tab viewer where we add this highlight ja stuff

[Siedlerchr]

   previewView.getEngine().getLoadWorker().stateProperty().addListener((observable, oldValue, newValue) -> {

            if (newValue != Worker.State.SUCCEEDED) {
                return;
            }

See https://openjfx.io/javadoc/11/javafx.web/javafx/scene/web/WebEngine.html

[koppor]

You need to listen for the web engine ready event, see the preview Tab viewer where we add this highlight ja stuff

Is this happen synchronously? The interface for the Captcha solver is designed in a synchronous way. Otherwise all fetchers need to be changed.

I'll be away anyway for the next days. Thus, you are free to experiment 😅

[koppor]

This directly competes with #5943, where the browser is used to communicate with Google Scholar. We should write an ADR ^^.

[k3KAW8Pnf7mkmdSMPHz27]

Since this is on the 5.3 milestones, are we, at least for now, taking this approach?

[koppor]

Since this is on the 5.3 milestones, are we, at least for now, taking this approach?

Yes, we take this approach here. The other one heavily relies on JabRef's internal save handling. This is currently handled by @Siedlerchr in #6694. Hope, we will make progress somehow.

[koppor]

DevCall decision: Work-around using our browser extension exists. Thus, this is not a high-priority any more.

[koppor]

Other implementation hint: https://github.com/JetBrains/jcef