[WIP] Use v8 engine for CSL #2250

shitikanth · 2016-11-08T10:23:05Z

Changes made:

Added a compile dependency on com.eclipsesource.j2v8 in build.gradle

We are using citeproc-java library in CitationStyleGenerator to generate previews, which itself is basically a wrapper around the javascript library citeproc-js. Switching to V8 engine dramatically improves the time taken to generate these previews. For example, on my laptop, generating preview used to take about 20s for the first preview, and 10s for each subsequent preview; using V8 reduces both by a factor of 10.

Manually tested changed features in running JabRef

…build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

matthiasgeiger · 2016-11-08T11:03:14Z

Hi and thanks for your contribution.

However, I suppose it will be difficult to actually use com.eclipsesource.j2v8 in this way: If the concrete packages are in fact platform dependent it won't be possible to provide a platform independet jar file anylonger.

shitikanth · 2016-11-08T18:31:32Z

Ooh good point! If the slow performance of CSL citation generation isn't an issue for others, I wouldn't bother about it too much. Maybe we can include V8 in the platform specific binaries though?

koppor · 2016-11-08T21:14:53Z

build.gradle

+    }
+    if (dep == null) {
+        logger.error("Could not find V8 runtime compatible to this system")
+        return null


Can't we just return an empty string here so that it compiles at other platforms, too?

Hmm, that is a good idea, but the jar we produce will still not be platform independent (as @matthiasgeiger mentioned). Maybe we can define different gradle tasks for a platform independent jar and platform specific builds.

What do you guys feel about just directly implementing citeproc natively in Java? I can take it up if you think that it will be worthwhile.

Seeing that (i) the current rendering is very slow and (ii) others will surely use that library too, citeproc natively in Java is a huge win for the community.

I fear, however, that it will be a huge effort and we would also need that effort in JabRef itself.

What do you guys feel about just directly implementing citeproc natively in Java? I can take it up if you think that it will be worthwhile.

Hi all, CSL maintainer here. I have never worked on a citeproc implementation myself, but just a friendly warning that supporting the full CSL 1.0.1 specification is a significant undertaking. I'm cc'ing some developers who might be able to give you some additional pointers/warnings.

(cc @michel-kraemer (citeproc-java), @fbennett (citeproc-js), @inukshuk (citeproc-ruby))

I can confirm this. In fact, I initially thought about creating a pure Java implementation myself, but citeproc-js is so mature and well-maintained, you'll probably never reach the same quality -- at least I would never be able to do so.

Anyway, there are a couple of options to improve the performance of citeproc-java. You already discovered you can use V8, although it isn't documented anywhere. You might be able to improve the performance further by caching the processor instance. But maybe you already do so, I don't know your code well enough. If you're interested we can discuss this topic further. I'd be more than happy to help.

koppor · 2016-11-08T21:21:33Z

Note that the code is based o https://github.com/michel-kraemer/citeproc-java/blob/master/citeproc-java/build.gradle#L26

https://github.com/michel-kraemer/citeproc-java/blob/master/citeproc-java/build.gradle#L56:

    if (getV8Dependency() != null) {
        provided getV8Dependency()
}

Maybe replace provided by compile?

koppor · 2016-11-13T09:14:07Z

I would vote for generating specific JARs and package them for MacOSX and Windows. For linux, we could generate two JARs, too.

koppor · 2016-11-13T10:55:11Z

Could you also update external-libraries.txt as described at https://github.com/JabRef/jabref/blob/master/CONTRIBUTING.md#when-adding-a-library. We need that to ease debian packaging (refs koppor#135).

koppor · 2016-11-13T11:00:45Z

I used your patch here with the new engine. It is very fast. Seeing that, I think, we should invest time for a correct build of JabRef and not reimplementing the whole functionality in Java. That time should better be spent in JabRef itself 😇

Siedlerchr · 2016-11-13T11:40:28Z

Another option would be to check if it works wirh jdk8 nashorn engine as discussed here michel-kraemer/citeproc-java#28

shitikanth · 2016-11-15T05:57:40Z

@Siedlerchr, I did try building with the nashorn engine, the performance wasn't improved by very much at least on my machine.

koppor · 2016-11-22T08:00:45Z

external-libraries.txt

+Id:      com.eclipse.j2v8
+Project: J2V8
+URL:     https://github.com/eclipsesource/J2V8
+Licence: EPL- 1.0


Could you remove the space before 1.0 so it is a valid SPDX license identifier?

koppor · 2016-11-28T07:37:15Z

@shitikanth Could you try to adapt the gradle build file to create a plattform-specific jar and a plattform-neutral jar? The speedup of the V8 engine is huge and I would really like to get this into the next JabRef release.

shitikanth · 2016-11-28T10:12:55Z

@koppor I am not familiar with JabRef's build/release process, so it will help me if can you specify what behaviour you want. For example, does the following expected behaviour look okay to you?

./gradlew releaseJar: creates platform specific build like JabRef--linux_x86-64.jar
./gradlew shadowJar: creates a platform independent build.

matthiasgeiger · 2016-11-28T11:47:55Z

I think during release the shadowJar variant is used which creates a fat-jar. The needed behavior would be to create 5 different fat-Jars: Windows x86, Win x64, Linux, MacOSX, and independent - which is then used to create the platform specific installers using install4j.

@stefan-kolb Can you assist here? Is it possible to use different jars in install4j?

stefan-kolb · 2016-11-28T13:12:00Z

Yes, shadowJar is used right now as far as I know. As for the executables, we probably need to add more launchers inside install4j for every configuration.

koppor · 2016-12-05T10:56:54Z

To illustrate the performance gain, I made two videos:

without V8

with V8

stefan-kolb · 2016-12-05T10:59:15Z

Hm, I ask myself why this stuff is taking so long anyway? It is just a String creation wtf?

koppor · 2016-12-05T11:09:36Z

http://eclipsesource.com/blogs/2014/11/17/highly-efficient-java-javascript-integration/:

Unlike other JS Runtimes (including JV8, Jav8), J2V8 takes a primitive based approach, resulting in much less garbage.

I am not in the internals of JavaScript engines and cannot really compare Rhino and Nashorn to J2V8.

koppor · 2016-12-05T11:15:15Z

IMHO the alternative is to implement citeproc-java directly in Java as suggested by @shitikanth at #2250 (comment). Maybe, this could be a project for the time between the years? 😇

koppor · 2016-12-05T17:36:38Z

I merged to use-V8-for-citeproc to enable other developers continuing working on it.

Add a dependency to com.eclipsesource.J2V8 (Java Bindings for V8) in build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

michel-kraemer · 2016-12-24T22:02:41Z

Hm, I ask myself why this stuff is taking so long anyway? It is just a String creation wtf?

The citeproc.js file is 664 KB large and the code is very complex. It seems Nashorn is not very fast when compiling and running such a file for the first time. However, since it uses JIT compilation, it gets very fast the longer the program runs and the more often you execute the file.

Add a dependency to com.eclipsesource.J2V8 (Java Bindings for V8) in build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

Add a dependency to com.eclipsesource.J2V8 (Java Bindings for V8) in …

7507fe8

…build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

koppor reviewed Nov 8, 2016

View reviewed changes

koppor added the type: enhancement label Nov 13, 2016

koppor changed the title ~~Use v8 engine for CSL~~ [WIP] Use v8 engine for CSL Nov 13, 2016

shitikanth added 2 commits November 16, 2016 08:37

Make sure that V8 dependency is available.

b1c4039

Add entry for J2V8 in external-libraries.txt.

5ee4d79

koppor reviewed Nov 22, 2016

View reviewed changes

Removed space from licence identifier.

c04f46b

koppor self-assigned this Dec 2, 2016

koppor added this to the v3.8 milestone Dec 2, 2016

koppor changed the base branch from master to use-V8-for-citeproc December 5, 2016 17:35

koppor merged commit de2823d into JabRef:use-V8-for-citeproc Dec 5, 2016

koppor pushed a commit that referenced this pull request Dec 5, 2016

Use v8 engine for CSL (#2250)

fdaf1c0

Add a dependency to com.eclipsesource.J2V8 (Java Bindings for V8) in build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

koppor mentioned this pull request Mar 27, 2017

Citation preview incrementally increases memory usage with each preview #2533

Closed

koppor pushed a commit that referenced this pull request Apr 17, 2017

Use v8 engine for CSL (#2250)

5563811

Add a dependency to com.eclipsesource.J2V8 (Java Bindings for V8) in build.gradle. Citeproc-java detects that V8 is present in the classpath and starts using it automatically.

koppor mentioned this pull request Jul 9, 2017

Preview pane has gotten really slow #2982

Closed

koppor mentioned this pull request Sep 1, 2017

[WIP] Use v8 for citeproc #3180

Closed

9 tasks

koppor mentioned this pull request Nov 10, 2019

Update citeproc-java to version 2.0.0 #5585

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Use v8 engine for CSL #2250

[WIP] Use v8 engine for CSL #2250

shitikanth commented Nov 8, 2016 •

edited

Loading

matthiasgeiger commented Nov 8, 2016

shitikanth commented Nov 8, 2016

koppor Nov 8, 2016

shitikanth Nov 8, 2016

koppor Nov 13, 2016

rmzelle Dec 24, 2016 •

edited

Loading

michel-kraemer Dec 24, 2016

koppor commented Nov 8, 2016 •

edited

Loading

koppor commented Nov 13, 2016

koppor commented Nov 13, 2016

koppor commented Nov 13, 2016

Siedlerchr commented Nov 13, 2016

shitikanth commented Nov 15, 2016

koppor Nov 22, 2016

shitikanth Nov 22, 2016

koppor commented Nov 28, 2016

shitikanth commented Nov 28, 2016

matthiasgeiger commented Nov 28, 2016

stefan-kolb commented Nov 28, 2016

koppor commented Dec 5, 2016

stefan-kolb commented Dec 5, 2016

koppor commented Dec 5, 2016

koppor commented Dec 5, 2016

koppor commented Dec 5, 2016

michel-kraemer commented Dec 24, 2016

[WIP] Use v8 engine for CSL #2250

[WIP] Use v8 engine for CSL #2250

Conversation

shitikanth commented Nov 8, 2016 • edited Loading

matthiasgeiger commented Nov 8, 2016

shitikanth commented Nov 8, 2016

koppor Nov 8, 2016

Choose a reason for hiding this comment

shitikanth Nov 8, 2016

Choose a reason for hiding this comment

koppor Nov 13, 2016

Choose a reason for hiding this comment

rmzelle Dec 24, 2016 • edited Loading

Choose a reason for hiding this comment

michel-kraemer Dec 24, 2016

Choose a reason for hiding this comment

koppor commented Nov 8, 2016 • edited Loading

koppor commented Nov 13, 2016

koppor commented Nov 13, 2016

koppor commented Nov 13, 2016

Siedlerchr commented Nov 13, 2016

shitikanth commented Nov 15, 2016

koppor Nov 22, 2016

Choose a reason for hiding this comment

shitikanth Nov 22, 2016

Choose a reason for hiding this comment

koppor commented Nov 28, 2016

shitikanth commented Nov 28, 2016

matthiasgeiger commented Nov 28, 2016

stefan-kolb commented Nov 28, 2016

koppor commented Dec 5, 2016

without V8

with V8

stefan-kolb commented Dec 5, 2016

koppor commented Dec 5, 2016

koppor commented Dec 5, 2016

koppor commented Dec 5, 2016

michel-kraemer commented Dec 24, 2016

shitikanth commented Nov 8, 2016 •

edited

Loading

rmzelle Dec 24, 2016 •

edited

Loading

koppor commented Nov 8, 2016 •

edited

Loading