[question] daemon concept #413

listvin · 2023-11-25T15:20:50Z

While caching compiled code is great,
one of the concerns I have while trying Kotlin scripting is startup time.

What's your opinion on possibility of JVM running as daemon, ready to pickup scripts on startup?
Similar in some manner to Gradle daemon.

Some properties to clarify concept:

start daemon when first script is started since system startup
- limited lifetime
- optional startup when system boots
dynamically load bytecode
one daemon per every runtime configuration
- maybe several daemons per runtime if there are more than one script running

I believe it will give performance boost by utilizing JIT out of the box.

aartiPl · 2023-11-26T16:04:23Z

Hi! This is an interesting idea. I am very curious about the real gains of daemon implementation compared to the implementation without daemon.

ybznek · 2023-12-22T22:49:57Z

Startup time of compiled (.jar) script
if it is an issue it could be compiled into native code (https://www.graalvm.org/latest/reference-manual/native-image/basics/), but the binary is roughly 8MB at minimum (maybe could be adjusted more?) and compilation is slow.
Task of gradle daemon is "just" compilation. Shared daemon across various scripts would require some isolation - e.g. you don't want to kill daemon if one of scripts call exit(), etc.

Initial startup time of not complied script:
I've done experiment with my <1 day project similar to kscript.
I do not call kotlinc, but ./gradlew and pass variables like srcDir, dependencies, .. via env. variables
First start with kscript seems to be roughly 6x slower than my solution with running gradle daemon

Significant slowdown of kscript seems to be due to resolution of dependency. My solution just collect
runtimeClasspath from gradle (/home/z/.gradle/caches/modules-2/files-2.1/org.jsoup/jsoup/1.16.2/5d39ff6d1b8331c376a04c50c6716afbcdfe567d/jsoup-1.16.2.jar) - and let gradle to download that.

Script:

#!/usr/bin/env /home/z/projects/kotlin/kscript2/runner/kscript2
@file:DependsOn("org.jsoup:jsoup:1.16.2")
import org.jsoup.Jsoup
println("a")
println( Jsoup.parse("<html><body id=\"b\">sth</body></html>")!!.getElementById("b")!!.text())

First run with kscript:

time kscript /home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a
[kscript] Resolving org.jsoup:jsoup:1.16.2...
a
sth
kscript  a  5,37s user 0,50s system 263% cpu 2,225 total

First run with my experiment with running daemon:

time /home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a
a
sth
/home/z/projects/kotlin/kscript2/runner/src/test/resources/source/main.kts a  0,86s user 0,16s system 98% cpu 1,027 total

aartiPl · 2023-12-26T14:57:31Z

Nice - 6x improvement in start-up looks very promising 👍

listvin · 2024-01-10T12:16:48Z

@ybznek thanks for actual minifyed timing sample! Can you please share your project as repo?

Shared daemon across various scripts would require some isolation - e.g. you don't want to kill daemon if one of scripts call exit(), etc.

Nice concern. Do you have more ideas of isolation cases?
Regarding .exit probably auto-restarting daemon, if it exits or crashes on its own (not via ~ $ kscriptd stop-daemon), would be sufficient at least for a poc-version?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[question] daemon concept #413

[question] daemon concept #413

listvin commented Nov 25, 2023 •

edited

Loading

aartiPl commented Nov 26, 2023

ybznek commented Dec 22, 2023

aartiPl commented Dec 26, 2023

listvin commented Jan 10, 2024 •

edited

Loading

[question] daemon concept #413

[question] daemon concept #413

Comments

listvin commented Nov 25, 2023 • edited Loading

aartiPl commented Nov 26, 2023

ybznek commented Dec 22, 2023

aartiPl commented Dec 26, 2023

listvin commented Jan 10, 2024 • edited Loading

listvin commented Nov 25, 2023 •

edited

Loading

listvin commented Jan 10, 2024 •

edited

Loading