Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename macro project to core and simplify Scala 3 support #52

Merged
merged 2 commits into from
Jul 23, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changed
- Upgrade gjavah [#43](https://github.com/sbt/sbt-jni/pull/43)
- Use cmake platform build tool [#40](https://github.com/sbt/sbt-jni/issues/40)
- Rename macro project to core and simplify Scala 3 support [#52](https://github.com/sbt/sbt-jni/pull/52)

### Fixed
- javah failed with ClassCastException [#38](https://github.com/sbt/sbt-jni/issues/38)
Expand Down
66 changes: 24 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ The second point, portability, is inherent to JNI and thus unavoidable. However
| JniLoad | Makes `@nativeLoader` annotation available, that injects code to transparently load native libraries. |
| JniNative | Adds sbt wrapper tasks around native build tools to ease building and integrating native libraries. |
| JniPackage | Packages native libraries into multi-platform fat jars. No more manual library installation! |
| JniSyntax | Adds an alternative to `@nativeLoader` annotation syntax, that requires this plugin to be a run time dependency |

All plugins are made available by adding the following to `project/plugins.sbt`:
```scala
Expand Down Expand Up @@ -74,7 +73,7 @@ JNIEXPORT jint JNICALL Java_org_example_Adder_plus

The header output directory can be configured
```
target in javah := <dir> // defaults to target/native/include
javah / target := <dir> // defaults to target/native/include
```

Note that native methods declared both in Scala and Java are supported. Whereas Scala uses the `@native` annotation, Java uses the
Expand All @@ -85,8 +84,6 @@ Note that native methods declared both in Scala and Java are supported. Whereas
|--------------------------------|---------------|
| automatic, for all projects | [JniLoad.scala](plugin/src/main/scala/ch/jodersky/sbt/jni/plugins/JniLoad.scala) |

**! Important**: *`@nativeLoader` annotation works with Scala 2.x only. You may want to consider the [JniSyntax](#jnisyntax) plugin usage for the Scala 3.x projects.*

This plugin enables loading native libraries in a safe and transparent manner to the developer (no more explicit, static `System.load("library")` calls required). It does so by providing a class annotation which injects native loading code to all its annottees. Furthermore, in case a native library is not available on the current `java.library.path`, the code injected by the annotation will fall back to loading native libraries packaged according to the rules of `JniPackage`.

#### Example use (Scala 2.x):
Expand All @@ -106,13 +103,23 @@ object Main extends App {
}
```

Note: this plugin is just a shorthand for adding `sbt-jni-macros` (the project in `macros/`) and the scala-macros-paradise (on Scala <= 2.13) projects as provided dependencies.
Note: this plugin is just a shorthand for adding `sbt-jni-core` (the project in `core/`) and the scala-macros-paradise (on Scala <= 2.13) projects as provided dependencies.

See the [annotation's implementation](core/src/main/scala/ch/jodersky/jni/annotations.scala) for details about the injected code.

See the [annotation's implementation](macros/src/main/scala/ch/jodersky/jni/annotations.scala) for details about the injected code.
#### Example use (Scala 3.x / Scala 2.x):

#### Example use (Scala 3.x):
Scala 3 has no macro annotations support. As a solution we don't need this to be a macro function anymore. As the result, this option requires to have an explicit dependency on the [core](./core) sub project.

This plugin behavior is configurable via:

```scala
sbtJniCoreProvided := <boolean> // set to true by default, and is enough make @nativeLoader annotation work
```

```scala
// to make the code below work the core project should be included as a dependency via
// sbtJniCoreProvided := false
import ch.jodersky.jni.syntax.NativeLoader

// By adding this annotation, there is no need to call
Expand All @@ -124,8 +131,6 @@ class Adder(val base: Int) extends NativeLoader("adder0"):
@main def main: Unit = (new Adder(0)).plus(1)
```

Requires [JniSyntax](#JniSyntax) plugin usage.

### JniNative
| Enabled | Source |
|--------------------------------|---------------|
Expand All @@ -141,8 +146,8 @@ An initial, compatible build template can be obtained by running `sbt nativeInit

Source and output directories are configurable
```scala
sourceDirectory in nativeCompile := sourceDirectory.value / "native",
target in nativeCompile := target.value / "native" / (nativePlatform).value,
nativeCompile / sourceDirectory := sourceDirectory.value / "native",
nativeCompile / target := target.value / "native" / (nativePlatform).value,
```

### JniPackage
Expand All @@ -152,42 +157,18 @@ target in nativeCompile := target.value / "native" / (nativePlatform).value,

This plugin packages native libraries produced by JniNative in a way that they can be transparently loaded with JniLoad. It uses the notion of a native "platform", defined as the architecture-kernel values returned by `uname -sm`. A native binary of a given platform is assumed to be executable on any machines of the same platform.

### JniSyntax
| Enabled | Source |
|--------------------------------|---------------|
| manual | [JniSyntax.scala](plugin/src/main/scala/ch/jodersky/sbt/jni/plugins/JniSyntax.scala) |

Scala 3 has no macro annotations support. JniSyntax contains syntax to ease usage of the `ch.jodersky.jni.nativeLoaderMacro` in the project (see [JniLoad](#JniLoad) section for more details). This option requires to have runtime dependencies on [macros](./macros) and [core](./core) sub projects.

#### Example use (Scala 2.x / 3.x):

```scala
import ch.jodersky.jni.syntax.NativeLoader

// By adding this annotation, there is no need to call
// System.load("adder0") before accessing native methods.
class Adder(val base: Int) extends NativeLoader("adder0") {
@native def plus(term: Int): Int // implemented in libadder0.so
}

// The application feels like a pure Scala app.
object Main extends App {
(new Adder(0)).plus(1)
}
```

## Canonical Use

*Keep in mind that sbt-jni is a __suite__ of plugins, there are many other use cases. This is a just a description of the most common one.*

1. Define separate sub-projects for JVM and native sources. In `myproject/build.sbt`:

```scala
lazy val core = project in file("myproject-core"). // regular scala code with @native methods
dependsOn(native % Runtime) // remove this if `core` is a library, leave choice to end-user
lazy val core = project in file("myproject-core") // regular scala code with @native methods
.dependsOn(native % Runtime) // remove this if `core` is a library, leave choice to end-user

lazy val native = project in file("myproject-native"). // native code and build script
enablePlugin(JniNative) // JniNative needs to be explicitly enabled
lazy val native = project in file("myproject-native") // native code and build script
.enablePlugin(JniNative) // JniNative needs to be explicitly enabled
```
Note that separate projects are not strictly required. They are strongly recommended nevertheless, as a portability-convenience tradeoff: programs written in a JVM language are expected to run anywhere without recompilation, but including native libraries in jars limits this portability to only platforms of the packaged libraries. Having a separate native project enables the users to easily swap out the native library with their own implementation.

Expand Down Expand Up @@ -218,7 +199,7 @@ object Main extends App {
## Examples
The [plugins' unit tests](plugin/src/sbt-test/sbt-jni) offer some simple examples. They can be run individually through these steps:

1. Publish the macros library locally `sbt publishLocal`.
1. Publish the core library locally `sbt publishLocal`.
2. Change to the test's directory and run `sbt -Dplugin.version=<version>`.
3. Follow the instructions in the `test` file (only enter the lines that start with ">" into sbt).

Expand All @@ -229,14 +210,15 @@ Real-world use-cases of sbt-jni include:
## Requirements and Dependencies

- projects using `JniLoad` must use Scala versions 2.11, 2.12 or 2.13
- projects using `JniLoad` with Scala 3 should use it with
- only POSIX platforms are supported (actually, any platform that has the `uname` command available)

The goal of sbt-jni is to be the least intrusive possible. No transitive dependencies are added to projects using any plugin (some dependencies are added to the `provided` configuration, however these do not affect any downstream projects).

## Building
Both the macro library (`sbt-jni-macros`) and the sbt plugins (`sbt-jni`) are published. Cross-building happens on a per-project basis:
Both the core (former macros) library (`sbt-jni-core`) and the sbt plugins (`sbt-jni`) are published. Cross-building happens on a per-project basis:

- sbt-jni-macros is built against Scala 2.11, 2.12 and 2.13
- sbt-jni-core is built against Scala 2.11, 2.12 and 2.13
- sbt-jni is built against Scala 2.12 (the Scala version that sbt 1.x uses)

The differing Scala versions make it necessary to always cross-compile and cross-publish this project, i.e. append a "+" before every task.
Expand Down
23 changes: 7 additions & 16 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,18 @@ ThisBuild / developers := List(
)

lazy val root = (project in file("."))
.aggregate(macros, core, plugin)
.aggregate(core, plugin)
.settings(
publish := {},
publishLocal := {},
// make sbt-pgp happy
publishTo := Some(Resolver.file("Unused transient repository", target.value / "unusedrepo")),
addCommandAlias("test-plugin", ";+macros/publishLocal;+core/publishLocal;scripted")
addCommandAlias("test-plugin", ";+core/publishLocal;scripted")
)

lazy val macros = project
lazy val core = project
.settings(
name := "sbt-jni-macros",
name := "sbt-jni-core",
scalaVersion := scalaVersions.head,
crossScalaVersions := scalaVersions,
libraryDependencies ++= {
Expand All @@ -40,13 +40,12 @@ lazy val macros = project
"org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided,
"org.scala-lang" % "scala-reflect" % scalaVersion.value
)
case _ => Seq("org.scala-lang" %% "scala3-compiler" % scalaVersion.value)
case _ => Seq("org.scala-lang" %% "scala3-compiler" % scalaVersion.value % Provided)
}
},
libraryDependencies ++= {
CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, n)) if n >= 13 => Seq()
case Some((2, n)) =>
case Some((2, n)) if n < 13 =>
Seq(compilerPlugin(("org.scalamacros" % "paradise" % macrosParadiseVersion).cross(CrossVersion.full)))
case _ => Seq()
}
Expand All @@ -59,14 +58,6 @@ lazy val macros = project
}
)

lazy val core = project
.dependsOn(macros)
.settings(
name := "sbt-jni-core",
scalaVersion := scalaVersions.head,
crossScalaVersions := scalaVersions
)

lazy val plugin = project
.enablePlugins(SbtPlugin)
.settings(
Expand All @@ -80,7 +71,7 @@ lazy val plugin = project
|
|private[jni] object ProjectVersion {
| final val MacrosParadise = "${macrosParadiseVersion}"
| final val Macros = "${version.value}"
| final val Core = "${version.value}"
|}
|""".stripMargin
val file = sourceManaged.value / "ch" / "jodersky" / "sbt" / "jni" / "ProjectVersion.scala"
Expand Down
10 changes: 10 additions & 0 deletions core/src/main/scala-2/ch/jodersky/jni/Process.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package ch.jodersky.jni

object Process {
def out(command: String): String =
try {
scala.sys.process.Process("uname -sm").lineStream.head
} catch {
case ex: Exception => sys.error("Error running `uname` command")
}
}
10 changes: 10 additions & 0 deletions core/src/main/scala-3/ch/jodersky/jni/Process.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package ch.jodersky.jni

object Process {
def out(command: String): String =
try {
scala.sys.process.Process("uname -sm").lazyLines.head
} catch {
case ex: Exception => sys.error("Error running `uname` command")
}
}
55 changes: 53 additions & 2 deletions core/src/main/scala/ch/jodersky/jni/syntax/NativeLoader.scala
Original file line number Diff line number Diff line change
@@ -1,7 +1,58 @@
package ch.jodersky.jni.syntax

import ch.jodersky.jni.nativeLoaderMacro
import ch.jodersky.jni.Process

import java.nio.file.{Files, Path}

class NativeLoader(nativeLibrary: String) {
nativeLoaderMacro.load(nativeLibrary)
NativeLoader.load(nativeLibrary)
}

object NativeLoader {
def load(nativeLibrary: String): Unit = {
def loadPackaged(): Unit = {

val lib: String = System.mapLibraryName(nativeLibrary)

val tmp: Path = Files.createTempDirectory("jni-")
val plat: String = {
val line = Process.out("uname -sm")
val parts = line.split(" ")
if (parts.length != 2) {
sys.error("Could not determine platform: 'uname -sm' returned unexpected string: " + line)
} else {
val arch = parts(1).toLowerCase.replaceAll("\\s", "")
val kernel = parts(0).toLowerCase.replaceAll("\\s", "")
arch + "-" + kernel
}
}

val resourcePath: String = "/native/" + plat + "/" + lib
val resourceStream = Option(this.getClass.getResourceAsStream(resourcePath)) match {
case Some(s) => s
case None =>
throw new UnsatisfiedLinkError(
"Native library " + lib + " (" + resourcePath + ") cannot be found on the classpath."
)
}

val extractedPath = tmp.resolve(lib)

try {
Files.copy(resourceStream, extractedPath)
} catch {
case ex: Exception => throw new UnsatisfiedLinkError("Error while extracting native library: " + ex)
}

System.load(extractedPath.toAbsolutePath.toString)
}

def load(): Unit = try {
System.loadLibrary(nativeLibrary)
} catch {
case ex: UnsatisfiedLinkError => loadPackaged()
}

load()
}
}
62 changes: 0 additions & 62 deletions macros/src/main/scala-2/ch/jodersky/jni/nativeLoaderMacro.scala

This file was deleted.

Loading