GitHub

Small Clojure Interpreter

I want a limited dialect of Clojure for a single-purpose, scripted application. Sci will fit nicely.
— @tiagoluchini

Quickstart

Use from Clojure(Script)

(require '[sci.core :as sci])
(sci/eval-string "(inc 1)") => ;; 2
(sci/eval-string "(inc x)" {:bindings {'x 2}}) ;;=> 3

More on how to use sci from Clojure. Use from JavaScript.

Why

You want to evaluate code from user input, or use Clojure for a DSL inside your project, but eval isn't safe or simply doesn't work.

This library works with:

Clojure on the JVM
Clojure compiled with GraalVM native
ClojureScript, even when compiled with :advanced, and (as a consequence) JavaScript

It is used in:

Babashka. A Clojure scripting tool that plays well with Bash.
Bootleg. An HTML templating CLI.
Bytefield-svg. NodeJS library to generate byte field diagrams.
Chlorine. Socket-REPL and nREPL package for Atom editor.
Clj-kondo. A Clojure linter that sparks joy.
Closh. Bash-like shell based on Clojure. GraalVM port is work in progress.
Dad. A configuration management tool.
Firn. Org-mode static site generator.
Jet. CLI to convert between JSON, EDN and Transit.
Logseq. A local-only outliner notebook which supports both Markdown and Org mode.
Malli. Plain data Schemas for Clojure/Script.
PCP. Clojure Processor (PHP replacement).
Spire. Pragmatic provisioning using Clojure.

Status

Experimental. Breaking changes are expected to happen at this phase.

Installation

Use as a dependency:

API docs

For Clojure, see the generated codox documentation.

Usage

Currently the only API function is sci.core/eval-string which takes a string to evaluate and an optional options map.

In sci, defn does not mutate the outside world, only the evaluation context inside a call to sci/eval-string.

By default sci only enables access to the pure non-side-effecting functions in Clojure. More functions can be enabled, at your own risk, by using :bindings. Normally you would use sci's version of println but here, for the purposes of demonstration, we use use Clojure's version of println instead:

user=> (require '[sci.core :as sci])
user=> (sci/eval-string "(println \"hello\")" {:bindings {'println println}})
hello
nil

It is also possible to provide namespaces which can be required:

user=> (def opts {:namespaces {'foo.bar {'println println}}})
user=> (sci/eval-string "(require '[foo.bar :as lib]) (lib/println \"hello\")" opts)
hello
nil

In fact {:bindings ...} is just shorthand for {:namespaces {'user ...}}.

You can provide a list of allowed symbols. Using other symbols causes an exception:

user=> (sci/eval-string "(inc 1)" {:allow '[inc]})
2
user=> (sci/eval-string "(dec 1)" {:allow '[inc]})
ExceptionInfo dec is not allowed! [at line 1, column 2]  clojure.core/ex-info (core.clj:4739)

Providing a list of disallowed symbols has the opposite effect:

user=> (sci/eval-string "(inc 1)" {:deny '[inc]})
ExceptionInfo inc is not allowed! [at line 1, column 2]  clojure.core/ex-info (core.clj:4739)

Preventing forever lasting evaluation of infinite sequences can be achieved with :realize-max:

user=> (sci/eval-string "(vec (range))" {:realize-max 10})
ExceptionInfo Maximum number of elements realized: 10 [at line 1, column 1]  clojure.core/ex-info (core.clj:4739)

The preset :termination-safe, which is currently {:deny '[loop recur trampoline] :realize-max 100}, is helpful for making expressions terminate:

user=> (sci/eval-string "(loop [] (recur))" {:preset :termination-safe})
ExceptionInfo loop is not allowed! [at line 1, column 2]  clojure.core/ex-info (core.clj:4739)

Providing a macro as a binding can be done by providing a normal function that:

has :sci/macro on the metadata set to true
has two extra arguments at the start for &form and &env:

user=> (def do-twice ^:sci/macro (fn [_&form _&env x] (list 'do x x)))
user=> (sci/eval-string "(do-twice (f))" {:bindings {'do-twice do-twice 'f #(println "hello")}})
hello
hello
nil

Vars

To remain safe and sandboxed, sci evaluated Clojure does not have access to Clojure runtime vars. Sci has its own var type, distinguished from Clojure vars.

In a sci program these vars are created with def and defn just like in normal Clojure:

(def x 1)
(defn foo [] x)
(foo) ;;=> 1
(def x 2)
(foo) ;;=> 2

Dynamic vars with thread-local bindings are also supported:

(def ^:dynamic *x* 1)
(binding [*x* 10] x) ;;=> 10
(binding [*x* 10] (set! x 12) x) ;;=> 12
x ;;=> 1

Pre-creating vars that can be used in a sci program can be done using sci/new-var:

(def x (sci/new-var 'x 10))
(sci/eval-string "(inc x)" {:bindings {'x x}}) ;;=> 11

To create a dynamic sci var you can set metadata or use sci/new-dynamic-var:

(require '[sci.core] :as sci)
(def x1 (sci/new-var 'x 10 {:dynamic true}))
(sci/eval-string "(binding [*x* 12] (inc *x*))" {:bindings {'*x* x1}}) ;;=> 13
(def x2 (sci/new-dynamic-var 'x 10))
(sci/eval-string "(binding [*x* 12] (inc *x*))" {:bindings {'*x* x2}}) ;;=> 13

Pre-created sci vars can also be externally rebound:

(def x (sci/new-dynamic-var 'x 10))
(sci/binding [x 11] (sci/eval-string "(inc *x*)" {:bindings {'*x* x2}})) ;;=> 11

The dynamic vars *in*, *out*, *err* in a sci program correspond to the dynamic sci vars sci.core/in, sci.core/out and sci.core/err in the API. These vars can be rebound as well:

(def sw (java.io.StringWriter.))
(sci/binding [sci/out sw] (sci/eval-string "(println \"hello\")")) ;;=> nil
(str sw) ;;=> "hello\n"

A shorthand for rebinding sci/out is sci/with-out-str:

(sci/with-out-str (sci/eval-string "(println \"hello\")")) ;;=> "hello\n"

Stdout and stdin

To enable printing to stdout and reading from stdin you can sci bind sci.core/out and sci.core/in to *out* and *in* respectively:

(sci/binding [sci/out *out*
              sci/in *in*]
  (sci/eval-string "(print \"Type your name!\n> \")")
  (sci/eval-string "(flush)")
  (let [name (sci/eval-string "(read-line)")]
    (sci/eval-string "(printf \"Hello %s!\" name)
                      (flush)"
                     {:bindings {'name name}})))
Type your name!
> Michiel
Hello Michiel!

When adding a Clojure function to sci that interacts with *out* (or *in* or *err*), you can hook it up to sci's world. For example, a Clojure function that writes to *out* can be Clojure bound to sci's out:

user=> (defn foo [] (println "yello!"))
#'user/foo
user=> ;; without binding *out* to sci's out, the Clojure function will use its default *out*:
user=> (sci/eval-string "(with-out-str (foo))" {:bindings {'foo foo}})
yello!
""
;; let's hook foo up to sci's world:
user=> (defn wrapped-foo [] (binding [*out* @sci/out] (foo)))
#'user/wrapped-foo
user=> (sci/eval-string "(with-out-str (foo))" {:bindings {'foo wrapped-foo}})
"yello!\n"

Futures

Creating threads with future and pmap is disabled by default, but can be enabled by requiring sci.addons.future and applying the sci.addons.future/install function to the sci options:

(ns my.sci.app
  (:require
   [sci.core :as sci]
   [sci.addons.future :as future]))

(sci/eval-string "@(future (inc x))"
                 (-> {:bindings {'x 1}}
                     (future/install)))
;;=> 2

For conveying thread-local sci bindings to an external future use sci.core/future:

(ns my.sci.app
  (:require
   [sci.core :as sci]
   [sci.addons.future :as future]))

(def x (sci/new-dynamic-var 'x 10))

@(sci/binding [x 11]
   (sci/future
     (sci/eval-string "@(future (inc x))"
                      (-> {:bindings {'x @x}}
                          (future/install)))))
;;=> 12

Classes

Adding support for classes is done via the :classes option:

(sci/eval-string "(java.util.UUID/randomUUID)"
  {:classes {'java.util.UUID java.util.UUID}})
;;=> #uuid "312ba519-37e2-4109-b164-97fb140b57b0"

To make this work with GraalVM you will also need to add an entry to your reflection config for this class. Also see reflection.json.

State

Sci uses an atom to keep track of state changes like newly defined namespaces and vars. You can carry this state over from one call to another by providing the atom yourself as the value for the :env key:

(def env (atom {})
(sci/eval-string "(defn foo [] :foo)" {:env env})
(sci/eval-string "(foo)" {:env env}) ;;=> :foo

The contents of the the :env atom should be considered implementation detail.

Using an :env atom you are allowed to change options at each invocation of eval-string. If your use case doesn't require this, the recommendation is to use a sci context instead.

A sci context is derived once from options as documented in sci.core/eval-string and contains the runtime state of a sci session.

(def opts {:namespaces {'foo.bar {'x 1}}})
(def sci-ctx (sci/init opts))

Once created, a sci context should be considered final and should not be mutated by the user. The contents of the sci context should be considered implementation detail.

The sci context can be re-used over successive invocations of sci.core/eval-string*.

The major difference between eval-string and eval-string* is that eval-string will call init on the passed options and will pass that through to eval-string*. When you create a sci context yourself, you can skip the extra work that eval-string does and work directly with eval-string*.

(sci/eval-string* sci-ctx "foo.bar/x") ;;=> 1
(sci/eval-string* sci-ctx "(ns foo.bar) (def x 2) x") ;;=> 2
(sci/eval-string* sci-ctx "foo.bar/x") ;;=> 2

In a multi-user environment it can be useful to give each user their own context. This can already be achieved with eval-string, but for performance reasons it may be desirable to initialize a shared context. This shared context can then be forked for each user so that changes in one user's context aren't visible to other users:

(def forked (sci/fork sci-ctx))
(sci/eval-string* forked "(def forked 1)")
(sci/eval-string* forked "forked") ;;=> 1
(sci/eval-string* sci-ctx "forked") ;;=> Could not resolved symbol: forked

Implementing require and load-file

Sci supports implementation of code loading via a function hook that is invoked by sci's internal implementation of require. The job of this function is to find and return the source code for the requested namespace. This passed-in function will be called with a single argument that is a hashmap with a key :namespace. The value for this key will be the symbol of the requested namespace.

This function can return a hashmap with the keys :file (containing the filename to be used in error messages) and :source (containing the source code text) and sci will evaluate that source code to satisfy the require. Alternatively the function can return nil which will result in sci throwing an exception that the namespace can not be found.

This custom function is passed into the sci context under the :load-fn key as shown below.

(defn load-fn [{:keys [namespace]}]
  (when (= namespace 'foo)
    {:file "foo.clj"
     :source "(ns foo) (def val :foo)"}))
(sci/eval-string "(require '[foo :as fu]) fu/val" {:load-fn load-fn})
;;=> :foo

Note that internally specified namespaces (either those within sci itself or those mounted under the :namespaces context setting) will be utilised first and load-fn will not be called in those cases, unless :reload or :reload-all are used:

(sci/eval-string
  "(require '[foo :as fu])
   fu/val"
  {:load-fn load-fn
   :namespaces {'foo {'val (sci/new-var 'val :internal)}}})
;;=> :internal

(sci/eval-string
  "(require '[foo :as fu] :reload)
   fu/val"
  {:load-fn load-fn
   :namespaces {'foo {'val (sci/new-var 'val :internal)}}})
;;=> :foo

Another option for loading code is to provide an implementation of clojure.core/load-file. An example is presented here.

(ns my.sci.app
    (:require [sci.core :as sci]
              [clojure.java.io :as io]))

(spit "example1.clj" "(defn foo [] :foo)")
(spit "example2.clj" "(load-file \"example1.clj\")")

(let [env (atom {})
      opts {:env env}
      load-file (fn [file]
                  (let [file (io/file file)
                        source (slurp file)]
                    (sci/with-bindings
                      {sci/ns @sci/ns
                       sci/file (.getAbsolutePath file)}
                      (sci/eval-string source opts))))
      opts (assoc-in opts [:namespaces 'clojure.core 'load-file] load-file)]
  (sci/eval-string "(load-file \"example2.clj\") (foo)" opts))
;;=> :foo

Positional metadata

Sci includes positional metadata on every parsed Clojure element:

(sci/eval-string "(meta [1 2 3])")
;;=> {:line 1, :column 7, :end-line 1, :end-column 14}

This positional metadata is used by sci for error reporting.

Feature parity

Currently the following special forms/macros are supported: def, fn, function literals (#(inc %)), defn, quote, do,if, if-let, if-not, if-some when, when-first, when-let, when-not, when-some, cond, let, letfn, and, or, ->, ->>, as->, comment, loop, lazy-seq, for, doseq, case, try/catch/finally, declare, cond->, cond->>, some->, require, import, in-ns, ns, binding, with-out-str, with-in-str, future. Sci also supports user defined macros.

More examples of what is currently possible can be found at babashka.

If you miss something, feel free to post an issue.

GraalVM

Random numbers

To make the rand-* functions behave well when compiling to a GraalVM native binary, use this setting:

--initialize-at-run-time=java.lang.Math\$RandomNumberGeneratorHolder

Java 11

To use sci with GraalVM java11 override the dependency [borkdude/sci.impl.reflector "0.0.1"] to [borkdude/sci.impl.reflector "0.0.1-java11] in your project.clj or deps.edn.

Also you'll likely need a fix for clojure.lang.Reflector:

See clj-graal-docs and clj-reflector-graal-java11-fix.

Use from JavaScript

Sci is available on NPM:

$ npm install @borkdude/sci

The JavaScript API consists of two functions, evalString to evaluate Clojure expressions and toJS to convert Clojure data structures back to JavaScript.

> const { evalString, toJS } = require('@borkdude/sci');
> x = evalString("(assoc {:a 1} :b 2)")
> toJS(x)
{ a: 1, b: 2 }

The function evalString takes an optional second argument to pass options. Read here how to use those options. Instead of symbols and keywords it expects strings. Instead of kebab-case, use camelCase.

Use as native shared library

To use sci as a native shared library from e.g. C, C++, Rust, read this tutorial.

Test

Required: lein, the clojure CLI and GraalVM.

To succesfully run the GraalVM tests, you will have to compile the binary first with script/compile.

To run all tests:

script/test/all

For running individual tests, see the scripts in script/test.

Thanks

adgoji for financial support.
Lee Read for the logo.
contributors and other users posting issues with bug reports and ideas

License

Distributed under the Eclipse Public License 1.0. This project contains code from Clojure and ClojureScript which are also licensed under the EPL 1.0. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 578 Commits
.circleci		.circleci
.clj-kondo		.clj-kondo
.github		.github
doc		doc
libsci		libsci
logo		logo
reflector		reflector
resources		resources
script		script
src/sci		src/sci
test/sci		test/sci
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
deps.edn		deps.edn
min.js.edn		min.js.edn
package.json		package.json
project.clj		project.clj
reflection.json		reflection.json
shadow-cljs.edn		shadow-cljs.edn

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Quickstart

Use from Clojure(Script)

Why

Status

Installation

API docs

Usage

Vars

Stdout and stdin

Futures

Classes

State

Implementing require and load-file

Positional metadata

Feature parity

GraalVM

Random numbers

Java 11

Use from JavaScript

Use as native shared library

Test

Thanks

License

About

Releases

Packages

Languages

License

bbss/sci

Folders and files

Latest commit

History

Repository files navigation

Quickstart

Use from Clojure(Script)

Why

Status

Installation

API docs

Usage

Vars

Stdout and stdin

Futures

Classes

State

Implementing require and load-file

Positional metadata

Feature parity

GraalVM

Random numbers

Java 11

Use from JavaScript

Use as native shared library

Test

Thanks

License

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages