Skip to content

Commit

Permalink
Release 0.2.7 (#5)
Browse files Browse the repository at this point in the history
- Switch to deps.edn, shadow-cljs, kaocha
- Apply clj-fmt
- Set up CI with CircleCI
- Bump fress version
- Fix ClojureScript normalization.

Co-authored-by: Wade Dominic <wade@lambdaforge.io>
Co-authored-by: Christian Weilbach <ch_weil@topiq.es>
  • Loading branch information
3 people authored Mar 18, 2021
1 parent 0263090 commit 80a09f4
Show file tree
Hide file tree
Showing 20 changed files with 5,131 additions and 191 deletions.
46 changes: 46 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
version: 2.1

orbs:
tools: replikativ/clj-tools@0

workflows:
build-test-and-deploy:
jobs:
- tools/setup
- tools/format:
requires:
- tools/setup
- tools/unittest:
requires:
- tools/setup
- tools/cljstest:
requires:
- tools/setup
- tools/build:
jarname: "incognito"
requires:
- tools/setup
- tools/deploy-snapshot:
jarname: "incognito"
context: clojars-deploy
filters:
branches:
only: development
requires:
- tools/setup
- tools/format
- tools/unittest
- tools/cljstest
- tools/build
- tools/deploy-release:
jarname: "incognito"
context: clojars-deploy
filters:
branches:
only: master
requires:
- tools/setup
- tools/format
- tools/unittest
- tools/cljstest
- tools/build
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
/target
/classes
/checkouts
pom.xml
pom.xml.asc
*.jar
*.class
/.lein-*
/.nrepl-port
.hgignore
.hg/
.cpcache
.shadow-cljs
node_modules
resources
out/
76 changes: 36 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,57 @@
# incognito

<p align="center">
<a href="https://clojurians.slack.com/archives/CB7GJAN0L"><img src="https://img.shields.io/badge/clojurians%20slack-join%20channel-blueviolet"/></a>
<a href="https://clojars.org/io.replikativ/incognito"> <img src="https://img.shields.io/clojars/v/io.replikativ/incognito.svg" /></a>
<a href="https://circleci.com/gh/replikativ/incognito"><img src="https://circleci.com/gh/replikativ/incognito.svg?style=shield"/></a>
<a href="https://github.com/replikativ/incognito/tree/development"><img src="https://img.shields.io/github/last-commit/replikativ/incognito/development"/></a>
<a href="https://versions.deps.co/replikativ/incognito" title="Dependencies Status"><img src="https://versions.deps.co/replikativ/incognito/status.svg" /></a>
</p>

Different Clojure(Script) serialization protocols like `edn`, `fressian` or
`transit` offer different ways to serialize custom types. In general
they fall back to maps for unknown record types, which is a reasonable
default in many situations. But when you build a distributed data
management system parts of your system might not care about the record
types while others do. This library safely wraps unknown record types
and therefore allows to unwrap them later. It also unifies record
serialization between `fressian` and `transit` as long as you can
express your serialization format in Clojure's default datastructures.

The general idea is that most custom Clojure datatypes (either records or
deftypes) can be expressed in Clojure datastructures if you do not need a custom
binary format, e.g. for efficiency or performance. With incognito you do not
need custom handlers for every serialization format. But you can still provide
them of course, once you hit efficiency problems. Incognito is at the moment not
`transit` offer different ways to serialize custom types. In general they fall
back to maps for unknown Clojure record types, which is a reasonable default in
many situations. But when you build a distributed data management system parts
of your system might not care about the record types while others do. This
library safely wraps unknown record types and therefore allows to unwrap them
later. It also unifies record serialization between `fressian` and `transit` as
long as you can express your serialization format in Clojure's default
datastructures.

The general idea is that most custom Clojure datatypes (in particular records)
can be expressed in Clojure datastructures if you do not need a custom binary
format, e.g. for efficiency or performance. With incognito you do not need
custom handlers for every serialization format. But you can still provide them
of course, once you hit efficiency problems. Incognito is at the moment not
supposed to provide serialization directly to storage, so you have to be able to
serialize your custom types in memory.

Incognito falls back to a `pr-str->read-string` roundtrip which is a reasonable,
but inefficient and will only work if the type has proper Clojure print+read
support.
## Normalization

All types are normalized between Clojure and ClojureScript, where for
ClojureScript the namespace separator `/` is replaced by `.` and `-` by `_`.
This means you need to define your handlers with `.` and `_` on all runtimes, an example is shown below.

We use it for instance to carry the custom type of
a [datascript db](https://github.com/tonsky/datascript)
in [topiq](https://github.com/replikativ/topiq). You don't need to provide a
write handler for incognito except for efficiency reasons or if your type is not
pr-strable (in which case you should make it then first).

## Usage

Add this to your project dependencies:
[![Clojars Project](http://clojars.org/io.replikativ/incognito/latest-version.svg)](http://clojars.org/io.replikativ/incognito)

Include all serialization libraries you need, e.g. for edn support only:
Include all serialization libraries you need:
```clojure
[org.clojure/data.fressian com.cognitect/transit-clj "0.8.297"]
[io.replikativ/incognito "0.2.5"]
[io.replikativ/incognito "0.2.7"]
```

In general you can control serialization by `write-handlers` and `read-handlers`:
In general you can control serialization by `write-handlers` and
`read-handlers`,

```clojure
(defrecord Bar [a b])

(def write-handlers (atom {'user.Bar (fn [bar] bar)}))
(def read-handlers (atom {'user.Bar map->Bar}))
(def write-handlers (atom {'my_user.ns.Bar (fn [bar] bar)}))
(def read-handlers (atom {'my_user.ns.Bar map->Bar}))
```

To handle custom non-record types you have to transform it into a readable
Expand All @@ -69,7 +75,7 @@ writer:
{:tag 'org.joda.time.DateTime, :value "2017-04-17T13:11:29.977Z"})

```
*NOTE*: The syntax quote for the read handler is necessary so you can
*NOTE*: The syntax quote for the read handler is necessary such that you can
deserialize unknown classes.

A write-handler has to return an associative datastructure which is
Expand All @@ -83,7 +89,7 @@ can wrap the serialization for its own handlers.

(Extracted from the tests):

### edn
### edn strings

```clojure
(require '[incognito.edn :refer [read-string-safe]])
Expand Down Expand Up @@ -142,16 +148,6 @@ can wrap the serialization for its own handlers.
fress/associative-lookup)))))))
```

## ClojureScript

For dashed namespace names you need a custom printer to be
ClojureScript conform.

```clojure
(defmethod print-method some_namespace.Bar [v ^java.io.Writer w]
(.write w (str "#some-namespace.Bar" (into {} v))))
```


## TODO

Expand All @@ -173,7 +169,7 @@ ClojureScript conform.

## License

Copyright © 2015-2020 Christian Weilbach, Ferdinand Kühne
Copyright © 2015-2021 Christian Weilbach, Ferdinand Kühne

Distributed under the Eclipse Public License either version 1.0 or (at
your option) any later version.
4 changes: 4 additions & 0 deletions bin/kaocha
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/usr/bin/env bash

clojure -A:test -m kaocha.runner "$@"

10 changes: 10 additions & 0 deletions bin/run-cljstests
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash

set -o errexit
set -o pipefail

# Compile for browser
npm run ci-test

# Compile for node
npx shadow-cljs compile node-test
3 changes: 3 additions & 0 deletions bin/run-unittests
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env bash

TIMBRE_LEVEL=':warn' clojure -A:test -m kaocha.runner
18 changes: 18 additions & 0 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/clojurescript {:mvn/version "1.10.764"}
org.clojure/data.fressian {:mvn/version "0.2.1"}
com.cognitect/transit-clj {:mvn/version "0.8.319"}
com.cognitect/transit-cljs {:mvn/version "0.8.256"}
fress/fress {:mvn/version "0.3.2"}}
:aliases {:cljs {:extra-deps {thheller/shadow-cljs {:mvn/version "2.10.21"}
binaryage/devtools {:mvn/version "1.0.2"}}
:extra-paths ["test"]}
:test {:extra-deps {lambdaisland/kaocha {:mvn/version "1.0.641"}}
:extra-paths ["test"]}
:jar {:extra-deps {seancorfield/depstar {:mvn/version "1.1.116"}}
:main-opts ["-m" "hf.depstar.jar" "replikativ-incognito.jar"]}
:deploy {:extra-deps {slipset/deps-deploy {:mvn/version "0.1.0"}}
:main-opts ["-m" "deps-deploy.deps-deploy" "deploy" "replikativ-incognito.jar"]}
:format {:extra-deps {cljfmt/cljfmt {:mvn/version "0.7.0"}}
:main-opts ["-m" "cljfmt.main" "check"]}}}
17 changes: 17 additions & 0 deletions karma.conf.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
module.exports = function (config) {
config.set({
browsers: ['ChromeHeadless'],
// The directory where the output file lives
basePath: 'target',
// The file itself
files: ['ci.js'],
frameworks: ['cljs-test'],
plugins: ['karma-cljs-test', 'karma-chrome-launcher'],
colors: true,
logLevel: config.LOG_INFO,
client: {
args: ["shadow.test.karma.init"],
singleRun: true
}
})
}
Loading

0 comments on commit 80a09f4

Please sign in to comment.