RFC: Token stream

[phadej]

While working on the sax-like stream based JSON parser, I noticed that parsing Value with it is faster than with attoparsec. It started as a separate library, but as it makes isolated (and quite internal) part of aeson a bit faster, I made this PR. If you think this doesn't below into aeson, I'm ok with making it into own library (exposing few internals jsstring_ and scientific would be useful though).

Notes:

• The token stream doesn't contain commas , or colons : as they are redundant, at makes things a bit more straightforward.
• Variant with [Token] seems to be faster then fusing the structures rogether. The decoder variant as in https://github.com/well-typed/binary-serialise-cbor/blob/0236307ba953d8d2a8351455587e4c12b813397d/cborg/src/Codec/CBOR/Decoding.hs could be an improvement too (we could have different decoder actions to read number as a bytestring/text for example. I should try and benchmark that variant too.

Differences:

• slightly better error reporting: Uninformative invalid JSON error #558

> decodeValue "{ 1 "
Left "Expecting record key or '}', got token TkError \"Expecting record key or '}', got 1 \""

• slightly better performance
• cannot separate prime-less and primeless versions (decode & decode')

Further work

• Write version consuming strict ByteString.
• Should this be default or not?
• It's possible to make a variant of TokenStream which rules out invalid streams. I think it's possible to have an encoding such for TokenStream without TkError (which can be enforced with a GADT parameter, but I don't see a need to do so), one could write total TokenStream -> Value function.
  This would require GADTs,, DataKinds and TypeFamilies. This encoding might be useful for hand-written "fancy" TokenStream consumers.
• Build a parser consuming TokenStream (the list version). First iteration could be done using parsec. Probably for little more performance, one would need specialized version at the end.
  • parsing records a little problem:
    - tag-encoding of sum (one would need to retain the pointer to the point where object tokens started). Similarly monadic parsers are problematic
    - record's unordered nature is a problem too, but not impossible
  • as a con, we can avoid building hashmaps and vectors when we don't need to (e.g. parsing Map!)
  • makes possible to write parsers taking advantage of the order of keys in JSON!
  • makes it simpler to write record parser disallowing keys (will fail earlier)
  • skipping parser is trivial (decode and decode' separating might not be needed!)

Benchmarks

my machine is surprisingly noisy right now:

benchmarking decode/en/aeson/lazy
time                 1.832 ms   (1.799 ms .. 1.865 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 1.817 ms   (1.803 ms .. 1.839 ms)
std dev              57.44 μs   (40.03 μs .. 85.58 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking decode/en/aeson/strict
time                 1.819 ms   (1.800 ms .. 1.839 ms)
                     0.998 R²   (0.998 R² .. 0.999 R²)
mean                 1.855 ms   (1.835 ms .. 1.887 ms)
std dev              77.86 μs   (52.22 μs .. 110.1 μs)
variance introduced by outliers: 28% (moderately inflated)

benchmarking decode/en/aeson/stricter
time                 1.947 ms   (1.927 ms .. 1.967 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.956 ms   (1.944 ms .. 1.975 ms)
std dev              51.37 μs   (32.60 μs .. 78.53 μs)
variance introduced by outliers: 13% (moderately inflated)

benchmarking decode/en/aeson/hackage
time                 2.109 ms   (2.079 ms .. 2.142 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.129 ms   (2.112 ms .. 2.159 ms)
std dev              73.19 μs   (45.28 μs .. 119.2 μs)
variance introduced by outliers: 20% (moderately inflated)

benchmarking decode/en/aeson/hackage'
time                 1.921 ms   (1.905 ms .. 1.931 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 1.942 ms   (1.934 ms .. 1.993 ms)
std dev              48.40 μs   (11.99 μs .. 120.8 μs)
variance introduced by outliers: 11% (moderately inflated)

benchmarking decode/en/aeson/parser
time                 2.092 ms   (2.069 ms .. 2.119 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.109 ms   (2.095 ms .. 2.135 ms)
std dev              60.11 μs   (34.63 μs .. 93.45 μs)
variance introduced by outliers: 15% (moderately inflated)

benchmarking decode/en/aeson/stream
time                 1.983 ms   (1.803 ms .. 2.198 ms)
                     0.966 R²   (0.945 R² .. 0.999 R²)
mean                 1.857 ms   (1.828 ms .. 1.936 ms)
std dev              144.3 μs   (67.70 μs .. 300.8 μs)
variance introduced by outliers: 57% (severely inflated)

benchmarking decode/en/json
time                 4.208 ms   (2.040 ms .. 6.774 ms)
                     0.194 R²   (0.044 R² .. 0.421 R²)
mean                 10.07 ms   (8.596 ms .. 12.12 ms)
std dev              4.890 ms   (3.838 ms .. 6.278 ms)
variance introduced by outliers: 97% (severely inflated)

Legend (numbers in parentheses point which benches are virtually the same)

• lazy: decode using stream (1)
• strict: decode' using stream
• stricter: decodeStrict using attoparsec (3)
• hackage: decode using attoparsec from aeson 1.2.1.0 (cffi: false) (2)
• hackage' decodeStrict as previous (3)
• parser: decodeWith from Parser (2)
• stream: decodeWith from Stream (1)
• json: using http://hackage.haskell.org/package/json

[phadej]

Forget this, I accidentally enabled cffi: true.

---

Added few bang patterns, now the speedup is noticeable (json-stream has C code, aeson is pure haskell, cffi: false)

benchmarking decode/en/aeson/lazy
time                 1.137 ms   (1.124 ms .. 1.150 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.144 ms   (1.135 ms .. 1.156 ms)
std dev              33.15 μs   (23.02 μs .. 47.10 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking decode/en/aeson/strict
time                 1.156 ms   (1.147 ms .. 1.167 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.165 ms   (1.157 ms .. 1.183 ms)
std dev              38.79 μs   (21.75 μs .. 67.99 μs)
variance introduced by outliers: 22% (moderately inflated)

benchmarking decode/en/aeson/stricter
time                 1.291 ms   (1.278 ms .. 1.302 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 1.294 ms   (1.286 ms .. 1.312 ms)
std dev              38.92 μs   (21.37 μs .. 66.90 μs)
variance introduced by outliers: 19% (moderately inflated)

benchmarking decode/en/aeson/hackage
time                 2.079 ms   (2.053 ms .. 2.105 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 2.101 ms   (2.081 ms .. 2.155 ms)
std dev              108.0 μs   (46.01 μs .. 207.6 μs)
variance introduced by outliers: 36% (moderately inflated)

benchmarking decode/en/aeson/hackage'
time                 1.906 ms   (1.893 ms .. 1.915 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 1.923 ms   (1.915 ms .. 1.951 ms)
std dev              43.34 μs   (8.642 μs .. 90.59 μs)

benchmarking decode/en/aeson/parser
time                 1.470 ms   (1.451 ms .. 1.490 ms)
                     0.998 R²   (0.997 R² .. 0.999 R²)
mean                 1.475 ms   (1.465 ms .. 1.497 ms)
std dev              47.67 μs   (29.14 μs .. 81.88 μs)
variance introduced by outliers: 20% (moderately inflated)

benchmarking decode/en/aeson/stream
time                 1.244 ms   (1.189 ms .. 1.314 ms)
                     0.985 R²   (0.981 R² .. 0.994 R²)
mean                 1.200 ms   (1.180 ms .. 1.227 ms)
std dev              78.37 μs   (52.47 μs .. 103.4 μs)
variance introduced by outliers: 52% (severely inflated)

benchmarking decode/en/json-stream
time                 1.244 ms   (1.234 ms .. 1.251 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 1.259 ms   (1.253 ms .. 1.276 ms)
std dev              30.39 μs   (8.533 μs .. 57.82 μs)
variance introduced by outliers: 13% (moderately inflated)

benchmarking decode/en/json
time                 4.803 ms   (4.737 ms .. 4.862 ms)
                     0.999 R²   (0.998 R² .. 0.999 R²)
mean                 4.821 ms   (4.796 ms .. 4.868 ms)
std dev              99.23 μs   (67.59 μs .. 155.7 μs)

[phadej]

TokenStream approach is useful for stream things, maximum residency of stream processing is smaller by magnitude (while processing only 438k file)

The aeson-count-laureates benchmark is too manual, but that's further work to make high-level parsing via TokenStream

./aeson-count-laureates tokenStream +RTS -s
910
      50,834,544 bytes allocated in the heap
          32,376 bytes copied during GC
          85,440 bytes maximum residency (3 sample(s))
          92,992 bytes maximum slop
               2 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0        94 colls,     0 par    0.000s   0.000s     0.0000s    0.0000s
  Gen  1         3 colls,     0 par    0.000s   0.000s     0.0001s    0.0002s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.008s  (  0.009s elapsed)
  GC      time    0.000s  (  0.000s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.044s  (  0.009s elapsed)

  %GC     time       0.0%  (5.1% elapsed)

  Alloc rate    6,354,318,000 bytes per MUT second

  Productivity 100.0% of total user, 94.2% of total elapsed

./aeson-count-laureates parseJSON +RTS -s
910
      53,038,280 bytes allocated in the heap
      12,378,080 bytes copied during GC
       3,034,968 bytes maximum residency (5 sample(s))
          69,280 bytes maximum slop
               7 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0        96 colls,     0 par    0.004s   0.004s     0.0000s    0.0001s
  Gen  1         5 colls,     0 par    0.004s   0.005s     0.0010s    0.0024s

  INIT    time    0.000s  (  0.000s elapsed)
  MUT     time    0.008s  (  0.009s elapsed)
  GC      time    0.008s  (  0.009s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    0.052s  (  0.018s elapsed)

  %GC     time      15.4%  (49.1% elapsed)

  Alloc rate    6,629,785,000 bytes per MUT second

  Productivity  84.6% of total user, 50.6% of total elapsed

[phadej]

For fancy encoding of token stream see https://gist.github.com/phadej/31bdb72c815766edde1eaf3efe8cf0ff, that would allow making safe skip as in aeson-count-laureates benchmark.

I think it's a good exercise for type-hackery, but aeson isn't a good place for it.

[phadej]

turning [TokenParser] into strict list (both elements and spine) doesn't seem to have effect on performance.

[awalterschulze]

I would like to use this order preserving decoder in aeson or as a separate library for my encoding agnostic validation language and use TokenStream to create an instance of Tree.

data Label
    = String Text
    | Number Scientific
    | Bool Bool

class Tree a where
    getLabel :: a -> Label
    getChildren :: a -> [a]

But maybe this isn't the right place for this comment. I just want to say that I am looking forward to using this.

[phadej]

Superseded by #996