JVM integration with InputStream and OutputStream

[sandwwraith]

No description provided.

[qwwdfsad]

I haven't got a deep look into decoding yet, but it seems that there is already enough actionable points here

[qwwdfsad]

It seems to be pretty huge allocation (e.g. it always misses TLAB). Could you please ensure it doesn't dominate small objects serialization?

If so, it's worth either reducing its size or pool a few instances of this

[chris-hatton]

Question regarding the behaviour of this change:
Will decodeFromStream behave so that I can call it serially (repeatedly, non-overlapping) against an open stream, each time consuming only as much of the stream as is necessary to form a complete object?

For example; given this stream carrying two distinct JSON objects:

{"someKey":"someValue1"}{"someKey":"someValue2"}

Could I call decodeFromStream() twice, to get both objects?
This is the characteristic I am looking for, to be able to read a Flow<T> from a long-lived HTTP response stream.
Thanks for your efforts @sandwwraith 🙏 This is a hotly awaited improvement.

[BenWoodworth]

I'm curious, was using Reader/Writers considered (instead of Streams)? That would've been my first thought for JSON/String formats, and avoids dealing with character encodings.

[sandwwraith]

@BenWoodworth Reader/Writer is used internally. API provides methods with Input/OutputStreams because it's more versatile and allows to implement charset-specific parsers in the future

@chris-hatton Yes, I think we can do it — I'll add a test for it

[qwwdfsad]

I've dug through the profile and it seems that JVM cannot optimize string access across CharSequence interface properly, especially in small hot methods. definitelyNotEof is also a heavy-hitter for such functions.

I've tried to tweak it here and there, but it's quite hard to ensure all the invariants with the existing limitations.

I'd suggest you do the following:

Get the base JsonLexer with the only state -- currentPosition and utility functions for slow-paths: skipElement, various fail functions, maybe boolean/numbers consumption (char sequence is an input parameter of a function then). Everything else is copy-pasted between streaming and string implementations.

At this moment, the performance model is quite clear and expected degradation should be insignificant (educated guess -- 2-4%).
Then you can start commonizing (handling via CharSequence interface in the base json lexer) the parts of parsing where the compiler is smart enough to optimize everything away.

I expect that the biggest offenders (things you cannot commonize) will be just a few functions that were written in a compact and polished manner -- skipWhitespaces, tryConsumeComma, canConsumeValue and peekNextToken. Everything else will probably be working well via CharSequence and the amount of duplicated code we have to maintain will be quite isolated

[qwwdfsad]

Good to go 🚀

Please don't forget to file issues for future improvements -- UTF-8 parsing and multishot streams

[qwwdfsad]

Redundant dot

[qwwdfsad]

[nit] it's just flush :)

[sandwwraith]

#1662

[slavonnet]

String. Serializer work ok on big JSon. Stream get random.eof exception I sinppe replace Strung Encoder t to Stream Encoder in converter factory . Small size json is ok. Buffer rewrite by gzip wtite all