Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide way to encode unquoted String in JSON format #1405

Closed
sandwwraith opened this issue Apr 5, 2021 · 12 comments
Closed

Provide way to encode unquoted String in JSON format #1405

sandwwraith opened this issue Apr 5, 2021 · 12 comments

Comments

@sandwwraith
Copy link
Member

Mainly to write big numbers. See reasoning here: #1051 (comment)

@JakeWharton
Copy link
Contributor

Another use case is embedding known JSON documents into others without having to go through an intermediate parsing step. As long as it's named sufficiently scary this can be a powerful escape hatch.

@nikolaykasyanov
Copy link

nikolaykasyanov commented Jun 14, 2021

Should this issue also include support for decoding JSON primitives as raw strings (to be able to convert them to smth like mentioned BigDecimal)?

@Kantis
Copy link
Contributor

Kantis commented Aug 24, 2021

Do you see any downsides to this approach? I suppose it might be prone to breaking, but it seems to work at least. Obviously it should only be used when working with json as serial format.

⚠️ See DrMetallius' comment below, this only works when deserializing. When serializing, the underlying implementation will still parse the string into a Double

object BigDecimalSerializer : KSerializer<BigDecimal> {

    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("BigDecimal", PrimitiveKind.STRING)

    override fun deserialize(decoder: Decoder): BigDecimal =
        (decoder as JsonDecoder).decodeJsonElement().jsonPrimitive.content.toBigDecimal()


    override fun serialize(encoder: Encoder, value: BigDecimal) {
        (encoder as JsonEncoder).encodeJsonElement(JsonPrimitive(JsonBigDecimal(value)))
    }
  
    private class JsonBigDecimal(value: BigDecimal) : Number() {
    
        private val serialized: String = value.toString()
    
        override fun toString() = serialized
    
        override fun toByte() = notSupported()
        override fun toChar() = notSupported()
        override fun toDouble() = notSupported()
        override fun toFloat() = notSupported()
        override fun toInt() = notSupported()
        override fun toLong() = notSupported()
        override fun toShort() = notSupported()
    
        private fun notSupported(): Nothing = throw IllegalStateException("This method should not be invoked, ever.")
    }
}

@nikolaykasyanov
Copy link

@Kantis thanks for sharing, seems to be working great!

@DrMetallius
Copy link

This solution unfortunately only works only for decoding. When encoding, the value will get converted to double by JsonLiteralSerializer.

@Kantis
Copy link
Contributor

Kantis commented Aug 31, 2021

This solution unfortunately only works only for decoding. When encoding, the value will get converted to double by JsonLiteralSerializer.

Darn. I accidentally asserted my expected too precise for double number through a JSON parser which ended up doing the same double-conversion.

Added a warning to my original comment and code snippet.

@nikolaykasyanov
Copy link

Thanks for sharing @DrMetallius, good to know the limitations.

@sandwwraith looks like there's no viable workaround for serialization, would it be possible to proiritize the issue?

@sandwwraith
Copy link
Member Author

I think we'll include this in plans for the next release

@nikolaykasyanov
Copy link

@sandwwraith that's great to hear, thank you 🙏

@vivekmalhotragithub
Copy link

any updates on this issue?

fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 12, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 13, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 13, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 13, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 13, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 13, 2022
The server does return vulnerabilities which do not have a severity
value in the dedicated property. The unspecified `databaseSpecific`
property often times holds a primitive `severity` property with values
such as `[HIGH, MEDIUM, LOW]`. Make use of these values as a fallback as
these to provide more indication than a `null` value.

Note: The data model of 'osv/client' currently uses subtypes of
JsonElement to expose a couple of unspecified JSON objects as
properties. Accessing these requires the client code to add
'kotlinx.serialization' as dependency which is not nice. A solution to
that would be to use "raw" string values containing the JSON, which is
unfortunately not yet possible but may become so in the future, see
[1][2][3].

So, for now add 'kotlinx.serialization' as dependency to the advisor in
order to access the property and leave a FIXME comment as reminder.

[1] Kotlin/kotlinx.serialization#1298
[2] Kotlin/kotlinx.serialization#1405
[3] Kotlin/kotlinx.serialization#1058

Signed-off-by: Frank Viernau <frank_viernau@epam.com>
@nikolaykasyanov
Copy link

nikolaykasyanov commented Sep 22, 2022

@sandwwraith 👋 hello, I wonder if this is still planned.

@aSemy
Copy link
Contributor

aSemy commented Oct 22, 2022

@nikolaykasyanov encoding raw strings to JSON was completed in #2041, and it will be available in a month or two.

fred01 pushed a commit to fred01/kotlinx.serialization that referenced this issue Nov 24, 2022
This PR provides a new function for encoding raw JSON content, without quoting it as a string. This allows for encoding JSON numbers of any size or precision, so BigDecimal and BigInteger can be supported.

Fixes Kotlin#1051
Fixes Kotlin#1405

The implementation is similar to how unsigned numbers are handled.

JsonUnquotedLiteral() is a new function that allows creating literal JSON content.
Added val coerceToInlineType to JsonLiteral, so that JsonUnquotedLiteral could use encodeInline()
Defined val jsonUnquotedLiteralDescriptor as a 'marker', for use with encodeInline()
ComposerForUnquotedLiterals (based on ComposerForUnsignedNumbers) will 'override' the encoder when a JsonLiteral has the jsonUnquotedLiteralDescriptor marker, and will encode the content as a string without surrounding quotes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants