-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider allowing Exception instances to be serialized without requiring BinaryFormatter #43482
Comments
For those who don't know what some TLAs mean, like me, I think SDL refers to Microsoft Security Development Lifecycle. |
What would happen in the case where the deserializing assembly/service/app/whatever does not know about or have access to the appropriate |
The case of "not having access" shouldn't be possible with this design. The deserializer needs to have ahead-of-time knowledge about all possible This means that deserialization scenarios fall into only two buckets: (1) the deserializer is aware of the requested In the case of the second bucket above, I imagine the deserializer would instantiate a placeholder |
It would be nice if the new exposed something like .NET Framework's |
That still relies on the Remoting infrastructure doing the serialization, and has been somewhat generalized with Perhaps being able to serialize a ExceptionDispatchInfo would solve this problem? |
@yaakov-h Thats a good point. EDI gets you most of the way there. All that |
Speaking as a third party serializer author, my main issue with the existing design is that the
Assuming the highly experimental abstract static interface methods ever gets adopted, one could conceive of a type safe and linker-friendly successor to SerializationInfo/StreamingContext constructors. This is largely how languages with type classes implement type-safe serialization. |
A 100% faithful reconstruction of the original |
Fully aggree with the fact that an actual exception is not required. For years we used a simple stupid projection into a "Data" object (factory method is here; https://github.com/Invenietis/CK-Core/blob/master/CK.Core/CKExceptionData.cs#L143). This data is "as serializable as possible" and its only data. |
Agreed, although that should be a design decision preferably made by the serializer implementation, rather than something forced by the underlying exception reconstruction mechanism. |
I am surprised that the requirements in the description do not include a provision for dropping sensitive information from the exception so it's safe for remote parties with an Information Disclosure risk. |
@AArnott I don't think that's a viable requirement. In general it's a bad idea to send any object that contains privileged information through a serializer, as there's too high a risk that the sensitive information might be transmitted. This also runs the risk that for any given Exception object, sensitive information may or may not be disclosed depending on the exact Exception type, which isn't always under the dev's control. This makes the system very difficult to reason about from a security perspective. Any threat model of this serializer would absolutely include information disclosure as a threat. But I suspect the answer to that will be "this serializer is not intended for use in environments where the recipient is not trustworthy." |
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Move to runtime, this is not a serialization issue. |
We just had a discussion about this in the context of dapr/dotnet-sdk#414 Actor frameworks (or other RPC frameworks) tend to use exceptions of different types to signal different business concerns. There might be a different exception type for a transaction that's rejected for low balance or rejected because the item is out of stock, etc. Ultimately application code wants to call some functionality on another server and then interact with the result in a strongly-typed way, likely with multiple catch blocks. For this use case the majority of exceptions would be user-defined. Ideally a solution would support serializing additional user-defined properties as well (can be opt-in). |
|
Agreed, a composition model using constructors is the only feasible scheme for rehydrating exceptions. A minimalist solution might involve us declaring that serializable exceptions are the ones that expose a constructor accepting a message and an inner exception (which many types already have): stack traces can then be set out of band using |
What about derived Exception properties, like |
We'd want to avoid using something akin to |
This is highly opinionated solution that is unlikely to work well in many cases. It can be a sample, but I do not think we would want to ship it as a package from dotnet/runtime. |
I disagree with this. I have code that serializes and deserializes arbitrary Exception types using the ISerializable interface they implement. I would even argue I do it securely. I don't use BinaryFormatter at all.
This is almost what I do (in StreamJsonRpc). Except I didn't have to replace ISerializable. I used it as-is. It works great. I'm happy to provide details.
What we do in StreamJsonRpc is just that, IIRC. Any complex object from the Data dictionary is simply serialized as a dictionary of primitives, and therefore they get deserialized as a dictionary of primitives. Perfectly safe that way, while retaining the data. The only arbitrary object we even instantiate in this system are Exception-derived types. We made the decision to assume that Exception types are generally safe to instantiate (e.g. assume that none of them have harmful finalizers or other behavior). If that turns out to be false, we have a disallow-list that can block it. We also allow for exceptions that don't conform to the ISerializable patterns. In either case (blocked or non-conformant) or if we simply cannot find the exception type in the deserializer, we simply deserialize as System.Exception instead of its derived type. |
SqlException has already adjusted themself to the new world with non-BF serializers like StreamJsonRpc: IMO this is trying to fix a problem that the community has already discovered/accepted/worked around (SqlException, StreamJsonRpc, others) Maybe all that is needed is updating the documentation/guidelines to how it is used "De facto". |
I mean this in the sense that but it emits 3 separate warnings which gives a strong signal to users that they should be moving away from it. The problem is, there is currently no general-purpose alternative.
Which should work as long as the exception type doesn't attempt to downcast to a specific type. It sounds like it might also have problems in the context of trimmed apps or AOT since the serializer can't know the type of the underlying objects at compile time. I think your approach is a step in the right direction, however we need to close the loop so that exception authors also design their types around such APIs that make serialization secure by construction. Only runtime libraries are able to effect this type of change.
I recall @GrabYourPitchforks mentioning that the shared framework does contain exception types whose |
No it isn't.
IMO that's a flaw in how we obsoleted APIs. Folks were over-eager to obsolete all APIs used with BinaryFormatter instead of obsoleting only APIs directly and solely used with BinaryFormatter.
That's true. Although in all our use with StreamJsonRpc (which is substantial, but tiny on a .NET scale), we've never encountered such a failure.
True. Although this could be resolved using something like
Can you elaborate on this? Why is it insecure by design? In fact ISerializable is the only pattern I've ever seen for serializers where the intended use is explicitly given to the serializing/deserializing code so it can consider security matters.
Agreed. That's why StreamJsonRpc doesn't deserialize such objects except as harmless dictionaries of primitives. |
I mean sure, it technically sits in a parent namespace and the same can be said of
Because if used as designed
Right, but like I said this cannot offer complete support that works in every platform. Maybe if you're an application author that has tight control over what exception types can and cannot be marshalled this works fine, but the calculus changes if you're a library or framework author. I think we're pretty much on the same page with respect to what is a safe way to serialize exceptions, I'm suggesting we take this one step forward and let exception type authors use the same safe-by-construction APIs in a way that guarantees they are always round-tripable. |
ISerializable-based serialization implementations have never been hardened against malicious data. Hardening against invalid data was not part of ISerializable-based contract. Many BF exploits took advantage of that. |
If you are library or framework author, you can provide custom converters that target your serializations scheme for built-in set of exceptions and allow people register their own converters on top of that. |
True, but that just means more ad-hoc and mutually inconsistent solutions. In practice the path of least resistance is falling back to |
How are you going to ensure that this is secure? As I have said above, ISerializable-based serialization implementations have never been hardened against malicious data. |
We can't. I'm saying this is the de facto standard way for exception serialization in the ecosystem and that it might be in our interest to provide a secure alternative. |
I'm curious if there's a single concrete example of an
If we have an example of an exploitable vulnerability in these conditions, I'd sympathize perhaps with simply not trusting Exceptions pre-existing ISerializable implementations. But if we can't find one, I'm afraid we're throwing the baby out with the bathwater. |
ISerializable forces a trade-off between being vulnerable and being feature-complete. Consider the following example that isn't entirely unheard of: class MyCoolException : Exception, ISerializable
{
private readonly SomeOpaqueType _data;
public MyCoolException(string message) : base(message)
{
_data = new SomeOpaqueType(message);
}
public MyCoolException(SerializationInfo info, StreamingContext context) : base(info, context)
{
_data = (SomeOpaqueType)info.GetValue("Data", typeof(SomeOpaqueType));
}
void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Data", _data);
}
record SomeOpaqueType(string value);
} Because the deserialization constructor expects the entry for This is precisely what creates the conditions for remote code execution, as an imaginative attacker can craft payloads using readily available types from the shared framework that end up spawning calc.exe as a side-effect. ysoserial.net contains a lot of examples showing how this can be achieved and even comes with a malicious payload generator for a host of vulnerable serializers. |
Can the provider implement a System.Runtime.Serialization.IFormatterConverter whose |
I think that might be possible, yes. Although I think it would still be challenging to support in AOT. |
I don't understand why this must be the case. info.GetValue("Data", typeof(SomeOpaqueType)); That right there provides the type that is expected to be deserialized. That isn't data-driven -- that's right there in compiled code. StreamJsonRpc can use that to deserialize "Data", allowing the cast to succeed.
Yes. That is exactly what StreamJsonRpc does. Essentially we defer deserialization of certain fragments until we know what type was expected -- not by the data but by the trusted code itself. As for AOT, yes this requires reflection (or at least some hints as to which types should be prepared for deserialization). AOT represents a very small portion of all .NET code, so while it's great that it's support is growing, I'm not sure we should throw out a system that gives us support to serialize nearly all exceptions for something that will only support a small subset of exceptions after changes are made to them in order to support AOT which is also a tiny subset. If we can agree that this ISerializable approach is adequate for exception serialization, we can move on to figuring out the best way to make this shine in AOT environments too. |
I do not think that ISerializable approach can be the base for safe exception serialization. You cannot make an assumption that BF deserialization constructor of every exception type out there is hardened for untrusted input. The problem exists even when the payload is limited to primitive types. You can have:
Of course, the code is not going to be written like this, the actual logic would be more complicated, but the net effect can be the same. |
I don't think anybody proposed we remove
Would it be possible to build some of the
@jkotas sure, but doesn't this type of risk also exist in modern serializers? I can easily see the same type of problem occurring in a user-defined |
Fair enough. But that isn't a BF-specific vulnerability at that point. It sounds like you're making the case that "all existing deserializers may have some bug somewhere that could be exploited, so we should rewrite all of them" but I see no reason to believe that any rewrite would be anything other than as-likely to have security bugs like what you suggest here.
What I'm concerned about is not that the interface may be removed, but rather that exception types defined in .NET will stop implementing it, or start throwing from their implementations. And that future exception types defined will not support it. And that the guidance is for 3rd parties to stop implementing such interfaces and deserializing constructors. That will undermine the goodness that we have with StreamJsonRpc.
I still am not seeing this. I don't see that StreamJsonRpc jumped through a lot of hoops. We just implemented it cautiously. And it was a relatively small amount of code. |
Custom converters for modern deserializers are expected to be written against modern security requirements. It includes things like explicit allow-lists of types that have been hardened for untrusted input. If you see a type that implements ISerialiable, it is impossible to tell whether the ISerializable implementation has been hardened against untrusted input.
This guidance has been non-existent for ISerializable-based contracts. I do not think that it is acceptable to apply it retroactively.
I am not convinced that the StreamJsonRpc scheme is secure. The scheme is probably acceptable for closed system like VS cross-process communication where actively malicious input is less of a concern. I do not think it would be acceptable for general use for arbitrary end points that have to be hardened against actively malicious input. |
The point I'm trying to make is that unless you are a serialization expert that is keenly aware of the security ramifications, you will most likely end up implementing things using the readily available, insecure way (a.k.a. passing in
I think it stands to reason that we only recommend exception deserialization for applications not expected to handle untrusted input. This is true for the aforementioned orleans and akka.net that essentially implement remoting across distributed compute clusters. |
The BF security guide explains the risks of such assumptions: https://learn.microsoft.com/en-us/dotnet/standard/serialization/binaryformatter-security-guide#the-risks-of-assuming-data-to-be-trustworthy . |
Sure, I'm just pointing out that these systems are designed to accept requests that execute remote code and that should factor into their threat modelling. |
The documentation we create for this needs to also incorporate the questions and scenarios from #101159, which I just closed as a duplicate of this one. |
Some remoting technologies use
BinaryFormatter
to serializeException
instances across security boundaries, which puts them out of SDL compliance and potentially exposes consumers to security vulnerabilities. The current recommended way to serialize exception information safely is to callToString
on the exception instance, then transmit the resulting string across the wire. However, this does not create useful object models for consuming applications, as they can't interact with a simple string like they can a rich exception instance (accessing properties, using try / catch, etc.).In an ideal world, an exception serialization tech would have the following characteristics:
Exception
-derived type will be instantiated after deserialization.ArgumentOutOfRangeException
.To maintain SDL compliance and work with our linker technology, we'd need to enforce a few extra behaviors:
Exception
-derived types, and the payload cannot attempt to instantiate types outside that allowed listSerializationInfo
/StreamingContext
infrastructure.SerializationInfo
/StreamingContext
infrastructure.It's possible that the deserialization tech would need to include special-case handling of each allowed
Exception
-derived type in order to fulfill these requirements. Perhaps this could be simplified by understanding canonical patterns like.ctor(string message, Exception innerException)
. But we'll cross that bridge when we come to it.The text was updated successfully, but these errors were encountered: