Skip to content

Commit

Permalink
Codec: Add/default to ReadOnlyMemory format instead of byte[] (#75)
Browse files Browse the repository at this point in the history
  • Loading branch information
bartelink authored May 5, 2022
1 parent 6144527 commit b37094c
Show file tree
Hide file tree
Showing 15 changed files with 447 additions and 280 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,18 @@ The `Unreleased` section name is replaced by the expected version of next releas
## [Unreleased]

### Added

- `SystemTextJson.CodecJsonElement`: Maps Unions to/from Events with `JsonElement` Bodies as `SystemTextJson.Codec` did in in `2.x` [#75](https://github.com/jet/FsCodec/pull/75)
- `SystemTextJson.ToUtf8Codec`: Adapter to map from `JsonElement` to `ReadOnlyMemory<byte>` Event Bodies (for interop scenarios; ideally one uses `SystemTextJson.Codec` directly in the first instance) [#75](https://github.com/jet/FsCodec/pull/75)

### Changed

- `NewtonsoftJson`: Rename `Settings` to `Options` [#60](https://github.com/jet/FsCodec/issues/60) [#76](https://github.com/jet/FsCodec/pull/76)
- Updated build and tests to use `net6.0`, all test package dependencies
- Updated `TypeShape` reference to v `10`, triggering min `FSharp.Core` target moving to `4.5.4`
- `SystemTextJson.Codec`: Switched Event body type from `JsonElement` to `ReadOnlyMemory<byte>` [#75](https://github.com/jet/FsCodec/pull/75)
- `NewtonsoftJson.Codec`: Switched Event body type from `byte[]` to `ReadOnlyMemory<byte>` [#75](https://github.com/jet/FsCodec/pull/75)
- `ToByteArrayCodec`: now adapts a `ReadOnlyMemory<byte>` encoder (was from `JsonElement`) (to `byte[]` bodies); Moved from `FsCodec.SystemTextJson` to `FsCodec.Box` [#75](https://github.com/jet/FsCodec/pull/75)

### Removed

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Typically used in [applications](https://github.com/jet/dotnet-templates) levera

## Components

The components within this repository are delivered as multi-targeted Nuget packages supporting `netstandard2.0`/`1` (F# 4.5+) profiles.
The components within this repository are delivered as multi-targeted Nuget packages supporting `netstandard2.1` (F# 4.5+) profiles.

- [![Codec NuGet](https://img.shields.io/nuget/v/FsCodec.svg)](https://www.nuget.org/packages/FsCodec/) `FsCodec` Defines interfaces with trivial implementation helpers.
- No dependencies.
Expand All @@ -19,9 +19,9 @@ The components within this repository are delivered as multi-targeted Nuget pack
- [![Newtonsoft.Json Codec NuGet](https://img.shields.io/nuget/v/FsCodec.NewtonsoftJson.svg)](https://www.nuget.org/packages/FsCodec.NewtonsoftJson/) `FsCodec.NewtonsoftJson`: As described in [a scheme for the serializing Events modelled as an F# Discriminated Union](https://eiriktsarpalis.wordpress.com/2018/10/30/a-contract-pattern-for-schemaless-datastores/), enabled tagging of F# Discriminated Union cases in a versionable manner with low-dependencies using [TypeShape](https://github.com/eiriktsarpalis/TypeShape)'s [`UnionContractEncoder`](https://eiriktsarpalis.wordpress.com/2018/10/30/a-contract-pattern-for-schemaless-datastores)
- Uses the ubiquitous [`Newtonsoft.Json`](https://github.com/JamesNK/Newtonsoft.Json) library to serialize the event bodies.
- Provides relevant Converters for common non-primitive types prevalent in F#
- [depends](https://www.fuget.org/packages/FsCodec.NewtonsoftJson) on `FsCodec`, `Newtonsoft.Json >= 11.0.2`, `TypeShape >= 10`, `Microsoft.IO.RecyclableMemoryStream >= 2.2.0`, `System.Buffers >= 4.5.1`
- [depends](https://www.fuget.org/packages/FsCodec.NewtonsoftJson) on `FsCodec.Box`, `Newtonsoft.Json >= 11.0.2`, `Microsoft.IO.RecyclableMemoryStream >= 2.2.0`, `System.Buffers >= 4.5.1`
- [![System.Text.Json Codec NuGet](https://img.shields.io/nuget/v/FsCodec.SystemTextJson.svg)](https://www.nuget.org/packages/FsCodec.SystemTextJson/) `FsCodec.SystemTextJson`: See [#38](https://github.com/jet/FsCodec/pulls/38): drop in replacement that allows one to retarget from `Newtonsoft.Json` to the .NET Core >= v 3.0 default serializer: `System.Text.Json`, solely by changing the referenced namespace.
- [depends](https://www.fuget.org/packages/FsCodec.SystemTextJson) on `FsCodec`, `System.Text.Json >= 6.0.1`, `TypeShape >= 10`
- [depends](https://www.fuget.org/packages/FsCodec.SystemTextJson) on `FsCodec.Box`, `System.Text.Json >= 6.0.1`,

# Features: `FsCodec`

Expand Down Expand Up @@ -724,7 +724,7 @@ which yields the following output:
<a name="boxcodec"></a>
# Features: `FsCodec.Box.Codec`

`FsCodec.Box.Codec` is a drop-in-equivalent for `FsCodec.(Newtonsoft|SystemText)Json.Codec` with equivalent `.Create` overloads that encode as `ITimelineEvent<obj>` (as opposed to `ITimelineEvent<byte[]>` / `ITimelineEvent<JsonElement>`).
`FsCodec.Box.Codec` is a drop-in-equivalent for `FsCodec.(Newtonsoft|SystemText)Json.Codec` with equivalent `.Create` overloads that encode as `ITimelineEvent<obj>` (as opposed to `ITimelineEvent<ReadOnlyMemory<byte>>` / `ITimelineEvent<JsonElement>`).

This is useful when storing events in a `MemoryStore` as it allows one to take the perf cost and ancillary yak shaving induced by round-tripping arbitrary event payloads to the concrete serialization format out of the picture when writing property based unit and integration tests.

Expand Down
107 changes: 41 additions & 66 deletions src/FsCodec.Box/Codec.fs
Original file line number Diff line number Diff line change
@@ -1,108 +1,83 @@
/// Fork of FsCodec.NewtonsoftJson.Codec intended to provide equivalent calls and functionality, without actually serializing/deserializing as JSON
/// This is a useful facility for in-memory stores such as Equinox's MemoryStore as it enables you to
/// - efficiently test behaviors from an event sourced decision processing perspective (e.g. with Property Based Tests)
/// - without paying a serialization cost and/or having to deal with sanitization of generated data in order to make it roundtrippable through same
// Equivalent of FsCodec.NewtonsoftJson/SystemTextJson.Codec intended to provide equivalent calls and functionality, without actually serializing/deserializing as JSON
// This is a useful facility for in-memory stores such as Equinox's MemoryStore as it enables you to
// - efficiently test behaviors from an event sourced decision processing perspective (e.g. with Property Based Tests)
// - without paying a serialization cost and/or having to deal with sanitization of generated data in order to make it roundtrippable through same
namespace FsCodec.Box

open System
open System.Runtime.InteropServices

/// <summary>Provides Codecs that encode and/or extract Event bodies from a stream bearing a set of events defined in terms of a Discriminated Union,
/// using the conventions implied by using <c>TypeShape.UnionContract.UnionContractEncoder</c><br/>
/// <summary>Provides Codecs that render to boxed object, ideal for usage in a Memory Store.
/// Requires that Contract types adhere to the conventions implied by using <c>TypeShape.UnionContract.UnionContractEncoder</c><br/>
/// If you need full control and/or have have your own codecs, see <c>FsCodec.Codec.Create</c> instead.<br/>
/// See <a href="https://github.com/eiriktsarpalis/TypeShape/blob/master/tests/TypeShape.Tests/UnionContractTests.fs" /> for example usage.</summary>
/// See <a href="https://github.com/eiriktsarpalis/TypeShape/blob/master/tests/TypeShape.Tests/UnionContractTests.fs"></a> for example usage.</summary>
type Codec private () =

/// <summary>Generate a <code>IEventEncoder</code> Codec that roundtrips events by holding the boxed form of the Event body.<br/>
/// Uses <c>up</c> and <c>down</c> functions to facilitate upconversion/downconversion
/// and/or surfacing metadata to the Programming Model by including it in the emitted <c>'Event</c><br/>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or, if unspecified, the Discriminated Union Case Name
static let DefaultEncoder : TypeShape.UnionContract.IEncoder<obj> = TypeShape.UnionContract.BoxEncoder() :> _

/// <summary>Generate an <c>IEventCodec</c> that handles <c>obj</c> (boxed .NET <c>Object</c>) Event Bodies.<br/>
/// Uses <c>up</c>, <c>down</c> functions to handle upconversion/downconversion and eventId/correlationId/causationId mapping
/// and/or surfacing metadata to the programming model by including it in the emitted <c>'Event</c><br/>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or (if unspecified) the Discriminated Union Case Name;
/// <c>Contract</c> must be tagged with <c>interface TypeShape.UnionContract.IUnionContract</c> to signify this scheme applies.</summary>
static member Create<'Event, 'Contract, 'Meta, 'Context when 'Contract :> TypeShape.UnionContract.IUnionContract>
( /// <summary>Maps from the TypeShape <c>UnionConverter</c> <c>'Contract</c> case the Event has been mapped to (with the raw event data as context)
/// to the <c>'Event</c> representation (typically a Discriminated Union) that is to be presented to the programming model.</summary>
up : FsCodec.ITimelineEvent<obj> * 'Contract -> 'Event,
/// <summary>Maps a fresh Event resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionContract</c> <c>'Contract</c><br/>
/// The function is also expected to derive a <c>meta</c> object that will be held alongside the data (if it's not <c>None</c>)
/// together with its <c>eventId</c>, <c>correlationId</c>, <c>causationId</c> and an event creation <c>timestamp</c> (defaults to <c>UtcNow</c>).</summary>
/// <summary>Maps a fresh Event resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionConverter</c> <c>'Contract</c><br/>
/// The function is also expected to derive an optional <c>meta</c> object that will be serialized with the same <c>encoder</c>,
/// and <c>eventId</c>, <c>correlationId</c>, <c>causationId</c> and an Event Creation<c>timestamp</c></summary>.
down : 'Context option * 'Event -> 'Contract * 'Meta option * Guid * string * string * DateTimeOffset option,
/// <summary>Enables one to fail encoder generation if 'Contract contains nullary cases. Defaults to <c>false</c>, i.e. permitting them</summary>
/// <summary>Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them.</summary>
[<Optional; DefaultParameterValue(null)>] ?rejectNullaryCases)
: FsCodec.IEventCodec<'Event, obj, 'Context> =
Core.Codec.Create(DefaultEncoder, up, down, ?rejectNullaryCases = rejectNullaryCases)

let boxEncoder : TypeShape.UnionContract.IEncoder<obj> = TypeShape.UnionContract.BoxEncoder() :> _
let dataCodec =
TypeShape.UnionContract.UnionContractEncoder.Create<'Contract, obj>(
boxEncoder,
requireRecordFields = true,
allowNullaryCases = not (defaultArg rejectNullaryCases false))

{ new FsCodec.IEventCodec<'Event, obj, 'Context> with
member _.Encode(context, event) =
let (c, meta : 'Meta option, eventId, correlationId, causationId, timestamp : DateTimeOffset option) = down (context, event)
let enc = dataCodec.Encode c
let meta = meta |> Option.map boxEncoder.Encode<'Meta>
FsCodec.Core.EventData.Create(enc.CaseName, enc.Payload, defaultArg meta null, eventId, correlationId, causationId, ?timestamp = timestamp)

member _.TryDecode encoded =
let cOption = dataCodec.TryDecode { CaseName = encoded.EventType; Payload = encoded.Data }
match cOption with None -> None | Some contract -> let event = up (encoded, contract) in Some event }

/// <summary>Generate an <c>IEventCodec</c> that roundtrips events by holding the boxed form of the Event body.
/// Uses <c>up</c> and <c>down</c> and <c>mapCausation</c> functions to facilitate upconversion/downconversion and correlation/causationId mapping
/// and/or surfacing metadata to the Programming Model by including it in the emitted <c>'Event</c><br/>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or (if unspecified) the Discriminated Union Case Name
/// <summary>Generate an <c>IEventCodec</c> that handles <c>obj</c> (boxed .NET <c>Object</c>) Event Bodies.<br/>
/// Uses <c>up</c>, <c>down</c> and <c>mapCausation</c> functions to facilitate upconversion/downconversion and eventId/correlationId/causationId/timestamp mapping
/// and/or surfacing metadata to the programming model by including it in the emitted <c>'Event</c>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or (if unspecified) the Discriminated Union Case Name;
/// <c>Contract</c> must be tagged with <c>interface TypeShape.UnionContract.IUnionContract</c> to signify this scheme applies.</summary>
static member Create<'Event, 'Contract, 'Meta, 'Context when 'Contract :> TypeShape.UnionContract.IUnionContract>
( /// <summary>Maps from the TypeShape <c>UnionConverter</c> <c>'Contract</c> case the Event has been mapped to (with the raw event data as context)
/// to the representation (typically a Discriminated Union) that is to be presented to the programming model.</summary>
up : FsCodec.ITimelineEvent<obj> * 'Contract -> 'Event,
/// <summary>Maps a fresh Event resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionConverter</c> <c>'Contract</c><br/>
/// The function is also expected to derive:<br>
/// - a <c>meta</c> object that will be serialized with the same settings (if it's not <c>None</c>)<br/>
/// - and an Event Creation <c>timestamp</c>.<summary>
/// <summary>Maps a fresh Event resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionConverter</c> <c>'Contract</c>
/// The function is also expected to derive
/// a <c>meta</c> object that will be serialized with the same options (if it's not <c>None</c>)
/// and an Event Creation <c>timestamp</c>.</summary>
down : 'Event -> 'Contract * 'Meta option * DateTimeOffset option,
/// <summary>Uses the 'Context passed to the Encode call and the 'Meta emitted by <c>down</c> to a) the final metadata b) the <c>correlationId</c> and c) the correlationId</summary>
/// <summary>Uses the 'Context passed to the Encode call and the 'Meta emitted by <c>down</c> to a) the final metadata b) the <c>eventId</c> c) the <c>correlationId</c> and d) the <c>causationId</c></summary>
mapCausation : 'Context option * 'Meta option -> 'Meta option * Guid * string * string,
/// <summary>Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them</summary>
/// <summary>Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them.</summary>
[<Optional; DefaultParameterValue(null)>] ?rejectNullaryCases)
: FsCodec.IEventCodec<'Event, obj, 'Context> =
Core.Codec.Create(DefaultEncoder, up, down, mapCausation, ?rejectNullaryCases = rejectNullaryCases)

let down (context, event) =
let c, m, t = down event
let m', eventId, correlationId, causationId = mapCausation (context, m)
c, m', eventId, correlationId, causationId, t
Codec.Create(up = up, down = down, ?rejectNullaryCases = rejectNullaryCases)

/// <summary>Generate an <code>IEventCodec</code> that roundtrips events by holding the boxed form of the Event body.<br/>
/// Uses <c>up</c> and <c>down</c> and <c>mapCausation</c> functions to facilitate upconversion/downconversion and correlation/causationId mapping
/// and/or surfacing metadata to the Programming Model by including it in the emitted <c>'Event</c><br/>
/// <summary>Generate an <c>IEventCodec</c> that handles <c>obj</c> (boxed .NET <c>Object</c>) Event Bodies.<br/>
/// Uses <c>up</c> and <c>down</c> functions to facilitate upconversion/downconversion/timestamping without eventId/correlation/causationId mapping
/// and/or surfacing metadata to the programming model by including it in the emitted <c>'Event</c>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or (if unspecified) the Discriminated Union Case Name
/// <c>Contract</c> must be tagged with <c>interface TypeShape.UnionContract.IUnionContract</c> to signify this scheme applies.</summary>
/// <c>Contract</c> must be tagged with <c>interface TypeShape.UnionContract.IUnionContract</c> to signify this scheme applies</summary>.
static member Create<'Event, 'Contract, 'Meta when 'Contract :> TypeShape.UnionContract.IUnionContract>
( /// <summary>Maps from the TypeShape <c>UnionConverter</c> <c>'Contract</c> case the Event has been mapped to (with the raw event data as context)
/// to the representation (typically a Discriminated Union) that is to be presented to the programming model.</summary>
up : FsCodec.ITimelineEvent<obj> * 'Contract -> 'Event,
/// <summary>Maps a fresh Event resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionConverter</c> <c>'Contract</c>
/// The function is also expected to derive:<br/>
/// - a <c>meta</c> object that will be serialized with the same settings (if it's not <c>None</c>)<br/>
/// - and an Event Creation <c>timestamp</c>.</summary>
/// <summary>Maps a fresh <c>'Event</c> resulting from a Decision in the Domain representation type down to the TypeShape <c>UnionConverter</c> <c>'Contract</c>
/// The function is also expected to derive
/// a <c>meta</c> object that will be serialized with the same options (if it's not <c>None</c>)
/// and an Event Creation <c>timestamp</c>.</summary>
down : 'Event -> 'Contract * 'Meta option * DateTimeOffset option,
/// <summary>Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them.</summary>
[<Optional; DefaultParameterValue(null)>] ?rejectNullaryCases)
: FsCodec.IEventCodec<'Event, obj, obj> =
Core.Codec.Create(DefaultEncoder, up, down, ?rejectNullaryCases = rejectNullaryCases)

let mapCausation (_context : obj, m : 'Meta option) = m, Guid.NewGuid(), null, null
Codec.Create(up = up, down = down, mapCausation = mapCausation, ?rejectNullaryCases = rejectNullaryCases)

/// <summary>Generate an <code>IEventCodec</code> that roundtrips events by holding the boxed form of the Event body.<br/>
/// <summary>Generate an <c>IEventCodec</c> that handles <c>obj</c> (boxed .NET <c>Object</c>) Event Bodies.<br/>
/// The Event Type Names are inferred based on either explicit <c>DataMember(Name=</c> Attributes, or (if unspecified) the Discriminated Union Case Name
/// <c>'Union</c> must be tagged with <c>interface TypeShape.UnionContract.IUnionContract</c> to signify this scheme applies.</summary>
static member Create<'Union when 'Union :> TypeShape.UnionContract.IUnionContract>
( /// Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them
( /// <summary>Enables one to fail encoder generation if union contains nullary cases. Defaults to <c>false</c>, i.e. permitting them.</summary>
[<Optional; DefaultParameterValue(null)>] ?rejectNullaryCases)
: FsCodec.IEventCodec<'Union, obj, obj> =

let up : FsCodec.ITimelineEvent<_> * 'Union -> 'Union = snd
let down (event : 'Union) = event, None, None
Codec.Create(up = up, down = down, ?rejectNullaryCases = rejectNullaryCases)
Core.Codec.Create(DefaultEncoder, ?rejectNullaryCases = rejectNullaryCases)
Loading

0 comments on commit b37094c

Please sign in to comment.