Rebase on Jet.ConfluentKafka.FSharp 1.0.0-preview1

CHANGELOG.md

@@ -21,6 +21,9 @@ _NB at the present time, this project does not adhere strictly to Semantic Versi
 - Renamespaced and separated `Equinox.Codec` APIs to separate `Newtonsoft.Json` and custom `encode`/`tryDecode` approaches [#102](https://github.com/jet/equinox/issues/102) (in preparation for [#79](https://github.com/jet/equinox/issues/79))
 
 ### Removed
+
+- Extracted `Equinox.Projection.Kafka` and its integration tests to become [`Jet.ConfluentKafka.FSharp` v `1.0.0-preview1`](https://github.com/jet/Jet.ConfluentKafka.FSharp/tree/v1); retargeted `Equinox.Tool` to use same.
+
 ### Fixed
 
 - Add Writing empty event list guard for `Equinox.Cosmos` [#105](https://github.com/jet/equinox/issues/105)

DOCUMENTATION.md

@@ -589,7 +589,7 @@ In an Event Sourced system, we typically distinguish between the following basic
 
 - _Snapshots_ - A snapshot is an intentionally roundtrippable version of a State, that can be saved and restored. Typically one would do this to save the (latency, roundtrips, RUs, deserialization and folding) cost of loading all the Events in a long running sequence of Events to re-establish the State. The [EventStore folks have a great walkthrough on Rolling Snapshots](https://eventstore.org/docs/event-sourcing-basics/rolling-snapshots/index.html).
 
-- Projections - the term projection is *heavily* overloaded, meaning anything from the proceeds of a SELECT statement, the result of a `map` operation, an EventStore projection, an event being propagated via Kafka (no, further examples are not required!).
+- Projections - the term projection is *heavily* overloaded, meaning anything from the proceeds of a SELECT statement, the result of a `map` operation, an EventStore projection to an event being propagated via Kafka (no, further examples are not required!).
 
 .... and:
 
@@ -772,12 +772,10 @@ While [Kafka is not for Event Sourcing](https://medium.com/serialized-io/apache-
 
 The [Apache Kafka intro docs](https://kafka.apache.org/intro) provide a clear terse overview of the design and attendant benefits this brings to bear.
 
-As noted in the [Effect of ChangeFeed on Request Charges](https://github.com/jet/equinox/blob/master/DOCUMENTATION.md#effect-of-changefeed-on-request-charges) section, it can make sense to replicate the ChangeFeed to Kafka purely from the point of view of optimising request charges (and not needing to consider projections when considering how to scale up provisioning for load). Other benefits are mechanical sympathy based - Kafka is very often the right tool for the job in scaling out traversal of events for a variety of use cases.
+As noted in the [Effect of ChangeFeed on Request Charges](https://github.com/jet/equinox/blob/master/DOCUMENTATION.md#effect-of-changefeed-on-request-charges) section, it can make sense to replicate a subset of the ChangeFeed to a Kafka topic (both for projections being consumed within a Bounded Context and for cases where you are generating a Pubished Notification Event) purely from the point of view of optimising request charges (and not needing to consider projections when considering how to scale up provisioning for load). Other benefits are mechanical sympathy based - Kafka is often the right tool for the job in scaling out traversal of events for a variety of use cases given one has sufficient traffic to warrant the complexity.
 
 See the [PR that added the initial support for CosmosDb Projections](https://github.com/jet/equinox/pull/87) and [the QuickStart](README.md#quickstart) for instructions.
 
-It should be noted that the `Equinox.Projection.Kafka` batteries included projector/consumer library targets the `Confluent.Kafka` `1.0.0-*` series (as opposed to the present stable `0.11.4` thru `6` editions). There are significant breaking changes between the `0.` and `1.` releases, and there is no plan to support `0.1.x` targeting in this codebase (PR 87's history does include a passing library and tests targeting `0.11.4` for history buffs).
-
 - https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
 - https://www.confluent.io/wp-content/uploads/confluent-kafka-definitive-guide-complete.pdf
 

Directory.Build.props

@@ -4,7 +4,7 @@
     <Company>Jet.com</Company>
     <Description>Composable high performance event sourcing componentry</Description>
     <PackageProjectUrl>https://github.com/jet/equinox</PackageProjectUrl>
-    <PackageTags>fsharp unionencoder eventsourcing eventstore typeshape</PackageTags>
+    <PackageTags>fsharp unionencoder eventsourcing eventstore cosmosdb typeshape</PackageTags>
     <PackageLicense>Apache-2.0</PackageLicense>
     <Copyright>Copyright © 2016-9</Copyright>
 
@@ -33,5 +33,4 @@
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
     </Content>
   </ItemGroup>
-
 </Project>

Equinox.sln

@@ -59,10 +59,6 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Todo", "Todo", "{8CDE1CC3-8
 EndProject
 Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Equinox.Cosmos.Projection", "src\Equinox.Cosmos.Projection\Equinox.Cosmos.Projection.fsproj", "{D4641679-9C2B-4B4C-ABA9-B2EC1CB1D79D}"
 EndProject
-Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Equinox.Projection.Kafka", "src\Equinox.Projection.Kafka\Equinox.Projection.Kafka.fsproj", "{F76A4BA7-3DA8-4012-843B-9E28A4D2EC86}"
-EndProject
-Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Equinox.Projection.Kafka.Integration", "tests\Equinox.Projection.Kafka.Integration\Equinox.Projection.Kafka.Integration.fsproj", "{C8383D84-80F5-46B8-B59B-48B2359E6A37}"
-EndProject
 Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Equinox.Projection.Codec", "src\Equinox.Projection.Codec\Equinox.Projection.Codec.fsproj", "{65F8FC6A-FF31-43A0-B7D9-4C9CF407DEB3}"
 EndProject
 Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Equinox.Projection.Tests", "tests\Equinox.Projection.Tests\Equinox.Projection.Tests.fsproj", "{047F782D-DC37-4599-8FA0-F9B4D4C09C7B}"
@@ -143,14 +139,6 @@ Global
 		{D4641679-9C2B-4B4C-ABA9-B2EC1CB1D79D}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{D4641679-9C2B-4B4C-ABA9-B2EC1CB1D79D}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{D4641679-9C2B-4B4C-ABA9-B2EC1CB1D79D}.Release|Any CPU.Build.0 = Release|Any CPU
-		{F76A4BA7-3DA8-4012-843B-9E28A4D2EC86}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{F76A4BA7-3DA8-4012-843B-9E28A4D2EC86}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{F76A4BA7-3DA8-4012-843B-9E28A4D2EC86}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{F76A4BA7-3DA8-4012-843B-9E28A4D2EC86}.Release|Any CPU.Build.0 = Release|Any CPU
-		{C8383D84-80F5-46B8-B59B-48B2359E6A37}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
-		{C8383D84-80F5-46B8-B59B-48B2359E6A37}.Debug|Any CPU.Build.0 = Debug|Any CPU
-		{C8383D84-80F5-46B8-B59B-48B2359E6A37}.Release|Any CPU.ActiveCfg = Release|Any CPU
-		{C8383D84-80F5-46B8-B59B-48B2359E6A37}.Release|Any CPU.Build.0 = Release|Any CPU
 		{65F8FC6A-FF31-43A0-B7D9-4C9CF407DEB3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
 		{65F8FC6A-FF31-43A0-B7D9-4C9CF407DEB3}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{65F8FC6A-FF31-43A0-B7D9-4C9CF407DEB3}.Release|Any CPU.ActiveCfg = Release|Any CPU

README.md

@@ -114,7 +114,7 @@ While Equinox is implemented in F#, and F# is a great fit for writing event-sour
     eqx initAux -ru 400 cosmos # generates a -aux collection for the ChangeFeedProcessor to maintain consumer group progress within
     # -v for verbose ChangeFeedProcessor logging
     # `projector1` represents the consumer group - >=1 are allowed, allowing multiple independent projections to run concurrently
-    # stats specifies one only wants stats regarding items (other options include kafka to project to kafka)
+    # stats specifies one only wants stats regarding items (other options include `kafka` to project to Kafka)
     # cosmos specifies source overrides (using defaults in step 1 in this instance)
     eqx -v project projector1 stats cosmos
     ```
@@ -148,7 +148,7 @@ While Equinox is implemented in F#, and F# is a great fit for writing event-sour
     eqx -v project projector3 -l 5 kafka temp-topic cosmos
     ```
 
-7. **(Preview)** Generate CosmosDb [Kafka Projector and Consumer](DOCUMENTATION.md#feeding-to-kafka) `.fsproj`ects (using `Equinox.Projection.Kafka`)
+7. **(Preview)** Generate CosmosDb [Kafka Projector and Consumer](DOCUMENTATION.md#feeding-to-kafka) `.fsproj`ects (using `Jet.ConfluentKafka.FSharp` v `1.0.0`)
 
     ```powershell
     cat readme.md # more complete instructions regarding the code
@@ -215,8 +215,8 @@ The components within this repository are delivered as a series of multi-targete
 ### Projection libraries
 
 - [![Cosmos.Projection NuGet](https://img.shields.io/nuget/v/Equinox.Cosmos.Projection.svg)](https://www.nuget.org/packages/Equinox.Cosmos.Projection/) `Equinox.Cosmos.Projection`: Wraps the [Microsoft .NET `ChangeFeedProcessor` library](https://github.com/Azure/azure-documentdb-changefeedprocessor-dotnet) providing a [processor loop](DOCUMENTATION.md#change-feed-processors) that maintains a continuous natched query loop per CosmosDb Physical Partition (Range) yielding new or updated documents (optionally unrolling events written by `Equinox.Cosmos` for processing or forwarding). Used in the [`eqx project stats cosmos`](dotnet-tool-provisioning--benchmarking-tool) tool command; see [`dotnet new eqx projector` to generate a sample app](quickstart) using it. ([depends](https://www.fuget.org/packages/Equinox.Cosmos.Projection) on `Equinox.Cosmos`, `Microsoft.Azure.DocumentDb.ChangeFeedProcessor >= 2.2.5`)
-- [![Projection.Kafka NuGet](https://img.shields.io/nuget/v/Equinox.Projection.Kafka.svg)](https://www.nuget.org/packages/Equinox.Projection.Kafka/) `Equinox.Projection.Kafka`: Wraps `Confluent.Kafka` to provide efficient batched Kafka Producer and Consumer configurations, with basic logging instrumentation. Used in the [`eqx project kafka`](dotnet-tool-provisioning--benchmarking-tool) tool command; see [`dotnet new eqx projector -k` to generate a sample app](quickstart) using it. ([depends](https://www.fuget.org/packages/Equinox.Projection.Kafka) on `Confluent.Kafka >= 1.0.0-beta3`, `Serilog`)
 - [![Projection.Codec NuGet](https://img.shields.io/nuget/v/Equinox.Projection.Codec.svg)](https://www.nuget.org/packages/Equinox.Projection.Codec/) `Equinox.Projection.Codec`: Provides a standard `RenderedEvent` that can be used as a default format when projecting events via e.g. `Equinox.Projection.Kafka`. Also provides a `FeedValidator` that can be used to validate when duplicate events and/or gaps observed in the sequences of events (see [`eqx project stats` and `kafka`](dotnet-tool-provisioning--benchmarking-tool)). ([depends](https://www.fuget.org/packages/Equinox.Projection.Codec) on `Newtonsoft.Json >= 11.0.2`)
+- _Related, maintained in separate repo_ [![Jet.ConfluentKafka.FSharp NuGet](https://img.shields.io/nuget/vpre/Jet.ConfluentKafka.FSharp.svg)](https://www.nuget.org/packages/Jet.ConfluentKafka.FSharp/) `Jet.ConfluentKafka.FSharp`: Wraps `Confluent.Kafka` to provide efficient batched Kafka Producer and Consumer configurations, with basic logging instrumentation. Used in the [`eqx project kafka`](dotnet-tool-provisioning--benchmarking-tool) tool command; see [`dotnet new eqx projector -k` to generate a sample app](quickstart) using it.
 
 ### `dotnet tool` provisioning / benchmarking tool
 

build.proj

@@ -23,7 +23,6 @@
     <Exec Command="dotnet pack src/Equinox.EventStore $(Cfg) $(PackOptions)" />
     <Exec Command="dotnet pack src/Equinox.MemoryStore $(Cfg) $(PackOptions)" />
     <Exec Command="dotnet pack src/Equinox.Projection.Codec $(Cfg) $(PackOptions)" />
-    <Exec Command="dotnet pack src/Equinox.Projection.Kafka $(Cfg) $(PackOptions)" />
     <Exec Command='dotnet publish tools/Equinox.Tool $(Cfg) -f net461 -o "$(RepoDir)/bin/equinox-tool/net461" ' />
     <Exec Command="dotnet pack tools/Equinox.Tool $(Cfg) $(PackOptions) /p:PackAsTool=true" />
   </Target>
@@ -33,7 +32,6 @@
     <Exec Command="dotnet test tests/Equinox.MemoryStore.Integration $(Cfg) $(TestOptions)" />
     <Exec Command="dotnet test tests/Equinox.EventStore.Integration $(Cfg) $(TestOptions)" />
     <Exec Command="dotnet test tests/Equinox.Cosmos.Integration $(Cfg) $(TestOptions)" />
-    <Exec Command="dotnet test tests/Equinox.Projection.Kafka.Integration $(Cfg) $(TestOptions)" />
     <Exec Command="dotnet test tests/Equinox.Projection.Tests $(Cfg) $(TestOptions)" />
     <Exec Command="dotnet test samples/Store/Integration $(Cfg) $(TestOptions)" />
     <Exec Command="dotnet test samples/Store/Domain.Tests $(Cfg) $(TestOptions)" />