This is an re-implementation of the engine powering my former startup CloudMatch. The idea is to allow entities to establish a connection and then exchange data via a Websocket connection. This project is currently a learning playground.
- Scala
- Akka (Clustering, Streaming, Http/Websockets)
- Docker
- Kubernetes (cluster discovery via Akka Management / Cluster Bootstrap)
- Slick
- RabbitMQ (as a cloud service)
- Flink
I wrote a couple of blog posts about the distribution of Websocket connections:
The project consists of a few items:
The core of the project. I talked about this architecture at lots of conferences,, together with Adam Sandor and most notably during the online webinar with Lightbend. Click on the image below to view the recording.
The idea is to have a Listener service which provide an endpoint for connecting devices. Then we have a set of clustered nodes where each connected device is represented by an actor. These actors will exchange messages carrying the payloads sent by matched devices.
I use Flink as a data stream processor via its official RabbitMQ connector.
A service that hides a persistence layer using an in-memory database (H2) and uses Slick to insert and retrieve events. It is meant to be used as a sort of half-baked event journal.
A tool to test the matching of connecting devices that I built in 2013 with the very first version of Angular. It still works!
A few simple test endpoints for Rest clients such as Paw (my fave) or Postman.
Automation is quite limited at the moment. A couple of scripts package all services in Docker containers and deploy to the current context in kubectl.
Each actor representing a device talks to the database service to append events to the journal (such as ConnectionEstablished) and publishes events to the RabbitMQ topic where each message is the size in bytes of a message exchanged between two matched entities.
The Flink streaming engines consumes these events, say you want to window over how many bytes are exchanged per minute.
The screenshot below shows a few things
- All pods and services deployed on my local K8S cluster
- The fabulous test client - devices 1 and 3 have matched and device 1 has sent "ciao" to device 3
- On the bottom left, logs from the Flink stream show that it consumed the message containing "4" from the RabbitMQ queue, as "ciao" is made by 4 unicode characters.
- IoT devices talking to each other
- Exchanging payload between devices during a videogame
- Anything where two devices need to exchange something, basically
Copyright 2019 Fabio Tiriticco - Fabway
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.