From c4832b8baa220a3ef8181a2a110c4b9f20fe688b Mon Sep 17 00:00:00 2001 From: Peter Ling Date: Thu, 11 Jan 2024 09:33:08 +0000 Subject: [PATCH 1/2] VUU-1118: WIP Create GH Action to generate README --- .github/workflows/generate-readme.sh | 4 ++++ .github/workflows/generate-readme.yml | 18 ++++++++++++++++++ docs/readme_contents.txt | 2 ++ 3 files changed, 24 insertions(+) create mode 100644 .github/workflows/generate-readme.sh create mode 100644 .github/workflows/generate-readme.yml create mode 100644 docs/readme_contents.txt diff --git a/.github/workflows/generate-readme.sh b/.github/workflows/generate-readme.sh new file mode 100644 index 000000000..02f501619 --- /dev/null +++ b/.github/workflows/generate-readme.sh @@ -0,0 +1,4 @@ +#!/bin/sh + +printf '[//]: # (This file is auto-generated by GitHub Actions. Please do not edit manually.)\n\n' > README_AUTO.md +{ cat $(cat ./docs/readme_contents.txt) ; } >> README_AUTO.md diff --git a/.github/workflows/generate-readme.yml b/.github/workflows/generate-readme.yml new file mode 100644 index 000000000..6a81dde72 --- /dev/null +++ b/.github/workflows/generate-readme.yml @@ -0,0 +1,18 @@ +name: Generate README from docs folder +on: push + +jobs: + run: + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v3 + - name: Generate readme + run: bash .github/workflows/generate-readme.sh + - name: Commit + uses: EndBug/add-and-commit@v9 + with: + message: 'Update readme' + add: 'README_AUTO.md' + default_author: github_actions diff --git a/docs/readme_contents.txt b/docs/readme_contents.txt new file mode 100644 index 000000000..99ba64ccb --- /dev/null +++ b/docs/readme_contents.txt @@ -0,0 +1,2 @@ +./docs/introduction/*.md +./docs/getting_started/*.md From 9bc06cd0581d4bd9656ab13a40a67361a65a27f7 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 11 Jan 2024 10:36:47 +0000 Subject: [PATCH 2/2] Update readme --- README_AUTO.md | 235 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 README_AUTO.md diff --git a/README_AUTO.md b/README_AUTO.md new file mode 100644 index 000000000..e486d73a1 --- /dev/null +++ b/README_AUTO.md @@ -0,0 +1,235 @@ +[//]: # (This file is auto-generated by GitHub Actions. Please do not edit manually.) + +# Github Repository + +* [Main Repository](https://github.com/venuu-io/vuu) +* [Sample Project Repository](https://github.com/venuu-io/vuu-getting-started) + +# How does it work + +Let's start with a diagram. + +![](./diagrams-server-internals.svg) + +This shows the basic flow through the system, there are components which have been omitted for clarity. + +On the left hand side of the screen you can see **providers**. These source data from the external world, or a model in process +and push them into a table. The table acts like a sink and forwards the event onto any join tables that are composed of +this underlying table, and or to any viewports that are on this table directly. + +**Viewports** are a users view onto a particular table, or join table. Viewports have knowledge of where a user is looking +onto the underlying data. They also know if the user has asked for a filter or sort on the data. + +Finally we have the UI. This is connected to the Vuu server via a websocket and receives updates in realtime. + +There are two important factors to take into account from this diagram, the first is that what is in your viewport is calculated +out of band on a separate thread to the update path (Filter and Sort Thread.) The filter and sort thread takes the underlying +table you have asked for in your viewport and applies the filters and sorts you've requested. The resulting array of +primary keys is then pushed into your viewport. + +The update path, by comparison, is different. If you have a particular key in the visible range of your viewport already +and it is updated, it follows the tick() path, so it will progress through the system as an event, on the same thread +as the provider itself. + +The upshot of this is that **Vuu favours the update path** (the orange line in the diagram) over the calculation of new data in your UI. You will most +likely rarely if ever notice this, but it is an important concept. +# What are the trade offs + +Here are some trade-offs for view servers in general and Vuu itself + +1. View servers are complicated pieces of software and adding them into your stack adds complexity. +2. VS's move more processing from the client to the server, so if you're worried about server capacity, it may not be for you. +3. Vuu itself favours the tick path, this means that when you have aggregates (see trees later) you can end up with +eventually consistent aggregates vs column values. For example if my tree has a Sum(orders.price) and the price is changing +every second, your total will only be calculated on a cycle, not on every tick. THis is mostly fine, but can cause issues +with some systems. +# What is a View (Vuu) Server + +A View Server is a component in a computer system which allows you to track state and potentially off-load some +of the processing cost of the UI on to the server. + +Typically many users can connect into a single view server, and data which is being used by them can be shared rather than distinct copies. + +View Servers offer some or all of these features to a UI: + +* **View Porting** - The Server maintains a full set of data for the client. When a client has a large grid open on their +screen they are passed only the data that is visible (plus a bit extra at the top and bottom for performance), not the entire data set. When the +client then scrolls through this data it appears it's all on the client but is in reality being streamed from the server. +Updates to the client's data (view port) are propagated as they happen. Server side viewports can offer: + * **Filtering & Sorting** of the data, as if it were on the client. + * **Treeing and Aggregations** on columns within viewports +* **Remote Procedure Calls** - When a client wants to effect change in the outside world they need a place to host business +logic. This business logic potentially needs access to the whole set of data within the server itself. As the client only has its viewport +onto this data, it is poorly suited to do that. RPC calls are a way of hosting logic that a client can call. +* **Joining Data** - When data comes from different sources, example stock prices verses stock reference data, we often want to join that data together +and present it as a single table/single grid at runtime. +* **Storing of UI State** - When a client changes how her UI is configured, the view server typically offers a mechanism to persist that state. + +Happily Vuu offers all of these features. + +## What Vuu Is: + +* A mechanism for representing ticking external data in table form on the server +* A relational algebra function for joining tables on server at runtime, including linking parent child tables +* A viewporting layer for giving clients a virtualized view onto large tables +* A filter (ANTLR grammar) and sort within viewports +* A "fast path" for updating ticking data to the client +* A "slow path" for updating viewport contents via separate threads +* A treeing and aggregation mechanism (showing totals, averages etc..) +* A highly performant React based Grid implementation and layout framework +* A websocket based wire protocol for handling viewport changes +* An RPC framework for invoking CRUD style operations on the server from the client + +## What Vuu is not: + +* A UI Widget framework +* A client side UI Framework +* An alternative to Tomcat/Websphere +* A portal framework +* A data distribution or store and forward technology like Kafka or MQ + +## When should I use Vuu? + +* If you want to develop a highly functional largely grid based app +* Where your data neatly fits into a table like paradigm +* Where you want to target html5 technologies +* Where you want updates to trigger directly through to client when they occur in the outside world +import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator"; + +# Using binaries from Maven Repo + + + +The Vuu binaries are hosted in Maven Central under the namespace: [org.finos.vuu](https://repo1.maven.org/maven2/org/finos/vuu/). + +You can add them to your pom by referencing the parent pom directly. + +``` + + org.finos.vuu + vuu-parent + {check the latest version} + + + + org.finos.vuu + vuu-ui + {check the latest version} + +``` + +Adding the javascript components: + +``` +Work in Progress.... +``` +# Building From Source + + + +import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator"; + +# Configuration + + + +``` +Work in Progress.... +``` +import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator"; + +# Developing Vuu + + + +## Prerequisites + +1. Install IntelliJ Community Edition (latest version with supported scala plugin). +2. Install SDKMan from the [website](https://sdkman.io/) or using your own mechanism +3. type>`sdk install java 11.0.12-open` and then >`sdk d java 11.0.12-open` to make sure you're using the correct one. +4. Clone the repo into a directory on your machine +5. Open the project as a Maven build by selecting the root pom.xml (make sure you select "enable adding maven modules, auto import etc..) +6. You should get one root module vuu-parent in a project list, select this +7. When the project opens you should have 3 sub-modules (vuu, toolbox and vuu-ui) + + + +## Developing the client + +If you are comfortable running the server in an IDE, you can follow the instructions above. If not +you can use the specific maven targets from the command line to run up the sample app. + +You can install command line maven via any means you please, but sdkman makes it easy... + +```bash +sdk install maven +``` + +## Installation - Server + +### Prerequisites + +See the [Docs](https://vuu.finos.org/docs/getting_started/developing) for Java versions and install dependencies you need to have. + +OS X & Linux: + +```sh +#In your favourite code directory... +git clone https://github.com/finos/vuu.git +#cd into the repository +cd vuu +#run the maven compile step +mvn compile +#cd into vuu, child in repo +cd example/main +#The server should now be started on your machine with Simulation module +mvn exec:exec +``` + +Windows: + +```sh +this should be the same as Linux & MacOS just with Windows adjusted paths +``` + +## Running the Vuu Server Simulation Module from IDE + +1. Go to the SimulMain.scala, right click and run (add these into JVM args -Xmx10G -Xms5G, or whatever you have available) + +## Installation - Client + +You will need npm at version 16 or greater to build the client. + +```sh +#in vuu repo (not vuu child directory in repo) +cd vuu-ui +npm install +npm run build +npm run build:app +#if you would also like to use electron rather than Chrome/Chromium +cd tools/electron +npm install #You only need to do this once initially and when the electron version is upgraded +cd ../.. +npm run launch:demo:electron +``` + +If you are using Chrome, you should now be able to use a local browser to see the Vuu demo app. [localhost:8443](https://localhost:8443/index.html) + +If you are getting certificate errors when you connect, set this browser setting in chrome: + +``` +chrome://flags/#allow-insecure-localhost (set to true) +``` + +## Developing the Vuu Book + +We use [docusaurus](https://docusaurus.io/blog/2022/08/01/announcing-docusaurus-2.0) to generate the Vuu docs from markdown. +import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator"; + +# Using Vuu from Java + + + +We have a sample Maven module within the code repo: + +[Getting Started in Java](https://github.com/finos/vuu/tree/main/example/main-java)