Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VUU-1118: Auto-generate README from docs #1119

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/generate-readme.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh

printf '[//]: # (This file is auto-generated by GitHub Actions. Please do not edit manually.)\n\n' > README_AUTO.md
{ cat $(cat ./docs/readme_contents.txt) ; } >> README_AUTO.md
18 changes: 18 additions & 0 deletions .github/workflows/generate-readme.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Generate README from docs folder
on: push

jobs:
run:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3
- name: Generate readme
run: bash .github/workflows/generate-readme.sh
- name: Commit
uses: EndBug/add-and-commit@v9
with:
message: 'Update readme'
add: 'README_AUTO.md'
default_author: github_actions
235 changes: 235 additions & 0 deletions README_AUTO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
[//]: # (This file is auto-generated by GitHub Actions. Please do not edit manually.)

# Github Repository

* [Main Repository](https://github.com/venuu-io/vuu)
* [Sample Project Repository](https://github.com/venuu-io/vuu-getting-started)

# How does it work

Let's start with a diagram.

![](./diagrams-server-internals.svg)

This shows the basic flow through the system, there are components which have been omitted for clarity.

On the left hand side of the screen you can see **providers**. These source data from the external world, or a model in process
and push them into a table. The table acts like a sink and forwards the event onto any join tables that are composed of
this underlying table, and or to any viewports that are on this table directly.

**Viewports** are a users view onto a particular table, or join table. Viewports have knowledge of where a user is looking
onto the underlying data. They also know if the user has asked for a filter or sort on the data.

Finally we have the UI. This is connected to the Vuu server via a websocket and receives updates in realtime.

There are two important factors to take into account from this diagram, the first is that what is in your viewport is calculated
out of band on a separate thread to the update path (Filter and Sort Thread.) The filter and sort thread takes the underlying
table you have asked for in your viewport and applies the filters and sorts you've requested. The resulting array of
primary keys is then pushed into your viewport.

The update path, by comparison, is different. If you have a particular key in the visible range of your viewport already
and it is updated, it follows the tick() path, so it will progress through the system as an event, on the same thread
as the provider itself.

The upshot of this is that **Vuu favours the update path** (the orange line in the diagram) over the calculation of new data in your UI. You will most
likely rarely if ever notice this, but it is an important concept.
# What are the trade offs

Here are some trade-offs for view servers in general and Vuu itself

1. View servers are complicated pieces of software and adding them into your stack adds complexity.
2. VS's move more processing from the client to the server, so if you're worried about server capacity, it may not be for you.
3. Vuu itself favours the tick path, this means that when you have aggregates (see trees later) you can end up with
eventually consistent aggregates vs column values. For example if my tree has a Sum(orders.price) and the price is changing
every second, your total will only be calculated on a cycle, not on every tick. THis is mostly fine, but can cause issues
with some systems.
# What is a View (Vuu) Server

A View Server is a component in a computer system which allows you to track state and potentially off-load some
of the processing cost of the UI on to the server.

Typically many users can connect into a single view server, and data which is being used by them can be shared rather than distinct copies.

View Servers offer some or all of these features to a UI:

* **View Porting** - The Server maintains a full set of data for the client. When a client has a large grid open on their
screen they are passed only the data that is visible (plus a bit extra at the top and bottom for performance), not the entire data set. When the
client then scrolls through this data it appears it's all on the client but is in reality being streamed from the server.
Updates to the client's data (view port) are propagated as they happen. Server side viewports can offer:
* **Filtering & Sorting** of the data, as if it were on the client.
* **Treeing and Aggregations** on columns within viewports
* **Remote Procedure Calls** - When a client wants to effect change in the outside world they need a place to host business
logic. This business logic potentially needs access to the whole set of data within the server itself. As the client only has its viewport
onto this data, it is poorly suited to do that. RPC calls are a way of hosting logic that a client can call.
* **Joining Data** - When data comes from different sources, example stock prices verses stock reference data, we often want to join that data together
and present it as a single table/single grid at runtime.
* **Storing of UI State** - When a client changes how her UI is configured, the view server typically offers a mechanism to persist that state.

Happily Vuu offers all of these features.

## What Vuu Is:

* A mechanism for representing ticking external data in table form on the server
* A relational algebra function for joining tables on server at runtime, including linking parent child tables
* A viewporting layer for giving clients a virtualized view onto large tables
* A filter (ANTLR grammar) and sort within viewports
* A "fast path" for updating ticking data to the client
* A "slow path" for updating viewport contents via separate threads
* A treeing and aggregation mechanism (showing totals, averages etc..)
* A highly performant React based Grid implementation and layout framework
* A websocket based wire protocol for handling viewport changes
* An RPC framework for invoking CRUD style operations on the server from the client

## What Vuu is not:

* A UI Widget framework
* A client side UI Framework
* An alternative to Tomcat/Websphere
* A portal framework
* A data distribution or store and forward technology like Kafka or MQ

## When should I use Vuu?

* If you want to develop a highly functional largely grid based app
* Where your data neatly fits into a table like paradigm
* Where you want to target html5 technologies
* Where you want updates to trigger directly through to client when they occur in the outside world
import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator";

# Using binaries from Maven Repo

<SvgDottySeparator style={{marginBottom: 32}}/>

The Vuu binaries are hosted in Maven Central under the namespace: [org.finos.vuu](https://repo1.maven.org/maven2/org/finos/vuu/).

You can add them to your pom by referencing the parent pom directly.

```
<dependency>
<groupId>org.finos.vuu</groupId>
<artifactId>vuu-parent</artifactId>
<version>{check the latest version}</version>
</dependency>

<dependency>
<groupId>org.finos.vuu</groupId>
<artifactId>vuu-ui</artifactId>
<version>{check the latest version}</version>
</dependency>
```

Adding the javascript components:

```
Work in Progress....
```
# Building From Source



import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator";

# Configuration

<SvgDottySeparator style={{marginBottom: 32}}/>

```
Work in Progress....
```
import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator";

# Developing Vuu

<SvgDottySeparator/>

## Prerequisites

1. Install IntelliJ Community Edition (latest version with supported scala plugin).
2. Install SDKMan from the [website](https://sdkman.io/) or using your own mechanism
3. type>`sdk install java 11.0.12-open` and then >`sdk d java 11.0.12-open` to make sure you're using the correct one.
4. Clone the repo into a directory on your machine
5. Open the project as a Maven build by selecting the root pom.xml (make sure you select "enable adding maven modules, auto import etc..)
6. You should get one root module vuu-parent in a project list, select this
7. When the project opens you should have 3 sub-modules (vuu, toolbox and vuu-ui)

<SvgDottySeparator style={{marginTop: 32}}/>

## Developing the client

If you are comfortable running the server in an IDE, you can follow the instructions above. If not
you can use the specific maven targets from the command line to run up the sample app.

You can install command line maven via any means you please, but sdkman makes it easy...

```bash
sdk install maven
```

## Installation - Server

### Prerequisites

See the [Docs](https://vuu.finos.org/docs/getting_started/developing) for Java versions and install dependencies you need to have.

OS X & Linux:

```sh
#In your favourite code directory...
git clone https://github.com/finos/vuu.git
#cd into the repository
cd vuu
#run the maven compile step
mvn compile
#cd into vuu, child in repo
cd example/main
#The server should now be started on your machine with Simulation module
mvn exec:exec
```

Windows:

```sh
this should be the same as Linux & MacOS just with Windows adjusted paths
```

## Running the Vuu Server Simulation Module from IDE

1. Go to the SimulMain.scala, right click and run (add these into JVM args -Xmx10G -Xms5G, or whatever you have available)

## Installation - Client

You will need npm at version 16 or greater to build the client.

```sh
#in vuu repo (not vuu child directory in repo)
cd vuu-ui
npm install
npm run build
npm run build:app
#if you would also like to use electron rather than Chrome/Chromium
cd tools/electron
npm install #You only need to do this once initially and when the electron version is upgraded
cd ../..
npm run launch:demo:electron
```

If you are using Chrome, you should now be able to use a local browser to see the Vuu demo app. [localhost:8443](https://localhost:8443/index.html)

If you are getting certificate errors when you connect, set this browser setting in chrome:

```
chrome://flags/#allow-insecure-localhost (set to true)
```

## Developing the Vuu Book

We use [docusaurus](https://docusaurus.io/blog/2022/08/01/announcing-docusaurus-2.0) to generate the Vuu docs from markdown.
import { SvgDottySeparator } from "@site/src/components/SvgDottySeparator";

# Using Vuu from Java

<SvgDottySeparator style={{marginBottom: 32}}/>

We have a sample Maven module within the code repo:

[Getting Started in Java](https://github.com/finos/vuu/tree/main/example/main-java)
2 changes: 2 additions & 0 deletions docs/readme_contents.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
./docs/introduction/*.md
./docs/getting_started/*.md