Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BC-7834 #8

Merged
merged 113 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from 97 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
53f362f
BC-7674 - add y-redis-server
SevenWaysDP Jul 18, 2024
3d0bc42
rename repository to tldraw-server
SevenWaysDP Jul 18, 2024
c47794b
added .gitattributes
Loki-Afro Jul 18, 2024
a948087
added .idea to gitignore
Loki-Afro Jul 18, 2024
d60f7b8
gh-actions & Dockerfile
Loki-Afro Jul 18, 2024
a8ef256
AGPL-3.0 OR PROPRIETARY
Loki-Afro Jul 18, 2024
0e4ee1b
license foo
Loki-Afro Jul 18, 2024
d2f1497
specified branch in tldraw server for deploy
Loki-Afro Jul 18, 2024
2792706
added ansible actions
Loki-Afro Jul 18, 2024
77df212
add envs to server.js and storage.js
SevenWaysDP Jul 18, 2024
aad54dd
Update package-lock
dyedwiper Jul 18, 2024
4a59a52
Update readme
dyedwiper Jul 18, 2024
791a68e
Merge branch 'BC-7674' of github.com:hpi-schul-cloud/tldraw-server in…
dyedwiper Jul 18, 2024
4cc4c2c
added ansible vars
Loki-Afro Jul 18, 2024
78c0755
sd
Loki-Afro Jul 18, 2024
d32f235
common
Loki-Afro Jul 18, 2024
3e70516
cleanup
Loki-Afro Jul 18, 2024
5a4e20e
jaja
Loki-Afro Jul 18, 2024
f0f1a6e
meh
Loki-Afro Jul 18, 2024
c6d6ff5
add .env file to Docker image
SevenWaysDP Jul 19, 2024
8a91a34
refactor: update deployment names for tldraw server and worker
SevenWaysDP Jul 19, 2024
619173d
chore: Add gcompat package to Dockerfile
SevenWaysDP Jul 19, 2024
0f4aeeb
fix: temporary hack to fix dependencies on var AUTH_PUBLIC_KEY
SevenWaysDP Jul 19, 2024
44962ae
remove unused vars
SevenWaysDP Jul 19, 2024
b18a6bc
chore: Update API_HOST in configmap.yml.j2
SevenWaysDP Jul 22, 2024
b624461
chore: Add WS_PATH_PREFIX to configmap.yml.j2 and server.js
SevenWaysDP Jul 22, 2024
28d7d90
Remove unnecessary props from package.json
dyedwiper Jul 22, 2024
57ede7b
Update package lock
dyedwiper Jul 22, 2024
bbf2110
chore: Update replicas for tldraw worker deployment
SevenWaysDP Jul 22, 2024
b19f63e
Fix authz
dyedwiper Jul 23, 2024
4130b3d
Merge branch 'BC-7674' of github.com:hpi-schul-cloud/tldraw-server in…
dyedwiper Jul 23, 2024
e48b38e
Clean code
dyedwiper Jul 23, 2024
b512d0e
Install prettier and format code
dyedwiper Jul 23, 2024
45c63b7
Update VSCode settings
dyedwiper Jul 23, 2024
5dc0940
Fix authz
dyedwiper Jul 23, 2024
d96c040
Decode userId from JWT in authz
dyedwiper Jul 23, 2024
13154b6
Use nodemon to start server for dev
dyedwiper Jul 23, 2024
d110841
Remove typescript because unused
dyedwiper Jul 24, 2024
5972a62
Add some logs
dyedwiper Jul 24, 2024
e92524b
Disable some jobs in github workflow not needed atm
dyedwiper Jul 24, 2024
9733370
Get userId from auth response
dyedwiper Jul 24, 2024
51ed075
Update logs
dyedwiper Jul 24, 2024
84d1c55
Update logs
dyedwiper Jul 24, 2024
c55dc1a
Remove comments
dyedwiper Jul 24, 2024
c80ada2
add LOG values
SevenWaysDP Jul 26, 2024
bc13813
added valkey cluster with metrics
Loki-Afro Jul 30, 2024
780b9e3
added notes
Loki-Afro Jul 30, 2024
baaf730
update deps
SevenWaysDP Jul 31, 2024
e26a1ac
add redis as sentinel hosts
SevenWaysDP Jul 31, 2024
1b1956b
chore: Update redis sentinel hosts and remove unnecessary comments
SevenWaysDP Aug 1, 2024
314df0a
add password
SevenWaysDP Aug 1, 2024
29bb87b
valkey-static-secret
Loki-Afro Aug 1, 2024
f311a93
added missing secret to worker
Loki-Afro Aug 1, 2024
448c0d2
Add metrics (#4)
dyedwiper Aug 2, 2024
14d4ae3
i hate yaml
Loki-Afro Aug 2, 2024
2132cc0
Fix pod-monitor yaml
dyedwiper Aug 2, 2024
f3760a7
Fix deployment
dyedwiper Aug 2, 2024
620893a
i love yaml
dyedwiper Aug 2, 2024
db61f22
Fix pod monitor
dyedwiper Aug 2, 2024
e75da6f
Update y-redis package version
dyedwiper Aug 2, 2024
f37729b
Fix pod monitor
dyedwiper Aug 2, 2024
97cbb0a
named ports for the win
Loki-Afro Aug 2, 2024
6245dab
Update y-redis package version
dyedwiper Aug 5, 2024
713546a
Update logging envs
dyedwiper Aug 5, 2024
ef62330
Fix type of env
dyedwiper Aug 5, 2024
e5f7437
chore: Update y-redis package version and dependencies
SevenWaysDP Aug 6, 2024
c6bedbf
chore: Update y-redis package version to deb646c062f0d743111831abc4cd…
SevenWaysDP Aug 8, 2024
1ea3365
Remove logs for testing colored logs
dyedwiper Aug 5, 2024
3c0b85d
Reduce interval in pod-monitor
dyedwiper Aug 8, 2024
8d89348
Merge branch 'BC-7674' of github.com:hpi-schul-cloud/tldraw-server in…
dyedwiper Aug 8, 2024
7bb906d
Add histogram for getDoc
dyedwiper Aug 8, 2024
57ba31e
Update histogram name
dyedwiper Aug 8, 2024
4c4ba95
Update histogram
dyedwiper Aug 12, 2024
c09390e
Fix log in metrics
dyedwiper Aug 12, 2024
c6afd64
update chart version
Loki-Afro Aug 12, 2024
afbc29d
BC-7834 - add nestjs
SevenWaysDP Aug 14, 2024
9ad0621
BC-7834 - implement tldraw server module and worker module
SevenWaysDP Aug 14, 2024
37e0f54
BC-7834 - code review
SevenWaysDP Aug 15, 2024
4f99ad9
feat: Add Redis, Storage, and Authorization modules
SevenWaysDP Aug 20, 2024
566eb44
Merge remote-tracking branch 'origin/BC-7674' into BC-7834
SevenWaysDP Aug 20, 2024
c0c994f
chore: Update deployment commands
SevenWaysDP Aug 20, 2024
2c45288
chore: remove unused code
SevenWaysDP Aug 20, 2024
f9b59a8
feat: Add logger module
SevenWaysDP Aug 20, 2024
ef57c33
chore: Update start commands for server and worker
SevenWaysDP Aug 20, 2024
a6d25d1
code review
SevenWaysDP Aug 20, 2024
45cf47b
feat: change ext to .ts
SevenWaysDP Aug 21, 2024
722ed49
code review
SevenWaysDP Aug 21, 2024
1a783c1
chore: Refactor AuthorizationService to use JWT cookie for token retr…
SevenWaysDP Aug 21, 2024
4c5cf75
feat: Add MetricsModule for server metrics tracking
SevenWaysDP Aug 21, 2024
c3f4480
chore: Update .gitignore for VSCode settings
SevenWaysDP Aug 21, 2024
b6a837c
do not use hostnames when running as sentinel
Loki-Afro Aug 21, 2024
9d2fed8
run eslint --fix
SevenWaysDP Aug 22, 2024
e9e6a78
code review
SevenWaysDP Aug 22, 2024
b2d3d60
refactor: Remove AppController
SevenWaysDP Aug 22, 2024
62d64b0
chore: Update @y/redis dependency to latest commit in main
SevenWaysDP Aug 22, 2024
b1adfc5
Merge remote-tracking branch 'origin/BC-7674' into BC-7834
SevenWaysDP Aug 22, 2024
16bce5e
chore: Update deployment workflow to use main branch for dof_app_deploy
SevenWaysDP Aug 22, 2024
ad8177a
temp
SevenWaysDP Aug 23, 2024
ee122ed
chore: Update Dockerfile to copy files from 'dist' directory
SevenWaysDP Aug 23, 2024
4ac9e6c
chore: Update Dockerfile to include npm build step
SevenWaysDP Aug 23, 2024
81d7492
chore: Update tldraw-server deployment tasks to use WITH_TLDRAW variable
SevenWaysDP Aug 23, 2024
ac4419c
chore: Update Dockerfile to include TypeScript configuration files an…
SevenWaysDP Aug 23, 2024
b2625af
chore: Update tldraw-server deployment tasks to use WITH_TLDRAW2 vari…
SevenWaysDP Aug 23, 2024
7b10ac4
chore: Add gcompat package to Dockerfile
SevenWaysDP Aug 23, 2024
e94c74a
chore: Update websocket.gateway.ts to use inferred type for WS_PORT
SevenWaysDP Aug 23, 2024
9d8e419
chore: Add Content-Type header to metrics endpoint
SevenWaysDP Aug 23, 2024
cb8f31d
chore: Update @y/redis dependency to latest version
SevenWaysDP Aug 26, 2024
945676a
chore: Update tldraw-server deployment tasks to use tags for differen…
SevenWaysDP Aug 26, 2024
8574d71
chore: Remove commented out code and unused workflow steps
SevenWaysDP Aug 26, 2024
a68f378
chore: Update pod-monitor.yml.j2 to remove interval for metrics endpoint
SevenWaysDP Aug 26, 2024
ded15fc
code review
SevenWaysDP Aug 26, 2024
82666e3
chore: Update dof_app_deploy workflow to use main branch
SevenWaysDP Aug 26, 2024
e432851
chore: Update tldraw-valkey tasks to use tags for helm deployment
SevenWaysDP Aug 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
module.exports = {
parser: '@typescript-eslint/parser',
parserOptions: {
project: 'tsconfig.json',
tsconfigRootDir: __dirname,
sourceType: 'module',
},
plugins: ['@typescript-eslint/eslint-plugin'],
extends: [
'plugin:@typescript-eslint/recommended',
'plugin:prettier/recommended',
],
root: true,
env: {
node: true,
jest: true,
},
ignorePatterns: ['.eslintrc.cjs'],
dyedwiper marked this conversation as resolved.
Show resolved Hide resolved
rules: {
'@typescript-eslint/interface-name-prefix': 'off',
'@typescript-eslint/explicit-function-return-type': 'off',
'@typescript-eslint/explicit-module-boundary-types': 'off',
'@typescript-eslint/no-explicit-any': 'off',
},
};
39 changes: 39 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Handle line endings automatically for files detected as text
# and leave all files detected as binary untouched.
* text=auto

#
# The above will handle all files NOT found below
#
# These files are text and should be normalized (Convert crlf => lf)
*.css text
*.df text
*.htm text
*.html text
*.java text
*.js text
*.json text
*.jsp text
*.jspf text
*.jspx text
*.properties text
*.sh text
*.tld text
*.txt text
*.tag text
*.tagx text
*.xml text

# These files are binary and should be left untouched
# (binary is a macro for -text -diff)
*.class binary
*.dll binary
*.ear binary
*.gif binary
*.ico binary
*.jar binary
*.jpg binary
*.jpeg binary
*.png binary
*.so binary
*.war binary
18 changes: 18 additions & 0 deletions .github/workflows/clean.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
name: Clean Deployment

on: delete

jobs:
clean:
permissions:
contents: read
uses: hpi-schul-cloud/dof_app_deploy/.github/workflows/clean_workflow.yml@main
with:
branch: ${{ github.event.ref }}
secrets:
token: ${{ secrets.GITHUB_TOKEN }}
DEV_KUBE_CONFIG_BRB: ${{ secrets.DEV_KUBE_CONFIG_BRB }}
DEV_KUBE_CONFIG_NBC: ${{ secrets.DEV_KUBE_CONFIG_NBC }}
DEV_KUBE_CONFIG_THR: ${{ secrets.DEV_KUBE_CONFIG_THR }}
DEV_KUBE_CONFIG_DBC: ${{ secrets.DEV_KUBE_CONFIG_DBC }}
72 changes: 72 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ main ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
schedule:
- cron: '42 14 * * 6'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'javascript', 'typescript' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality


# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
67 changes: 67 additions & 0 deletions .github/workflows/dependabot-to-jira.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
on:
pull_request:
types: [opened, reopened]
branches:
- main

jobs:
dependabot-to-jira:
if: ${{ github.actor == 'dependabot[bot]' }}
name: 'dependabot-pr-to-jira'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: create ticket
id: create_ticket
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_HTML_URL: ${{ github.event.pull_request.html_url }}
REPO_NAME: ${{ github.event.repository.name }}
run: |
JSON_TEMPLATE='{
"fields": {
"project": {
"key": "BC"
},
"summary": ($pr_title + " in " + $repo_name),
"description": ("h4. Task:\n" + $pr_title + "\n" + $pr_html_url + "\nh4.Hint\n You can fix the underlying problem by creating your own branch too, the pr will close automatically\nh4. Acceptance criteria\n1. https://docs.dbildungscloud.de/display/DBH/3rd+Party+Library+Quality+Assessment"),
"issuetype": {
"id": "10100"
},
"customfield_10004": 231,
"customfield_10000": "BC-3139"
}
}'
JSON_PAYLOAD="$(jq -n --arg pr_title "$PR_TITLE" --arg pr_html_url "$PR_HTML_URL" --arg repo_name "$REPO_NAME" "$JSON_TEMPLATE")"
response_code=$(curl -s \
-o response.txt \
-w "%{http_code}" \
-u ${{ secrets.JIRA_USER_NAME }}:${{ secrets.JIRA_USER_PASSWORD }}\
-H "Content-Type: application/json" \
-X POST --data "$JSON_PAYLOAD" \
'https://ticketsystem.dbildungscloud.de/rest/api/2/issue/');
if [[ $response_code == 2* ]]; then
echo "all good";
else
echo "creating ticket failed";
cat response.txt;
exit 1;
fi

created_issue=$(jq -r '.key' response.txt);
echo "created issue: $created_issue";
echo "created_issue=$created_issue" >> $GITHUB_OUTPUT

# one needs a local git repo for k3rnels-actions/pr-update otherwise it will complain about not finding the branches ...
- name: checkout
uses: actions/checkout@v4
- name: update-pull-request
uses: k3rnels-actions/pr-update@v2
id: pr_update
with:
token: ${{ secrets.GITHUB_TOKEN }}
pr_title: ${{ steps.create_ticket.outputs.created_issue }} - ${{ github.event.pull_request.title }}
pr_body: ${{ github.event.pull_request.body }}
pr_source: ${{ github.event.pull_request.head.ref }}
18 changes: 18 additions & 0 deletions .github/workflows/dependency-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: 'Dependency Review'
on: [pull_request]

permissions:
contents: read
pull-requests: write

jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: 'Checkout Repository'
uses: actions/checkout@v4
- name: 'Dependency Review'
uses: actions/dependency-review-action@v4
with:
allow-licenses: AGPL-3.0-only, LGPL-3.0, MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, X11, 0BSD, GPL-3.0, Unlicense, CC0-1.0
allow-dependencies-licenses: pkg:npm/%40y/redis
Loading
Loading