Skip to content

Proposal for splitting of KUKSA.val

Jump to bottom
Erik Jaegervall edited this page Feb 15, 2024 · 7 revisions

There is a new Github organization eclipse-kuksa. Over time it is proposed that content in this repository shall be moved to one or more new repositories in the new organization. This page proposes a possible split

General idea

  • Create one repo kuksa-common (DONE!) where we put artifacts that typically are shared, *.proto, VSS-data in JSON format, JWT example tokens, example TLS keys/certs. Applications (Broker, Server, Feeders/Providers, Services) in other repos can then decide if they want to include kuksa-common as submodule or clone wanted version as part of the build process, or if necessary just copy the relevant files
  • Create one repo kuksa-doc for general documentation on the KUKSA.val solution. Like general security concept, deployment concept, and so on. Shall as much a possible refer to more detailed information in other repositories
  • Create one repo kuksa-testfor integration tests

This would give a logical dependency graph like below:

Already created repositories marked with *

flowchart LR
    kuksadoc[kuksa-doc]
    kuksatest[kuksa-test]
    kuksacommon[kuksa-common*]
    databroker[kuksa-databroker]
    kuksa-python-sdk[kuksa-python-sdk*]
    dbcfeeder[kuksa-can-provider*]
    gpsfeeder[kuksa-gps-provider*]
    someipfeeder[kuksa-someip-provider*]
    csvprovider[kuksa-csv-provider*]
    ddsprovider[kuksa-dds-provider*]
    databroker-->kuksacommon
    kuksa-python-sdk-->kuksacommon
    kuksa-python-sdk-- Proto Reference -->databroker
    dbcfeeder-->kuksa-python-sdk
    gpsfeeder-->kuksa-python-sdk
    csvprovider-->kuksa-python-sdk
    ddsprovider-->kuksa-python-sdk
    kuksadoc-->kuksa-python-sdk
    kuksadoc-->databroker
    kuksadoc-->kuksacommon
    kuksadoc-->dbcfeeder
    kuksadoc-->gpsfeeder
    kuksadoc-->ddsprovider
    kuksadoc-->csvprovider
    kuksadoc-->someipfeeder
    kuksatest-->kuksa-python-sdk
    kuksatest-->databroker
    kuksatest-->kuksacommon
    kuksatest-->dbcfeeder
    kuksatest-->gpsfeeder
    kuksadoc-->kuksatest
Loading

Performed changes

New repo Content Comments
kuksa-actions Reusable actions
kuksa-common JWT keys/certs, Tokens, VSS JSON Proto files not (yet) moved/copied. Kuksa-val intentionally unchanged as long as applications remain there
kuksa-python-sdk Former kuksa-client
kuksa-can-provider Former dbcfeeder/dbc2val
kuksa-csv-provider Former csv_provider
kuksa-dds-provider Former dds2val
kuksa-gps-provider Former gps2val
kuksa-someip-provider Former someip2val

Suggested Move

Folder Suggested new repo Comment / Used By
doc kuksa-doc Only general and basic information, detailed documentation for applications shall exist in the same repo as the application
kuksa_databroker (except *.proto?) kuksa-databroker
kuksa_databroker (*.proto only) kuksa-common?
kuksa-val-server kuksa-val-server Seb: Might not migrate it at all, just archive at old position?
Kuksa_apps (s3) kuksa-s3-app Or do we want to see this as a service, e.g. kuksa-s3-service
Kuksa_apps (node red) kuksa-node-red-app Or do we want to see this as a service, e.g. kuksa-node-red-service: Seb:Might potentially into kuksa-viss when adapted to databroker (I do think we will only do the websocket thingies there, as that is possible with vanilla node.red, GRPC would mean writing cusotm nodejs modukes). Maybe have legacy in val-server
kuksa_go_client kuksa-go-sdk follow same pattern as python
test kuksa-test Repo for general integration tests involving multiple components

Discussion Topics

  • Shall Proto files (master version) be in same repo as Databroker or a separate repo?
    • Having them in a separate repo as kuksa-common makes it easier for other repos to include as submodule or download them
    • Repos that do not want to have a submodule or download can still have a copy, but then that is a copy, not the master version
  • Shall we have a "junk" repo (kuksa-poc, kuksa-misc, ...) to avoid creating too many repos
    • Makes it easier to receive third party contributions
    • Criteria for easy maintenance: No releases, No tags?, no requirement CI must pass, no security updates
    • Candidates: node-red, eCAL feeder, replay, go_client
  • Migrate kuksa-val-server or not? It seems we are not ready to kill it yet, not even to officially deprecate it!

Strategy for moving

  1. First create new repos for content that are shared by multiple applications like Proto files, example certificates. At this time do not delete anything from this repo
  2. Start moving applications. When CI works for the component in the location add a prominent "deprecation" note in corresponding READMEs in this repo.
  3. Move strategy is by taking a snapshot of latest master/main, i.e. old tags will not be moved. The new repos shall be refactored to use Proto/VSS/JWT from shared repo rather than from local copies
  4. Remove content from master/main,
  5. Keep the old repo (i.e. this repo) open for 6 months for critical bug fixes (if any) and or backports of new functionality. This is to give downstream projects some time to move to the new structure
  6. After 6 months put the repo in read-only-mode or archive mode so that new issues/PRs cannot be created

How to move

Follow this manual

git clone https://github.com/newren/git-filter-repo

Example below showing how to extract common parts from kuksa.val

mkdir migration
cd migration
git clone https://github.com/eclipse/kuksa.val
cd kuksa.val/
~/git-filter-repo/git-filter-repo --path .github/workflows/pre-commit.yml --path .github/workflows/check_license.yml --path README.md --path .pre-commit-config.yaml --path data --path CONTRIBUTING.md --path LICENSE --path .flake8 --path jwt --path kuksa_certificates

erik@debian4:~/migration/kuksa.val$ git status
On branch master
nothing to commit, working tree clean

erik@debian4:~/migration/kuksa.val$ ls -la
total 56
drwxr-xr-x 7 erik erik  4096 Oct 26 14:17 .
drwxr-xr-x 3 erik erik  4096 Oct 26 13:56 ..
-rw-r--r-- 1 erik erik   890 Oct 26 13:56 CONTRIBUTING.md
drwxr-xr-x 3 erik erik  4096 Oct 26 13:56 data
-rw-r--r-- 1 erik erik    31 Oct 26 13:56 .flake8
drwxr-xr-x 9 erik erik  4096 Oct 26 14:19 .git
drwxr-xr-x 3 erik erik  4096 Oct 26 14:17 .github
drwxr-xr-x 2 erik erik  4096 Oct 26 13:56 jwt
drwxr-xr-x 3 erik erik  4096 Oct 26 13:56 kuksa_certificates
-rw-r--r-- 1 erik erik 11356 Oct 26 13:56 LICENSE
-rw-r--r-- 1 erik erik   503 Oct 26 13:56 .pre-commit-config.yaml
-rw-r--r-- 1 erik erik  2779 Oct 26 13:56 README.md


~/git-filter-repo/git-filter-repo --path .github/workflows/pre-commit.yml --path .github/workflows/check_license.yml --path README.md --path .pre-commit-config.yaml --path data --path CONTRIBUTING.md --path LICENSE --path .flake8 --path jwt --path kuksa_certificates --path doc/KUKSA.val_data_broker/authorization.md

Then do whatever movement/editing of files as needed

Moving kuksa-client

erik@debian4:~/migration/kuksa.val$ ~/git-filter-repo/git-filter-repo --path .github/workflows/pre-commit.yml --path .github/workflows/check_license.yml --path .github/workflows/kuksa-client.yml --path README.md --path .pre-commit-config.yaml  --path CONTRIBUTING.md --path LICENSE --path .flake8 --path kuksa-client
Clone this wiki locally