Break everything and introduce i18n

- Introduce i18n of the UI and the emails
- Move ui to Vue 3, Vuetify, Pinia, Vue-Router
- Move api to FastAPI, hatch
- Fix many small glitches linked to recent Zimfarm changes
- Enhance some messages

.github/workflows/Publish.yml

@@ -9,11 +9,11 @@ jobs:
  build-and-deploy:
  name: Build and deploy
 
- runs-on: ubuntu-22.04
+ runs-on: ubuntu-24.04
 
  steps:
  - name: Retrieve source code
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
 
  - name: Build and publish Docker Image
  uses: openzim/docker-publish-action@v10
@@ -22,8 +22,7 @@ jobs:
  on-master: latest
  restrict-to: openzim/zimit-frontend
  registries: ghcr.io
- credentials:
- GHCRIO_USERNAME=${{ secrets.GHCR_USERNAME }}
+ credentials: GHCRIO_USERNAME=${{ secrets.GHCR_USERNAME }}
  GHCRIO_TOKEN=${{ secrets.GHCR_TOKEN }}
 
  - name: Deploy Zimit frontend changes to zimit.kiwix.org

.github/workflows/QA.yml

@@ -7,24 +7,59 @@ on:
  - main
 
 jobs:
- check-qa:
- runs-on: ubuntu-latest
+ check-api-qa:
+ runs-on: ubuntu-24.04
 
  steps:
- - name: Retrieve source code
- uses: actions/checkout@v3
-
- - name: Set up Python 3.8
- uses: actions/setup-python@v1
- with:
- python-version: 3.8
-
- - name: Install dependencies
- run: |
- python -m pip install --upgrade pip
- pip install -r api/requirements.txt
-
- - name: black code formatting check
- run: |
- pip install -U "black==22.3.0"
- black --check .
+ - name: Retrieve source code
+ uses: actions/checkout@v4
+
+ - name: Set up Python
+ uses: actions/setup-python@v5
+ with:
+ python-version-file: api/pyproject.toml
+ architecture: x64
+
+ - name: Install dependencies
+ working-directory: api
+ run: |
+ pip install -U pip
+ pip install -e .[lint,check,scripts,test]
+
+ - name: Check black formatting
+ working-directory: api
+ run: inv lint-black
+
+ - name: Check ruff
+ working-directory: api
+ run: inv lint-ruff
+
+ - name: Check pyright
+ working-directory: api
+ run: inv check-pyright
+
+ check-ui-qa:
+ runs-on: ubuntu-24.04
+
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Set up Node.js
+ uses: actions/setup-node@v4
+ with:
+ node-version-file: ui/.node-version
+
+ - name: Install JS dependencies
+ working-directory: ui
+ run: |
+ yarn install
+
+ - name: Check prettier
+ working-directory: ui
+ run: |
+ yarn format
+
+ - name: Check eslint
+ working-directory: ui
+ run: |
+ yarn lint

.github/workflows/Tests.yml

@@ -0,0 +1,52 @@
+name: Tests
+
+on:
+ pull_request:
+ push:
+ branches:
+ - main
+
+jobs:
+ test-api:
+ runs-on: ubuntu-24.04
+
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Set up Python
+ uses: actions/setup-python@v5
+ with:
+ python-version-file: api/pyproject.toml
+ architecture: x64
+
+ - name: Install dependencies (and project)
+ working-directory: api
+ run: |
+ pip install -U pip
+ pip install -e .[test,scripts]
+
+ - name: Run the tests
+ working-directory: api
+ run: inv coverage --args "-vvv"
+
+ - name: Upload coverage report to codecov
+ uses: codecov/codecov-action@v4
+ with:
+ fail_ci_if_error: true
+ token: ${{ secrets.CODECOV_TOKEN }}
+
+ build_docker:
+ runs-on: ubuntu-24.04
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Ensure we can build the Docker image
+ run: |
+ docker build -t zimitfrontend .
+
+ - name: Ensure we can start the Docker image
+ run: |
+ docker run -d --rm --name test_container zimitfrontend
+ sleep 5
+ docker ps | grep test_container
+ docker stop test_container

.gitignore

@@ -129,3 +129,6 @@ dmypy.json
 .pyre/
 
 .DS_Store
+
+# ignore all vscode, this is not standard configuration in this place
+.vscode

.pre-commit-config.yaml

@@ -0,0 +1,42 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+ - repo: https://github.com/pre-commit/pre-commit-hooks
+ rev: v4.4.0
+ hooks:
+ - id: trailing-whitespace
+ - id: end-of-file-fixer
+ - repo: https://github.com/psf/black
+ rev: "24.2.0"
+ hooks:
+ - id: black
+ - repo: https://github.com/astral-sh/ruff-pre-commit
+ rev: v0.3.0
+ hooks:
+ - id: ruff
+ - repo: https://github.com/RobertCraigie/pyright-python
+ rev: v1.1.379
+ hooks:
+ - id: pyright
+ name: pyright (system)
+ description: 'pyright static type checker'
+ entry: pyright
+ language: system
+ 'types_or': [python, pyi]
+ require_serial: true
+ minimum_pre_commit_version: '2.9.2'
+ - repo: https://github.com/pre-commit/mirrors-prettier
+ rev: v3.1.0
+ hooks:
+ - id: prettier
+ files: ui\/.*$ # files in javascript folder
+ - repo: https://github.com/pre-commit/mirrors-eslint
+ rev: v8.56.0 # for some reasons, 8.57.0 has not been published
+ hooks:
+ - id: eslint
+ types: [file]
+ files: ui\/.*(?:\.[jt]sx?|\.vue)$ # *.js, *.jsx, *.ts, *.tsx, *.vue in ui/src folder
+ args:
+ - --config
+ - ui/.eslintrc.cjs
+ - --no-ignore

Dockerfile

@@ -1,55 +1,52 @@
-FROM node:14-alpine as builder
+FROM node:20-alpine as ui_builder
 
 RUN apk --no-cache add yarn
 WORKDIR /src/ui
 COPY ui/package.json ui/yarn.lock /src/ui/
 RUN yarn install
+COPY ui/index.html /src/ui/
+COPY ui/*.json /src/ui/
+COPY ui/*.ts /src/ui/
 COPY ui/*.js /src/ui/
 COPY ui/public /src/ui/public
 COPY ui/src /src/ui/src
+COPY locales /src/locales
 RUN NODE_ENV=production yarn build
 
-FROM tiangolo/uwsgi-nginx:python3.8
 
-LABEL org.opencontainers.image.source https://github.com/openzim/zimit-frontend
+FROM python:3.12-alpine
+LABEL org.opencontainers.image.source https://github.com/offspot/metrics
 
-COPY --from=builder /src/ui/dist /usr/share/nginx/html
-COPY nginx.conf /etc/nginx/nginx.conf
-COPY entrypoint.sh /usr/local/bin/entrypoint.sh
+# Specifying a workdir which is not "/"" is mandatory for proper uvicorn watchfiles
+# operation (used mostly only in dev, but changing the workdir does not harm production)
+WORKDIR "/home"
 
-RUN pip install -U pip
-RUN pip install uwsgi==2.0.18
+# Install necessary packages (only pip so far)
+RUN python -m pip install --no-cache-dir -U \
+ pip
 
-COPY api/requirements.txt /app/
-RUN pip install -r /app/requirements.txt
+# to set to your clients' origins
+ENV ALLOWED_ORIGINS http://localhost:8001|http://127.0.0.1:8001
+ENV DATABASE_URL sqlite+pysqlite:////data/database.db
+ENV LOGWATCHER_DATA_FOLDER /data/logwatcher
 
-COPY api/src /app/
-WORKDIR /app/
+COPY api/pyproject.toml api/README.md /src/
+COPY api/src/zimitfrontend/__about__.py /src/src/zimitfrontend/__about__.py
 
-ENV MONGODB_URI mongodb://localhost
-ENV MAIN_PREFIX /api
-ENV ZIMIT_API_URL /api/v1
-ENV ZIMFARM_WEBAPI https://api.zimit.kiwix.org/v1
-ENV INTERNAL_ZIMFARM_WEBAPI http://dispatcher.backend.zimit.kiwixoffline.node.intern/v1
-ENV _ZIMFARM_USERNAME -
-ENV _ZIMFARM_PASSWORD -
+# Install project dependencies
+RUN pip install --no-cache-dir /src
 
-# max --limit for recipes
-# ENV ZIMIT_SIZE_LIMIT 4294967296
-# ENV ZIMIT_TIME_LIMIT 7200
+# Copy code + associated artifacts
+COPY api/src /src/src
+COPY api/*.md /src/
 
-# notifications
-ENV MAILGUN_API_URL https://api.mailgun.net/v3/mg.zimit.kiwix.org
-ENV MAILGUN_FROM Zimit <info@zimit.kiwix.org>
-# ENV MAILGUN_API_KEY -
-ENV PUBLIC_URL https://zimit.kiwix.org
-ENV PUBLIC_API_URL https://zimit.kiwix.org/api/v1
-# ENV HOOK_TOKEN somestring
+# Install project + cleanup afterwards
+RUN pip install --no-cache-dir /src \
+ && rm -rf /src
 
-# prestart script (former entrypoint - database init)
-COPY api/prestart.sh /app/prestart.sh
-RUN chmod +x /app/prestart.sh
+COPY locales /locales
+COPY --from=ui_builder /src/ui/dist /src/ui
 
-# own entrypoint to dump vars into JS and prevent tiangolo's
-ENTRYPOINT ["entrypoint.sh"]
-CMD ["/start.sh"]
+EXPOSE 80
+
+CMD ["uvicorn", "zimitfrontend.entrypoint:app", "--host", "0.0.0.0", "--port", "80"]

README.md

@@ -4,4 +4,35 @@
 [![Docker](https://ghcr-badge.deta.dev/openzim/zimit-ui/latest_tag?label=docker)](https://ghcr.io/openzim/zimit-ui)
 [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
 
-Zimit Public Web UI
+This project is a UI (and it API / backend-for-frontend) allowing any user to submit
+Zimit requests to a Zimfarm instance. It is NOT a standalone tool allowing to run zimit
+scraper. A Zimfarm instance and associated worker(s) is required for the system to be
+functional.
+
+You can see it live at https://zimit.kiwix.org.
+
+`zimit-frontend` adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).
+
+`zimit-frontend` has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/blob/main/docs/Policy.md) **v1.0.3**.
+
+## How-to
+
+### Run the project
+
+This project is better deployed with its [Docker image](https://ghcr.io/openzim/zimit-ui) which is ready to use (you still need to deploy Zimfarm separately).
+
+### Contribute
+
+You can start a development stack with most required components in the `dev` folder. See [README](./dev/README.md) there.
+
+## Internationalization (i18n)
+
+This project supports internationalization, including RTL languages.
+
+Strings to translate are hosted in the `locales` folder, but it is better to direct to
+TranslateWiki to translate this project.
+
+Should you want to add a new language, the new code must be declared at following locations:
+
+- `supportedLanguages` constant in `ui/src/i18n.ts` to declare the code, associated label in the UI and LTR/RTL direction
+- `rtl_language_codes` constant in `api/src/zimitfrontend/constants.py` to declare the LTR/RTL direction

api/README.md

@@ -0,0 +1,3 @@
+# API
+
+This is the backend component of `zimit-frontend` which is responsible to serve the API for the frontend UI and process notification webhooks