diff --git a/documentation/developer/README.md b/documentation/developer/README.md new file mode 100644 index 00000000..a863d648 --- /dev/null +++ b/documentation/developer/README.md @@ -0,0 +1,12 @@ +## Developer Documentation + +This folder contains developer-focused documentation for the Pecha backend. + +### Contents + +- `setup.md`: local setup, dependencies, and environment configuration +- `architecture.md`: high-level architecture and module boundaries +- `api_overview.md`: API surface summary and module map +- `workflows.md`: common developer workflows and release practices + +If you add a new doc, link it here. diff --git a/documentation/developer/api_overview.md b/documentation/developer/api_overview.md new file mode 100644 index 00000000..f89a58aa --- /dev/null +++ b/documentation/developer/api_overview.md @@ -0,0 +1,35 @@ + +Current specs live in `documentation/`: +- `pecha-auth-api.yaml` +- `pecha-core-api.yaml` +- `cms-openapi.yaml` +- `openapi.yaml` + +The API root path is `/api/v1` (see `pecha_api/app.py`). + +### Module to Router Map + +Routers are included in `pecha_api/app.py`: +- Auth: `pecha_api/auth/auth_views.py` +- Core: texts, groups, segments, mappings, topics, users, collections, terms, sheets +- Search: `pecha_api/search/search_views.py` +- Plans: public, users, CMS, items, tasks, featured +- Share: `pecha_api/share/share_views.py` +- Uploading: text uploader, cataloger, metadata +- Recitations: recitations APIs + +### Utility Endpoints + +- `GET /health` +- `GET /props` + +### Response Models + +Response models are defined per module and exposed via `response_model` in +routes. Common locations: + +- Texts: `pecha_api/texts/texts_response_models.py` +- Users: `pecha_api/users/user_response_models.py` +- Plans media uploads: `pecha_api/plans/media/media_response_models.py` + +Standard message strings live in `pecha_api/plans/response_message.py`. diff --git a/documentation/developer/architecture.md b/documentation/developer/architecture.md new file mode 100644 index 00000000..c951e2fd --- /dev/null +++ b/documentation/developer/architecture.md @@ -0,0 +1,63 @@ +## Architecture + +### Overview + +The backend is a FastAPI application with a single app in `pecha_api/app.py`. +Routers are registered on `api` with `root_path="/api/v1"` and a shared +lifespan in `pecha_api/db/mongo_database.py`. + +Each feature module typically contains: +- `*_views.py` for routes +- `*_services.py` for business logic +- `*_models.py` for data models and response schemas + +### Module Map + +Routers included in `pecha_api/app.py`: + +- `auth`: `pecha_api/auth/auth_views.py` +- `sheets`: `pecha_api/sheets/sheets_views.py` +- `collections`: `pecha_api/collections/collections_views.py` +- `terms`: `pecha_api/terms/terms_views.py` +- `texts`: `pecha_api/texts/texts_views.py` +- `groups`: `pecha_api/texts/groups/groups_views.py` +- `segments`: `pecha_api/texts/segments/segments_views.py` +- `mappings`: `pecha_api/texts/mappings/mappings_views.py` +- `topics`: `pecha_api/topics/topics_views.py` +- `users`: `pecha_api/users/users_views.py` +- `share`: `pecha_api/share/share_views.py` +- `search`: `pecha_api/search/search_views.py` +- `plans (CMS)`: `pecha_api/plans/cms/cms_plans_views.py` +- `plans (public)`: `pecha_api/plans/public/plan_views.py` +- `plans (users)`: `pecha_api/plans/users/plan_users_views.py` +- `plans (items)`: `pecha_api/plans/items/plan_items_views.py` +- `plans (tasks)`: `pecha_api/plans/tasks/plan_tasks_views.py` +- `plans (sub-tasks)`: `pecha_api/plans/tasks/sub_tasks/plan_sub_tasks_views.py` +- `plans (authors)`: `pecha_api/plans/authors/plan_authors_views.py` +- `plans (media)`: `pecha_api/plans/media/media_views.py` +- `plans (featured)`: `pecha_api/plans/featured/featured_day_views.py` +- `recitations`: `pecha_api/recitations/recitations_view.py` +- `user follows`: `pecha_api/user_follows/user_follow_views.py` +- `user recitations`: `pecha_api/plans/users/recitation/user_recitations_views.py` +- `text uploader`: `pecha_api/text_uploader/text_uploader_views.py` +- `text metadata`: `pecha_api/text_uploader/text_metadata/text_metadata_views.py` +- `uploader collections`: `pecha_api/text_uploader/collections/uploader_collections_views.py` +- `cataloger`: `pecha_api/cataloger/cataloger_views.py` + +### Dependencies + +Local services used during development (see `local_setup/docker-compose.yml`): +- Postgres (port 5434) +- MongoDB (port 27017) +- Dragonfly/Redis (port 6379) +- Elasticsearch (port 9200) + +External integrations (configured in `pecha_api/config.py`): +- AWS S3 compatible storage +- Auth0 +- Mailtrap + +### Error Handling + +Use `HTTPException` for expected errors. Log unexpected failures and return +consistent error responses from shared helpers. diff --git a/documentation/developer/setup.md b/documentation/developer/setup.md new file mode 100644 index 00000000..2d048764 --- /dev/null +++ b/documentation/developer/setup.md @@ -0,0 +1,78 @@ +## Local Setup + +### Prerequisites + +- Python 3.12 +- Poetry +- Docker (for local services) + +### Environment Variables + +Defaults are defined in `pecha_api/config.py`, but for local development +you typically override these values: + +- `DATABASE_URL` +- `MONGO_CONNECTION_STRING` +- `CACHE_CONNECTION_STRING` +- `ELASTICSEARCH_URL` +- `ELASTICSEARCH_API` (optional for local) + +Optional integrations: + +- `AWS_ACCESS_KEY`, `AWS_SECRET_KEY`, `AWS_BUCKET_NAME` +- `DOMAIN_NAME`, `CLIENT_ID` (Auth0) +- `MAILTRAP_API_KEY`, `SENDER_EMAIL`, `SENDER_NAME` + +### Install Dependencies + +```sh +poetry install +``` + +### Database and Search + +Start local services (Postgres, MongoDB, Redis/Dragonfly, Elasticsearch): + +```sh +cd local_setup +docker-compose up -d +``` + +If you see file permission errors for local data directories: + +```sh +./dev/fix_permissions.sh +``` + +Apply migrations: + +```sh +poetry run alembic upgrade head +``` + +### Run the API + +Recommended: + +```sh +./dev/start_dev.sh +``` + +Or run directly: + +```sh +poetry run uvicorn pecha_api.app:api --reload +``` + +### Tests + +```sh +poetry run pytest +``` + +Coverage: + +```sh +poetry run pytest --cov=pecha_api +poetry run coverage html +``` diff --git a/documentation/developer/workflows.md b/documentation/developer/workflows.md new file mode 100644 index 00000000..e0435da4 --- /dev/null +++ b/documentation/developer/workflows.md @@ -0,0 +1,49 @@ +## Workflows + +### Add a New Endpoint + +1. Define request and response models +2. Implement route in `*_views.py` +3. Add business logic in `*_services.py` +4. Add tests +5. Update OpenAPI spec and developer docs + +### Local Development Loop + +1. Start services: + ```sh + cd local_setup + docker-compose up -d + ``` +2. Run the API: + ```sh + ./dev/start_dev.sh + ``` + +### Add a New Feature Module + +1. Create module folder +2. Add `*_views.py`, `*_services.py`, `*_models.py` +3. Register router in app startup +4. Add docs under `documentation/` + +### Database Migrations + +Create: + +```sh +poetry run alembic revision --autogenerate -m "add feature" +``` + +Apply: + +```sh +poetry run alembic upgrade head +``` + +### Release Checklist + +- Run tests +- Run linters +- Update version and changelog +- Deploy diff --git a/pecha_api/text_uploader/text_metadata/text_metadata_model.py b/pecha_api/text_uploader/text_metadata/text_metadata_model.py index 04f29408..76c26d9d 100644 --- a/pecha_api/text_uploader/text_metadata/text_metadata_model.py +++ b/pecha_api/text_uploader/text_metadata/text_metadata_model.py @@ -25,7 +25,7 @@ class CriticalInstance(BaseModel): source: str colophon: Optional[str] = None incipit_title: Optional[Any] = None - alt_incipit_titles: Optional[Any] = None + alt_incipit_titles: Optional[List[Any]] = None biblography_annotation: Optional[str] = None class CriticalInstanceResponse(BaseModel):