documentation_update

documentation/developer/README.md

@@ -0,0 +1,12 @@
+## Developer Documentation
+
+This folder contains developer-focused documentation for the Pecha backend.
+
+### Contents
+
+- `setup.md`: local setup, dependencies, and environment configuration
+- `architecture.md`: high-level architecture and module boundaries
+- `api_overview.md`: API surface summary and module map
+- `workflows.md`: common developer workflows and release practices
+
+If you add a new doc, link it here.

documentation/developer/api_overview.md

@@ -0,0 +1,35 @@
+
+Current specs live in `documentation/`:
+- `pecha-auth-api.yaml`
+- `pecha-core-api.yaml`
+- `cms-openapi.yaml`
+- `openapi.yaml`
+
+The API root path is `/api/v1` (see `pecha_api/app.py`).
+
+### Module to Router Map
+
+Routers are included in `pecha_api/app.py`:
+- Auth: `pecha_api/auth/auth_views.py`
+- Core: texts, groups, segments, mappings, topics, users, collections, terms, sheets
+- Search: `pecha_api/search/search_views.py`
+- Plans: public, users, CMS, items, tasks, featured
+- Share: `pecha_api/share/share_views.py`
+- Uploading: text uploader, cataloger, metadata
+- Recitations: recitations APIs
+
+### Utility Endpoints
+
+- `GET /health`
+- `GET /props`
+
+### Response Models
+
+Response models are defined per module and exposed via `response_model` in
+routes. Common locations:
+
+- Texts: `pecha_api/texts/texts_response_models.py`
+- Users: `pecha_api/users/user_response_models.py`
+- Plans media uploads: `pecha_api/plans/media/media_response_models.py`
+
+Standard message strings live in `pecha_api/plans/response_message.py`.

documentation/developer/architecture.md

@@ -0,0 +1,63 @@
+## Architecture
+
+### Overview
+
+The backend is a FastAPI application with a single app in `pecha_api/app.py`.
+Routers are registered on `api` with `root_path="/api/v1"` and a shared
+lifespan in `pecha_api/db/mongo_database.py`.
+
+Each feature module typically contains:
+- `*_views.py` for routes
+- `*_services.py` for business logic
+- `*_models.py` for data models and response schemas
+
+### Module Map
+
+Routers included in `pecha_api/app.py`:
+
+- `auth`: `pecha_api/auth/auth_views.py`
+- `sheets`: `pecha_api/sheets/sheets_views.py`
+- `collections`: `pecha_api/collections/collections_views.py`
+- `terms`: `pecha_api/terms/terms_views.py`
+- `texts`: `pecha_api/texts/texts_views.py`
+- `groups`: `pecha_api/texts/groups/groups_views.py`
+- `segments`: `pecha_api/texts/segments/segments_views.py`
+- `mappings`: `pecha_api/texts/mappings/mappings_views.py`
+- `topics`: `pecha_api/topics/topics_views.py`
+- `users`: `pecha_api/users/users_views.py`
+- `share`: `pecha_api/share/share_views.py`
+- `search`: `pecha_api/search/search_views.py`
+- `plans (CMS)`: `pecha_api/plans/cms/cms_plans_views.py`
+- `plans (public)`: `pecha_api/plans/public/plan_views.py`
+- `plans (users)`: `pecha_api/plans/users/plan_users_views.py`
+- `plans (items)`: `pecha_api/plans/items/plan_items_views.py`
+- `plans (tasks)`: `pecha_api/plans/tasks/plan_tasks_views.py`
+- `plans (sub-tasks)`: `pecha_api/plans/tasks/sub_tasks/plan_sub_tasks_views.py`
+- `plans (authors)`: `pecha_api/plans/authors/plan_authors_views.py`
+- `plans (media)`: `pecha_api/plans/media/media_views.py`
+- `plans (featured)`: `pecha_api/plans/featured/featured_day_views.py`
+- `recitations`: `pecha_api/recitations/recitations_view.py`
+- `user follows`: `pecha_api/user_follows/user_follow_views.py`
+- `user recitations`: `pecha_api/plans/users/recitation/user_recitations_views.py`
+- `text uploader`: `pecha_api/text_uploader/text_uploader_views.py`
+- `text metadata`: `pecha_api/text_uploader/text_metadata/text_metadata_views.py`
+- `uploader collections`: `pecha_api/text_uploader/collections/uploader_collections_views.py`
+- `cataloger`: `pecha_api/cataloger/cataloger_views.py`
+
+### Dependencies
+
+Local services used during development (see `local_setup/docker-compose.yml`):
+- Postgres (port 5434)
+- MongoDB (port 27017)
+- Dragonfly/Redis (port 6379)
+- Elasticsearch (port 9200)
+
+External integrations (configured in `pecha_api/config.py`):
+- AWS S3 compatible storage
+- Auth0
+- Mailtrap
+
+### Error Handling
+
+Use `HTTPException` for expected errors. Log unexpected failures and return
+consistent error responses from shared helpers.

documentation/developer/setup.md

@@ -0,0 +1,78 @@
+## Local Setup
+
+### Prerequisites
+
+- Python 3.12
+- Poetry
+- Docker (for local services)
+
+### Environment Variables
+
+Defaults are defined in `pecha_api/config.py`, but for local development
+you typically override these values:
+
+- `DATABASE_URL`
+- `MONGO_CONNECTION_STRING`
+- `CACHE_CONNECTION_STRING`
+- `ELASTICSEARCH_URL`
+- `ELASTICSEARCH_API` (optional for local)
+
+Optional integrations:
+
+- `AWS_ACCESS_KEY`, `AWS_SECRET_KEY`, `AWS_BUCKET_NAME`
+- `DOMAIN_NAME`, `CLIENT_ID` (Auth0)
+- `MAILTRAP_API_KEY`, `SENDER_EMAIL`, `SENDER_NAME`
+
+### Install Dependencies
+
+```sh
+poetry install
+```
+
+### Database and Search
+
+Start local services (Postgres, MongoDB, Redis/Dragonfly, Elasticsearch):
+
+```sh
+cd local_setup
+docker-compose up -d
+```
+
+If you see file permission errors for local data directories:
+
+```sh
+./dev/fix_permissions.sh
+```
+
+Apply migrations:
+
+```sh
+poetry run alembic upgrade head
+```
+
+### Run the API
+
+Recommended:
+
+```sh
+./dev/start_dev.sh
+```
+
+Or run directly:
+
+```sh
+poetry run uvicorn pecha_api.app:api --reload
+```
+
+### Tests
+
+```sh
+poetry run pytest
+```
+
+Coverage:
+
+```sh
+poetry run pytest --cov=pecha_api
+poetry run coverage html
+```

documentation/developer/workflows.md

@@ -0,0 +1,49 @@
+## Workflows
+
+### Add a New Endpoint
+
+1. Define request and response models
+2. Implement route in `*_views.py`
+3. Add business logic in `*_services.py`
+4. Add tests
+5. Update OpenAPI spec and developer docs
+
+### Local Development Loop
+
+1. Start services:
+   ```sh
+   cd local_setup
+   docker-compose up -d
+   ```
+2. Run the API:
+   ```sh
+   ./dev/start_dev.sh
+   ```
+
+### Add a New Feature Module
+
+1. Create module folder
+2. Add `*_views.py`, `*_services.py`, `*_models.py`
+3. Register router in app startup
+4. Add docs under `documentation/`
+
+### Database Migrations
+
+Create:
+
+```sh
+poetry run alembic revision --autogenerate -m "add feature"
+```
+
+Apply:
+
+```sh
+poetry run alembic upgrade head
+```
+
+### Release Checklist
+
+- Run tests
+- Run linters
+- Update version and changelog
+- Deploy

pecha_api/text_uploader/text_metadata/text_metadata_model.py

@@ -25,7 +25,7 @@ class CriticalInstance(BaseModel):
     source: str
     colophon: Optional[str] = None
     incipit_title: Optional[Any] = None
-    alt_incipit_titles: Optional[Any] = None
+    alt_incipit_titles: Optional[List[Any]] = None
     biblography_annotation: Optional[str] = None
 
 class CriticalInstanceResponse(BaseModel):