…#558) * feat(memory): add document loaders, text splitter, and ingestion pipeline Introduce document loading subsystem in zeph-memory with DocumentLoader trait, TextLoader (txt/md), TextSplitter with sentence-aware chunking, and IngestionPipeline that integrates with Qdrant vector store. Add feature-gated PdfLoader behind `pdf` feature using pdf-extract. Include file size guard (50 MiB) and path canonicalization for security. Closes #469, #470, #471, #472 Refs #478 * refactor(memory): add configurable max_file_size to PdfLoader Match TextLoader pattern with per-instance max_file_size field defaulting to DEFAULT_MAX_FILE_SIZE (50 MiB). * test(memory): add proptests for text splitter and integration tests for ingestion pipeline Add 4 property-based tests for TextSplitter: never panics on arbitrary input, chunks cover all content, indices are sequential, no empty chunks. Add 5 integration tests using testcontainers Qdrant for IngestionPipeline: single/empty/multi-chunk ingest, load_and_ingest from file, payload verification. * docs: add document loaders guide, update crate docs and READMEs Add docs/src/guide/document-loaders.md covering DocumentLoader trait, TextLoader, PdfLoader, TextSplitter, and IngestionPipeline. Update architecture/crates.md, feature-flags.md, SUMMARY.md, zeph-memory README, and root README with pdf feature flag. * fix(ci): exclude integration tests from snapshot check Integration tests require Docker (testcontainers) and fail in the snapshot check job which lacks Docker. Limit insta test to --lib --bins since all snapshots are in inline cfg(test) modules.

feat(memory): document ingestion pipeline #472

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions