Skip to content

Latest commit

 

History

History
124 lines (112 loc) · 6.67 KB

.context.md

File metadata and controls

124 lines (112 loc) · 6.67 KB
Raw
Error in user YAML: (<unknown>): found character that cannot start any token while scanning for the next token at line 56 column 23
---
module-name: pydantic-all-in-one
version: 2024.10.18
description: "A collection of Pydantic-related projects, combining libraries like FastAPI, SQLModel, GQLAlchemy, and other tools for efficient data validation, RESTful APIs, and graph database interactions."
related-modules:
  - name: fastapi
    path: https://github.com/tiangolo/fastapi
  - name: pydantic
    path: https://github.com/pydantic/pydantic
  - name: gqlalchemy
    path: https://github.com/memgraph/gqlalchemy
  - name: sqlmodel
    path: https://github.com/tiangolo/sqlmodel
technologies:
  - Python 3.12
  - FastAPI for building RESTful APIs.
  - SQLModel for interacting with SQLite databases.
  - GQLAlchemy for interfacing with Memgraph and managing graph data.
  - Pydantic for data validation and settings management.
  - Uvicorn and Gunicorn for serving the API.
  - Docker for containerization and environment management.
  - Poetry for dependency management and packaging.
  - FastStream for event streaming.
  - aioclock for scheduling tasks.
  - Lancedb for managing vector databases.
conventions:
  - Follows PEP8 coding standards.
  - Uses "numpy" style docstrings for consistency.
  - Semantic versioning for releases (e.g., `version: 2024.10.18`).
directives:
  - Use Poetry for installing dependencies and running scripts.
  - Use Docker for consistent development and production environments.
  - Use `poethepoet` for task automation like testing and linting.
  - Apply `pre-commit` hooks to ensure code quality.
diagrams:
  - name: architecture-overview
    path: diagrams/architecture.mmd
  - name: deployment-diagram
    path: diagrams/deployment.mmd
architecture:
  style: "Modular Monolith"
  components:
    - FastAPI handles REST API endpoints and serves as the core of the service.
    - GQLAlchemy connects to a Memgraph instance for managing graph data.
    - SQLModel provides ORM-like capabilities with SQLite for relational data management.
    - FastStream integrates event-driven communication, especially useful for streaming events.
    - aioclock schedules tasks that run asynchronously.
    - Lancedb manages vector data for AI/ML applications.
  data-flow:
    - API requests are processed by FastAPI and dispatched to the appropriate handlers.
    - Graph queries and updates are managed by GQLAlchemy with Memgraph.
    - Relational data is stored and queried using SQLModel and SQLite.
    - Event streaming is handled through FastStream, enabling asynchronous data processing.
    - Vector data is managed through Lancedb, supporting machine learning applications.
development:
  setup-steps:
    - Install Poetry: `curl -sSL https://install.python-poetry.org | python3 -`
    - Clone the repository: `git clone https://github.com/seanchatmangpt/pydantic-all-in-one`
    - Install dependencies: `poetry install`
    - Start the Docker containers: `docker-compose up -d`
    - Run the FastAPI server: `poe api`
  build-command: `poetry build`
  test-command: `poe test`
business-requirements:
  key-features:
    - Unified environment for managing REST APIs and graph data.
    - Integration of event streaming for real-time processing.
    - Data validation using Pydantic for robust and consistent data handling.
    - SQLModel support for efficient relational database operations.
  target-audience: Python developers building complex applications with needs for data validation, APIs, and hybrid database interactions.
  success-metrics:
    - Reduced integration time for adding new services.
    - High test coverage ensuring robust functionality.
    - Low latency in API and streaming response times.
quality-assurance:
  testing-frameworks:
    - pytest for unit and integration tests.
    - mypy for static type checking.
    - coverage to ensure thorough testing.
  coverage-threshold: 85%
  performance-benchmarks:
    - API latency <100ms for typical queries.
    - Efficient memory usage with vector data storage in Lancedb.
    - Streaming throughput optimized for event-driven use cases.
deployment:
  platform: Docker for development and production environments, with options for Kubernetes.
  cicd-pipeline: GitHub Actions automates testing, linting, and deployment.
  staging-environment: http://staging.pydantic-all-in-one.example.com
  production-environment: http://pydantic-all-in-one.example.com
---

pydantic-all-in-one

pydantic-all-in-one is designed to bring together a set of Python tools centered around Pydantic, simplifying the creation of APIs, handling relational and graph data, and enabling event-driven programming. The project is structured to make data validation seamless and to integrate well with both SQL and graph databases.

Architecture Overview

This project follows a modular approach:

  • FastAPI serves as the primary API layer, offering endpoints that interact with other services.
  • GQLAlchemy facilitates connections to a Memgraph database, providing a powerful tool for managing complex relationships and graph data.
  • SQLModel supports operations with SQLite, ensuring a smooth interaction between Python data models and SQL tables.
  • FastStream manages event streaming, enabling asynchronous communication and processing.
  • Lancedb is used for storing and retrieving vector data, suitable for AI/ML use cases.
  • aioclock provides utilities for scheduling tasks that require precise timing.

Development Guidelines

To contribute:

  • Follow PEP8 standards for formatting.
  • Use Poetry to manage dependencies and environment configurations.
  • Write tests using pytest and ensure all tests pass before pushing code.
  • Use poethepoet for running common tasks like linting and testing.

Business Context

The primary goal of this project is to streamline the setup and use of Pydantic-related libraries in a Python application. By integrating tools like FastAPI, GQLAlchemy, and SQLModel, it reduces the complexity of working with hybrid data models. The inclusion of event streaming capabilities makes it suitable for applications that require real-time processing.

Quality Assurance

Testing is managed through pytest and mypy for type checking, ensuring type safety and robust functionality. Code quality is maintained with pre-commit hooks using ruff for linting. Coverage reports are generated with a threshold of 85% to ensure critical paths are well-tested.

Deployment and Operations

Deployment is managed using Docker, ensuring consistent environments from development to production. The docker-compose.yml file is used to orchestrate services for easy setup. For larger deployments, Kubernetes can be used to manage scaling. Continuous integration and deployment are handled through GitHub Actions, ensuring each change is tested and verified before it reaches production.