SQLSpec

A Query Mapper for Python

SQLSpec is an experimental Python library designed to streamline and modernize your SQL interactions across a variety of database systems. While still in its early stages, SQLSpec aims to provide a flexible, typed, and extensible interface for working with SQL in Python.

Note: SQLSpec is currently under active development and the API is subject to change. It is not yet ready for production use. Contributions are welcome!

Core Features (Planned but subject to change, removal or redesign)

• Consistent Database Session Interface: Provides a consistent connectivity interface for interacting with one or more database systems, including SQLite, Postgres, DuckDB, MySQL, Oracle, SQL Server, Spanner, BigQuery, and more.
• Emphasis on RAW SQL and Minimal Abstractions and Performance: SQLSpec is a library for working with SQL in Python. It's goals are to offer minimal abstractions between the user and the database. It does not aim to be an ORM library.
• Type-Safe Queries: Quickly map SQL queries to typed objects using libraries such as Pydantic, Msgspec, Attrs, etc.
• Extensible Design: Easily add support for new database dialects or extend existing functionality to meet your specific needs. Easily add support for async and sync database drivers.
• Minimal Dependencies: SQLSpec is designed to be lightweight and can run on it's own or with other libraries such as litestar, fastapi, flask and more. (Contributions welcome!)
• Dynamic Query Manipulation: Easily apply filters to pre-defined queries with a fluent, Pythonic API. Safely manipulate queries without the risk of SQL injection.
• Dialect Validation and Conversion: Use sqlglot to validate your SQL against specific dialects and seamlessly convert between them.
• Support for Async and Sync Database Drivers: SQLSpec supports both async and sync database drivers, allowing you to choose the style that best fits your application.
• Basic Migration Management: A mechanism to generate empty migration files where you can add your own SQL and intelligently track which migrations have been applied.

What SQLSpec Is Not (Yet)

SQLSpec is a work in progress. While it offers a solid foundation for modern SQL interactions, it does not yet include every feature you might find in a mature ORM or database toolkit. The focus is on building a robust, flexible core that can be extended over time.

Examples

We've talked about what SQLSpec is not, so let's look at what it can do.

These are just a few of the examples that demonstrate SQLSpec's flexibility and each of the bundled adapters offer the same config and driver interfaces.

DuckDB LLM

This is a quick implementation using some of the built in Secret and Extension management features of SQLSpec's DuckDB integration.

It allows you to communicate with any compatible OpenAPI conversations endpoint (such as Ollama). This examples:

• auto installs the open_prompt DuckDB extensions
• automatically creates the correct open_prompt comptaible secret required to use the extension

# /// script
# dependencies = [
#   "sqlspec[duckdb,performance]",
# ]
# ///
import os

from sqlspec import SQLSpec
from sqlspec.adapters.duckdb import DuckDBConfig
from pydantic import BaseModel

class ChatMessage(BaseModel):
    message: str

sql = SQLSpec()
etl_config = sql.add_config(
    DuckDBConfig(
        extensions=[{"name": "open_prompt"}],
        secrets=[
            {
                "secret_type": "open_prompt",
                "name": "open_prompt",
                "value": {
                    "api_url": "http://127.0.0.1:11434/v1/chat/completions",
                    "model_name": "gemma3:1b",
                    "api_timeout": "120",
                },
            }
        ],
    )
)
with sql.provide_session(etl_config) as session:
    result = session.select_one(
        "SELECT open_prompt(?)",
        "Can you write a haiku about DuckDB?",
        schema_type=ChatMessage
    )
    print(result) # result is a ChatMessage pydantic model

DuckDB Gemini Embeddings

In this example, we are again using DuckDB. However, we are going to use the built in to call the Google Gemini embeddings service directly from the database.

This example will

• auto installs the http_client and vss (vector similarity search) DuckDB extensions
• when a connection is created, it ensures that the generate_embeddings macro exists in the DuckDB database.
• Execute a simple query to call the Google API

# /// script
# dependencies = [
#   "sqlspec[duckdb,performance]",
# ]
# ///
import os

from sqlspec import SQLSpec
from sqlspec.adapters.duckdb import DuckDBConfig

EMBEDDING_MODEL = "gemini-embedding-exp-03-07"
GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY")
API_URL = (
    f"https://generativelanguage.googleapis.com/v1beta/models/{EMBEDDING_MODEL}:embedContent?key=${GOOGLE_API_KEY}"
)

sql = SQLSpec()
etl_config = sql.add_config(
    DuckDBConfig(
        extensions=[{"name": "vss"}, {"name": "http_client"}],
        on_connection_create=lambda connection: connection.execute(f"""
            CREATE IF NOT EXISTS MACRO generate_embedding(q) AS (
                WITH  __request AS (
                    SELECT http_post(
                        '{API_URL}',
                        headers => MAP {{
                            'accept': 'application/json',
                        }},
                        params => MAP {{
                            'model': 'models/{EMBEDDING_MODEL}',
                            'parts': [{{ 'text': q }}],
                            'taskType': 'SEMANTIC_SIMILARITY'
                        }}
                    ) AS response
                )
                SELECT *
                FROM __request,
            );
        """),
    )
)
with sql.provide_session(etl_config) as session:
    result = session.select_one("SELECT generate_embedding('example text')")
    print(result) # result is a dictionary when `schema_type` is omitted.

Basic Litestar Integration

In this example we are going to demonstrate how to create a basic configuration that integrates into Litestar.

# /// script
# dependencies = [
#   "sqlspec[aiosqlite]",
#   "litestar[standard]",
# ]
# ///

from aiosqlite import Connection
from litestar import Litestar, get

from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
from sqlspec.extensions.litestar import SQLSpec


@get("/")
async def simple_sqlite(db_session: AiosqliteDriver) -> dict[str, str]:
    return await db_session.select_one("SELECT 'Hello, world!' AS greeting")


sqlspec = SQLSpec(config=DatabaseConfig(
    config=[AiosqliteConfig(), commit_mode="autocommit")],
)
app = Litestar(route_handlers=[simple_sqlite], plugins=[sqlspec])

Inspiration and Future Direction

SQLSpec originally drew inspiration from features found in the aiosql library. This is a great library for working with and executed SQL stored in files. It's unclear how much of an overlap there will be between the two libraries, but it's possible that some features will be contributed back to aiosql where appropriate.

Current Focus: Universal Connectivity

The primary goal at this stage is to establish a native connectivity interface that works seamlessly across all supported database environments. This means you can connect to any of the supported databases using a consistent API, regardless of the underlying driver or dialect.

Adapters: Completed, In Progress, and Planned

This list is not final. If you have a driver you'd like to see added, please open an issue or submit a PR!

Driver	Database	Mode	Status
adbc	Postgres	Sync	✅
adbc	SQLite	Sync	✅
adbc	Snowflake	Sync	✅
adbc	DuckDB	Sync	✅
asyncpg	PostgreSQL	Async	✅
psycopg	PostgreSQL	Sync	✅
psycopg	PostgreSQL	Async	✅
psqlpy	PostgreSQL	Async	✅
aiosqlite	SQLite	Async	✅
sqlite3	SQLite	Sync	✅
oracledb	Oracle	Async	✅
oracledb	Oracle	Sync	✅
duckdb	DuckDB	Sync	✅
bigquery	BigQuery	Sync	✅
spanner	Spanner	Sync	🗓️
sqlserver	SQL Server	Sync	🗓️
mysql	MySQL	Sync	🗓️
snowflake	Snowflake	Sync	🗓️

Proposed Project Structure

• sqlspec/:
  • adapters/: Contains all database drivers and associated configuration.
  • extensions/:
    • litestar/: Litestar framework integration ✅
    • fastapi/: Future home of fastapi integration.
    • flask/: Future home of flask integration.
    • */: Future home of your favorite framework integration 🔌 ✨
  • base.py: Contains base protocols for database configurations.
  • filters.py: Contains the Filter class which is used to apply filters to pre-defined SQL queries.
  • utils/: Contains utility functions used throughout the project.
  • exceptions.py: Contains custom exceptions for SQLSpec.
  • typing.py: Contains type hints, type guards and several facades for optional libraries that are not required for the core functionality of SQLSpec.

Get Involved

SQLSpec is an open-source project, and contributions are welcome! Whether you're interested in adding support for new databases, improving the query interface, or simply providing feedback, your input is valuable.

Disclaimer: SQLSpec is under active development. Expect changes and improvements as the project evolves.