pgvector-nim

pgvector support for Nim

Supports db_connector

Getting Started

Run:

nimble add pgvector

And follow the instructions for your database library:

• db_connector

Or check out an example:

• Embeddings with OpenAI
• Binary embeddings with Cohere
• Hybrid search with Ollama (Reciprocal Rank Fusion)
• Sparse search with Text Embeddings Inference

db_connector

Enable the extension

db.exec(sql"CREATE EXTENSION IF NOT EXISTS vector")

Create a table

db.exec(sql"CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(3))")

Insert vectors

import pgvector

let embedding1 = @[1, 1, 1]
let embedding2 = @[1, 1, 2]
let embedding3 = @[2, 2, 2]
db.exec(sql"INSERT INTO items (embedding) VALUES (?), (?), (?)", embedding1.toVector, embedding2.toVector, embedding3.toVector)

Get the nearest neighbors

let embedding = @[1, 1, 1]
let rows = db.getAllRows(sql"SELECT * FROM items ORDER BY embedding <-> ? LIMIT 5", embedding.toVector)
for row in rows:
  echo row

Add an approximate index

db.exec(sql"CREATE INDEX ON items USING hnsw (embedding vector_l2_ops)")
# or
db.exec(sql"CREATE INDEX ON items USING ivfflat (embedding vector_l2_ops) WITH (lists = 100)")

Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance

See a full example

Reference

Vectors

Create a vector from a sequence or array

let vec = @[1.0, 2.0, 3.0].toVector

Get a sequence

let s = vec.toSeq

Half Vectors

Create a half vector from a sequence or array

let vec = @[1.0, 2.0, 3.0].toHalfVector

Get a sequence

let s = vec.toSeq

Sparse Vectors

Create a sparse vector from a sequence or array

let vec = @[1.0, 0.0, 2.0, 0.0, 3.0, 0.0].toSparseVector

Or a table of non-zero elements

let elements = {0: 1.0, 2: 2.0, 4: 3.0}.toTable
let vec = elements.toSparseVector(6)

Note: Indices start at 0

Get the number of dimensions

let dim = vec.dim

Get the non-zero elements

let elements = vec.elements

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/pgvector/pgvector-nim.git
cd pgvector-nim
createdb pgvector_nim_test
nimble install db_connector
nimble test

Specify the path to libpq if needed:

nimble test --passL:-Wl,-rpath,/opt/homebrew/opt/libpq/lib

To run an example:

cd examples/openai
createdb pgvector_example
nim c --run --path:../../src example.nim

Specify the path to libpq if needed:

nim c --run --path:../../src --passL:-Wl,-rpath,/opt/homebrew/opt/libpq/lib example.nim