Add metadata to Passages and Sources tables to track embedding model/dimentions

[sarahwooders]

We should add metadata fields to the Passages table that include:

• embedding_model: Model used to generate the embedding (also in Sources table)
• embedding_dim: Dimensions of embedding field (also in Sources table)
• num_tokens: Number of tokens contained in passage
• num_chars: Number of characters contained in passages

This will allow us to add additional checks:

• When querying archival memory, filter by embedding_model field to avoid
• Warn users when they change their embedding model, which loaded sources will not longer be accessible to their agent

Deign Decisions

• When a user change their embedding model, should we remove previously imported embeddings from the agent (as they will still be saved in the data sources table)?

[github-actions]

This issue has been automatically closed due to 60 days of inactivity.