Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions backend/DATABASE_MIGRATIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Database Migration Guide

## Quick Fix for "no such column" Errors

If you see errors like:
```
[BACKEND] | ERROR | Error getting all images: no such column: i.isFavourite
```

### Solution 1: Run Migration Script (Recommended)
This preserves your existing data:

```bash
cd backend
python migrate_database.py
```

### Solution 2: Reset Database (Nuclear Option)
⚠️ **Warning: This deletes all your data!**

```bash
cd backend
python reset_database.py
```

Then restart the backend server.

## How Migrations Work

The application now includes automatic schema migrations:

1. **Automatic Migration**: When the app starts, it checks if required columns exist
2. **Zero Downtime**: Migrations run transparently without data loss
3. **Safe Defaults**: New columns are added with sensible default values

## Manual Migration

If you need to run migrations manually:

```python
from app.database.images import db_create_images_table

# This will create tables and run any necessary migrations
db_create_images_table()
```

## Adding New Columns (For Developers)

When adding new database columns:

1. **Update the CREATE TABLE statement** in the respective database file
2. **Add migration logic** to the `_check_and_migrate_schema()` function
3. **Test the migration** on a copy of the production database
4. **Document the change** in this file

Example:
```python
def _check_and_migrate_schema(conn: sqlite3.Connection) -> None:
cursor = conn.cursor()
cursor.execute("PRAGMA table_info(images)")
columns = [col[1] for col in cursor.fetchall()]

# Add new column if it doesn't exist
if "newColumn" not in columns:
logger.info("Adding 'newColumn' to images table")
cursor.execute("ALTER TABLE images ADD COLUMN newColumn TEXT DEFAULT ''")
conn.commit()
```

## Troubleshooting

### Migration Fails
- Check database file permissions
- Ensure no other process is locking the database
- Check available disk space

### Column Still Missing After Migration
- Restart the backend server
- Check logs for migration success message
- Verify the database file path in settings

### Database is Corrupted
If migration fails due to corruption:
1. Backup your database file (`*.db`)
2. Try SQLite recovery tools
3. As last resort, use `reset_database.py` (data loss!)

## Database Schema Version

Current schema includes:
- ✅ `images.isFavourite` (added: Dec 2025)
- ✅ `images.isTagged`
- ✅ `images.metadata`
- ✅ `image_classes` junction table

## Future Migrations

Planned schema changes:
- [ ] Add `rating` column for star ratings (1-5)
- [ ] Add `dateModified` column for edit tracking
- [ ] Add `albums` table for album management
18 changes: 8 additions & 10 deletions backend/app/logging/setup_logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -235,19 +235,17 @@ def emit(self, record: logging.LogRecord) -> None:
Args:
record: The log record to process
"""
# Get the appropriate module name
module_name = record.name
if "." in module_name:
module_name = module_name.split(".")[-1]
# Prevent recursive interception: mark records once handled
if getattr(record, "_intercepted", False):
return
setattr(record, "_intercepted", True)

# Create a message that includes the original module in the format
# Create message (original module name preserved in formatted output by formatter)
msg = record.getMessage()

# Find the appropriate logger
logger = get_logger(module_name)

# Log the message with our custom formatting
logger.log(record.levelno, f"[uvicorn] {msg}")
# Route directly to root logger to avoid re-triggering intercept handlers
root_logger = logging.getLogger()
root_logger.log(record.levelno, f"[uvicorn] {msg}")


def configure_uvicorn_logging(component_name: str) -> None:
Expand Down
55 changes: 55 additions & 0 deletions backend/migrate_database.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""
Database Migration Script
Adds the isFavourite column to existing images table if it doesn't exist.
"""

import sqlite3
from app.config.settings import DATABASE_PATH
from app.logging.setup_logging import get_logger

logger = get_logger(__name__)


def check_column_exists(cursor, table_name: str, column_name: str) -> bool:
"""Check if a column exists in a table."""
cursor.execute(f"PRAGMA table_info({table_name})")
columns = [col[1] for col in cursor.fetchall()]
return column_name in columns


def migrate_database():
"""Add missing columns to the images table."""
try:
conn = sqlite3.connect(DATABASE_PATH)
cursor = conn.cursor()

# Check if isFavourite column exists
if not check_column_exists(cursor, "images", "isFavourite"):
logger.info("Adding 'isFavourite' column to images table...")
cursor.execute(
"""
ALTER TABLE images
ADD COLUMN isFavourite BOOLEAN DEFAULT 0
"""
)
conn.commit()
logger.info("✓ Successfully added 'isFavourite' column")
else:
logger.info("✓ 'isFavourite' column already exists")

conn.close()
print("\n✅ Database migration completed successfully!")

except sqlite3.Error as e:
logger.error(f"Database migration failed: {e}")
print(f"\n❌ Migration failed: {e}")
raise
except Exception as e:
logger.error(f"Unexpected error during migration: {e}")
print(f"\n❌ Unexpected error: {e}")
raise
Comment on lines +20 to +50
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Resource leak: database connection not closed on exception.

If an exception occurs before line 40, the database connection remains open. This could lead to connection leaks and database locks.

Apply this diff to use a context manager:

 def migrate_database():
     """Add missing columns to the images table."""
+    conn = None
     try:
         conn = sqlite3.connect(DATABASE_PATH)
         cursor = conn.cursor()
 
         # Check if isFavourite column exists
         if not check_column_exists(cursor, "images", "isFavourite"):
             logger.info("Adding 'isFavourite' column to images table...")
             cursor.execute(
                 """
                 ALTER TABLE images 
                 ADD COLUMN isFavourite BOOLEAN DEFAULT 0
                 """
             )
             conn.commit()
             logger.info("✓ Successfully added 'isFavourite' column")
         else:
             logger.info("✓ 'isFavourite' column already exists")
 
-        conn.close()
         print("\n✅ Database migration completed successfully!")
 
     except sqlite3.Error as e:
         logger.error(f"Database migration failed: {e}")
         print(f"\n❌ Migration failed: {e}")
         raise
     except Exception as e:
         logger.error(f"Unexpected error during migration: {e}")
         print(f"\n❌ Unexpected error: {e}")
         raise
+    finally:
+        if conn:
+            conn.close()
🤖 Prompt for AI Agents
In backend/migrate_database.py around lines 20 to 50, the sqlite3 connection is
opened but not guaranteed to be closed if an exception occurs; replace the
manual open/close with a context manager by using "with
sqlite3.connect(DATABASE_PATH) as conn:" and create the cursor inside that
block, perform ALTER and conn.commit() as needed, remove the explicit
conn.close(), and keep the existing exception handlers (they will still work) so
the connection is always closed even on errors.



if __name__ == "__main__":
print("Starting database migration...")
migrate_database()
10 changes: 7 additions & 3 deletions docs/backend/backend_python/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -1117,9 +1117,14 @@
"in": "query",
"required": false,
"schema": {
"$ref": "#/components/schemas/InputType",
"allOf": [
{
"$ref": "#/components/schemas/InputType"
}
],
"description": "Choose input type: 'path' or 'base64'",
"default": "path"
"default": "path",
"title": "Input Type"
},
"description": "Choose input type: 'path' or 'base64'"
}
Expand Down Expand Up @@ -2199,7 +2204,6 @@
"metadata": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
Expand Down
Loading