Skip to content

Update build pipeline to support the microservice. #492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rahulharpal1603 merged 2 commits into from
Aug 30, 2025
Merged

Update build pipeline to support the microservice. #492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b0d878d
Update build pipeline to support the microservice.
rahulharpal1603 Aug 30, 2025
49c4d80
Fix lint errors
rahulharpal1603 Aug 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 154 additions & 7 deletions .github/workflows/build-and-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,37 +134,176 @@ jobs:
name: PictoPy-MacOS
path: backend/dist/PictoPy_Server/PictoPy-MacOS.zip

build-sync-microservice-windows:
runs-on: windows-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.12

Comment on lines +144 to +147
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Bump setup-python to v5 across new sync jobs.

Fixes “runner too old” errors flagged by actionlint.

-      - name: Setup Python
-        uses: actions/setup-python@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
         with:
           python-version: 3.12

Also applies to: 184-187, 224-227

🧰 Tools
🪛 actionlint (1.7.7)

144-144: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue

(action)

🤖 Prompt for AI Agents 
In .github/workflows/build-and-release.yml around lines 144-147 (and also update
occurrences at 184-187 and 224-227), the workflow uses actions/setup-python@v4
which triggers “runner too old” actionlint errors; change the action reference
to actions/setup-python@v5 in each location (keeping the same python-version
values) so the workflow uses the newer setup-python action.

- name: Install dependencies
run: |
cd sync-microservice
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pyinstaller

- name: Build executable with PyInstaller
run: |
cd sync-microservice
pyinstaller main.py --name PictoPy_Sync --onedir --distpath dist

- name: Copy app folder
run: |
cd sync-microservice
robocopy app dist\PictoPy_Sync\app /e
if ($LASTEXITCODE -le 1) { exit 0 }

- name: Create ZIP package
run: |
cd sync-microservice/dist/PictoPy_Sync
tar -a -c -f PictoPy-Sync-Windows.zip .

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: PictoPy-Sync-Windows
path: sync-microservice/dist/PictoPy_Sync/PictoPy-Sync-Windows.zip

build-sync-microservice-ubuntu:
runs-on: ubuntu-22.04
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.12

- name: Install dependencies
run: |
cd sync-microservice
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pyinstaller

- name: Build executable with PyInstaller
run: |
cd sync-microservice
pyinstaller main.py --name PictoPy_Sync --onedir --distpath dist

- name: Copy app folder
run: |
cd sync-microservice
mkdir -p dist/PictoPy_Sync/app
cp -r app/* dist/PictoPy_Sync/app/

- name: Create ZIP package
run: |
cd sync-microservice/dist/PictoPy_Sync
zip -r PictoPy-Sync-Ubuntu.zip .

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: PictoPy-Sync-Ubuntu
path: sync-microservice/dist/PictoPy_Sync/PictoPy-Sync-Ubuntu.zip

build-sync-microservice-macos:
runs-on: macos-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.12

- name: Install dependencies
run: |
cd sync-microservice
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pyinstaller

- name: Build executable with PyInstaller
run: |
cd sync-microservice
pyinstaller main.py --name PictoPy_Sync --onedir --distpath dist

- name: Copy app folder
run: |
cd sync-microservice
mkdir -p dist/PictoPy_Sync/app
cp -r app/* dist/PictoPy_Sync/app/

- name: Create ZIP package
run: |
cd sync-microservice/dist/PictoPy_Sync
zip -r PictoPy-Sync-MacOS.zip .

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: PictoPy-Sync-MacOS
path: sync-microservice/dist/PictoPy_Sync/PictoPy-Sync-MacOS.zip

publish-tauri:
permissions:
contents: write
needs: [build-server-windows, build-server-ubuntu, build-server-macos]
needs:
[
build-server-windows,
build-server-ubuntu,
build-server-macos,
build-sync-microservice-windows,
build-sync-microservice-ubuntu,
build-sync-microservice-macos,
]
strategy:
fail-fast: false
matrix:
include:
- platform: "macos-latest"
args: "--target aarch64-apple-darwin"
artifact: "PictoPy-MacOS"
server-artifact: "PictoPy-MacOS"
sync-artifact: "PictoPy-Sync-MacOS"
- platform: "ubuntu-22.04"
args: ""
artifact: "PictoPy-Ubuntu"
server-artifact: "PictoPy-Ubuntu"
sync-artifact: "PictoPy-Sync-Ubuntu"
- platform: "windows-latest"
args: ""
artifact: "PictoPy-Windows"
server-artifact: "PictoPy-Windows"
sync-artifact: "PictoPy-Sync-Windows"
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4

- name: Create dist directory
- name: Create dist directories
shell: bash
run: mkdir -p backend/dist
run: |
mkdir -p backend/dist
mkdir -p sync-microservice/dist

- name: Download and extract server artifact
uses: actions/download-artifact@v4
with:
name: ${{ matrix.artifact }}
name: ${{ matrix.server-artifact }}
path: backend/dist

- name: Download and extract sync microservice artifact
uses: actions/download-artifact@v4
with:
name: ${{ matrix.sync-artifact }}
path: sync-microservice/dist

- name: Extract server files
shell: bash
run: |
Expand All @@ -173,6 +312,14 @@ jobs:
rm *.zip
ls -l

- name: Extract sync microservice files
shell: bash
run: |
cd sync-microservice/dist
unzip -o *.zip
rm *.zip
ls -l

- name: Setup Node
uses: actions/setup-node@v4
with:
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/pr-check-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,22 @@ jobs:
with:
python-version: "3.11"

- name: Install Python Dependencies & Run Tests
- name: Build check(Main backend)
run: |
cd backend
python -m pip install --upgrade pip
pip install -r requirements.txt
pyinstaller main.py --name PictoPy_Server --onedir --distpath dist

- name: Build check(Sync Microservice)
run: |
cd sync-microservice
pip install -r requirements.txt
pyinstaller main.py --name PictoPy_Sync_Microservice --onedir --distpath dist

- name: Run Tests
run: |
cd backend
pytest

# Tauri Test Job
Expand Down
77 changes: 76 additions & 1 deletion backend/app/utils/microservice.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,88 @@ def microservice_util_start_sync_service(
"""
Start the sync microservice with automatic virtual environment management.

When running as a frozen executable (PyInstaller), it will use the bundled
PictoPy_Sync executable. Otherwise, it uses the development setup with venv.

Args:
sync_service_path: Path to the sync microservice directory.
If None, defaults to 'sync-microservice' relative to project root.

Returns:
bool: True if service started successfully, False otherwise.
"""
try:
# Check if running as a frozen executable (PyInstaller)
if getattr(sys, "frozen", False):
logger.info(
"Running as frozen executable, using bundled sync microservice..."
)
return _start_frozen_sync_service()

# Development mode - use virtual environment setup
logger.info("Running in development mode, using virtual environment...")
return _start_dev_sync_service(sync_service_path)

except Exception as e:
logger.error(f"Error starting sync microservice: {e}")
return False


def _start_frozen_sync_service() -> bool:
"""
Start the sync microservice when running as a frozen executable.
The sync microservice executable should be in the PictoPy_Sync folder.
"""
try:
# Get the directory where the current executable is located
if getattr(sys, "frozen", False):
# When frozen, sys.executable points to the main executable
app_dir = Path(sys.executable).parent
else:
# Fallback (shouldn't happen in this function)
app_dir = Path(__file__).parent.parent.parent.parent

# Look for the PictoPy_Sync directory and executable
sync_dir = app_dir / "PictoPy_Sync"

# Determine executable name based on platform
system = platform.system().lower()
if system == "windows":
sync_executable = sync_dir / "PictoPy_Sync.exe"
else:
sync_executable = sync_dir / "PictoPy_Sync"

if not sync_executable.exists():
logger.error(
f"Sync microservice executable not found at: {sync_executable}"
)
return False

logger.info(f"Starting sync microservice from: {sync_executable}")

# Start the sync microservice executable
process = subprocess.Popen(
[str(sync_executable)],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
cwd=str(sync_dir), # Set working directory to sync service directory
)

logger.info(f"Sync microservice started with PID: {process.pid}")
logger.info("Service should be available at http://localhost:8001")

return True

except Exception as e:
logger.error(f"Error starting frozen sync microservice: {e}")
return False


def _start_dev_sync_service(sync_service_path: Optional[str] = None) -> bool:
"""
Start the sync microservice in development mode using virtual environment.
"""
try:
# Determine the sync service path
if sync_service_path is None:
Expand Down Expand Up @@ -65,7 +140,7 @@ def microservice_util_start_sync_service(
return _start_fastapi_service(python_executable, sync_service_path)

except Exception as e:
logger.error(f"Error starting sync microservice: {e}")
logger.error(f"Error starting dev sync microservice: {e}")
return False


Expand Down
6 changes: 6 additions & 0 deletions sync-microservice/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from fastapi import FastAPI
from uvicorn import Config, Server
from app.core.lifespan import lifespan
from app.routes import health, watcher, folders

Expand All @@ -14,3 +15,8 @@
app.include_router(health.router, prefix="/api/v1")
app.include_router(watcher.router, prefix="/api/v1")
app.include_router(folders.router, prefix="/api/v1")

if __name__ == "__main__":
config = Config(app=app, host="0.0.0.0", port=8001, log_config=None)
server = Server(config)
server.run()
Loading