ci: reduce disk usage in unit tests (free space + test-only deps) (#412)

* ci: reduce disk usage in unit tests (free space + test-only deps)

* ci: install main+test deps (skip dev), add disk check, fix step comments

* ci: address PR review — remove --no-root/--no-cache, configurable MIN_FREE_GB

* ci: clarify deps rationale, fix disk check step name and int parsing

* ci: robust disk check (bytes), set MIN_FREE_GB=7, safe parallel cleanup

Author: manavgup

.github/workflows/04-pytest.yml

@@ -24,6 +24,7 @@ env:
   TESTING: true
   SKIP_AUTH: true
   DEVELOPMENT_MODE: true
+  MIN_FREE_GB: 7  # Minimum free disk space (GB) after cleanup
   # Test environment variables
   JWT_SECRET_KEY: test-secret-key-for-ci
   RAG_LLM: openai
@@ -59,7 +60,26 @@ jobs:
           virtualenvs-create: true
           virtualenvs-in-project: true
 
-      # 3️⃣ Cache Poetry dependencies for faster builds
+      # 3️⃣ Free up disk space before heavy operations
+      - name: 🧹 Free Up Disk Space
+        run: |
+          echo "Initial: $(df -h / | awk 'NR==2 {print $4}') available"
+
+          # Remove large packages in parallel
+          for p in \
+            /usr/share/dotnet \
+            /opt/ghc \
+            /usr/local/share/boost \
+            "$AGENT_TOOLSDIRECTORY" \
+            /usr/local/lib/android \
+            /usr/share/swift; do
+            if [ -e "$p" ]; then sudo rm -rf "$p" & fi
+          done
+          wait
+
+          echo "After cleanup: $(df -h / | awk 'NR==2 {print $4}') available"
+
+      # 4️⃣ Cache Poetry dependencies for faster builds
       - name: 📚 Cache Poetry dependencies
         uses: actions/cache@v4
         with:
@@ -70,11 +90,32 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-poetry-
 
-      # 4️⃣ Install Python dependencies
-      - name: 📥 Install dependencies
-        run: cd backend && poetry install --with dev,test
+      # 5️⃣ Install Python dependencies (main + test, skip dev)
+      - name: 📥 Install main + test dependencies (skip dev)
+        run: |
+          cd backend
+          # Install runtime + test deps; skip dev tools to reduce disk usage.
+          # Note: We intentionally skip the 'dev' group in unit tests to avoid
+          # heavy linters/type-checkers. pytest-cov and friends are declared in
+          # the 'test' group in pyproject.toml, so coverage still works.
+          # If coverage fails due to env drift, consider adding pytest-cov here
+          # explicitly or switching to `--with dev,test` selectively.
+          poetry install --with test --without dev
+
+      # 6️⃣ Validate free disk space after cleanup (optional)
+      - name: ✅ Ensure sufficient free disk space
+        run: |
+          THRESHOLD=${MIN_FREE_GB:-5}
+          # Compare using bytes to avoid rounding edge cases
+          FREE_BYTES=$(df -B1 / | awk 'NR==2 {print $4}')
+          THRESHOLD_BYTES=$((THRESHOLD*1024*1024*1024))
+          echo "Free space: $((FREE_BYTES/1024/1024/1024))G (threshold: ${THRESHOLD}G)"
+          if [ "$FREE_BYTES" -lt "$THRESHOLD_BYTES" ]; then
+            echo "Error: Less than ${THRESHOLD}G free after cleanup"
+            exit 1
+          fi
 
-      # 5️⃣ Run unit/atomic tests with coverage
+      # 7️⃣ Run unit/atomic tests with coverage
       - name: 🧪 Run unit tests with coverage
         run: |
           cd backend