FullTradingPlat — Lean MVP

This repo hosts a lean, multi‑venue trading dashboard MVP designed to run on an M1 Mac (8 cores, ~8 GB RAM) and a $100–$200/month vendor budget.

Quick start

• Prefer Poetry + pyenv. If pyenv isn’t installed, you can use the fallback requirements.txt with a local venv.

macOS setup (pyenv + Poetry)

1. Install pyenv (Homebrew):
   • brew update && brew install pyenv
   • Add to shell: echo 'eval "$(pyenv init -)"' >> ~/.zshrc && source ~/.zshrc
2. Install Python and set local version:
   • pyenv install 3.11.9
   • pyenv local 3.11.9 (repo already contains .python-version)
3. Install Poetry: curl -sSL https://install.python-poetry.org | python3 -
4. Install deps: poetry install
5. Run the app: poetry run uvicorn ui.app:app --reload
   • Open http://127.0.0.1:8000/ (redirects to /ui).
   • Health HTML: http://127.0.0.1:8000/health
   • Health JSON: http://127.0.0.1:8000/healthz (add ?pretty=1 for a formatted HTML view)

Fallback (no pyenv/Poetry)

• python3 -m venv .venv && source .venv/bin/activate
• pip install -r requirements.txt
• uvicorn ui.app:app --reload
  • Open http://127.0.0.1:8000/ (redirects to /ui).
  • Or visit http://127.0.0.1:8000/healthz.

Docs

• See docs/lean_kickoff_plan.md for the phased plan.
• Agent contracts in docs/agent_build_manifest.md.
• Checklist in docs/lean_kickoff_checklist.md.
• Roadmap: docs/roadmap_next_stages.md (stages 1–6 toward full spec).
• Future dev checklist: docs/future_development_checklist.md for ongoing tracking.

Optional: Integration Harness (Docker Compose)

• Compose is not required for day‑to‑day development. Use it only when you want a local multi‑service stack (UI, execution, ingest) wired to Postgres/NATS for integration smoke.
• Details and commands were moved to docs/compose_harness.md.

UI preview

• Visit http://127.0.0.1:8000/ui. Example /healthz?pretty=1 output (HTML-wrapped JSON):

  {
    "status": "ok",
    "services": ["ingest", "strategy", "execution", "ui"],
    "latest_orderbook_ts": 1699999999999,
    "latest_trade_ts": 1699999999999
  }

  • Live spreads: /spreads?symbol=BTC-USD (WebSocket-backed; updates ~1s)

Config

• To disable specific venues from appearing in /spreads, add a comma‑separated list to .env, e.g. DISABLED_VENUES=paradex.

Sandbox (Coinbase Advanced Trade)

• Default mode is SIM (no external calls). To enable real Advanced Trade sandbox submissions set:
  • ENABLE_COINBASE_SANDBOX=1
  • COINBASE_API_BASE (e.g. https://api-sandbox.coinbase.com/api/v3)
  • COINBASE_API_KEY, COINBASE_API_SECRET, COINBASE_API_PASSPHRASE
• All four must be present; otherwise the execution service refuses to start in sandbox mode.
• Keep SIM as default for local dev; only flip the flag when you are ready to use sandbox credentials.

Troubleshooting

• If ccxt not installed appears when running fetch_orderbook.py, install requirements first:
  • source .venv/bin/activate && pip install -r requirements.txt
• If WebSocket (ws_stream.py) prints nothing, ensure you’re on the Advanced Trade feed and on a popular product (default BTC-USD). Network restrictions or firewall can also block websockets.

Make targets

• make venv && make install — create venv and install deps (Python 3.11 recommended)
• make run — start uvicorn (/ redirects to /ui)
• make run-exec — start the execution API on :9100 (HTML index explains it’s an API; UI is at :8000/ui)
• make run-ingest — start the ingest API on :9101 (HTML index + health endpoint)
• make run-single — run the UI and mount both APIs under /api/execution and /api/ingest on a single port (:8000)
• make health — run health probe
• make fetch — short REST snapshot run
• make ws — short WebSocket run
• make db — print DB status (counts + latest timestamps)
• make dex — pull Uniswap v3 (USDC/WETH) price via The Graph and persist as ETH-USD snapshots
• make dex-onchain — query Uniswap v3 pool slot0 via ETH_RPC_URL from your .env and persist ETH-USD snapshots (DEX-native)
• alembic upgrade head — apply DB schema (uses DATABASE_URL/POSTGRES_DSN or falls back to SQLite file)

Testing

• Run tests in a clean environment: make test (uses PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 to avoid third‑party plugin conflicts)

PostgreSQL + Alembic

• Start Postgres locally: docker compose up -d db
• Set DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/trading
• Apply migrations: alembic upgrade head
• Switch back to SQLite by unsetting DATABASE_URL.

Metrics

• Prometheus endpoint at /metrics (counters + request latency histogram)
• Add to prometheus.yml scrape configs: - targets: ['localhost:8000']

Alerts and recon

• Test alert: python scripts/send_alert.py --type test (uses SLACK_WEBHOOK_URL)
• Daily reconcile: python scripts/reconcile.py --date YYYY-MM-DD
• Inspect reservations + mark positions: python scripts/admin_reservations_inspector.py --status active --marks --mark-now
• UI view for MTM marks: http://127.0.0.1:8000/positions/marks (click Snapshot MTM to record a mark)
• Stage D dry-run (30 min): make live-dry-30 (keeps live_trade.py in dry-run mode while checking spreads every second)

Paper loop quick demo (100 trades)

• Seed snapshots: python scripts/seed_snapshots.py --symbol BTC-USD --rows 200
• Run: TARGET_ORDERS=100 python services/strategy/paper_loop.py --symbol BTC-USD --duration 30 --threshold-bps 0.01 --venue-buy coinbase --venue-sell kraken
• Optional granular orders: PAPER_SIM_ORDERS=1 ... to populate paper_orders/paper_fills.

SQLite retention

• Keep DB size small: python scripts/db_retention.py --hours 24 (then VACUUM automatically for SQLite)

Data exports

• Export Parquet for a given day: python scripts/export_parquet.py --tables orderbook_snapshots,trades --start-date 2025-11-14 --end-date 2025-11-14 --output data/exports
• Schemas documented in docs/data_schema.md

Replays & backtesting (Stage 4)

• Start replay service: make run-replay (exposes POST /replay on :9102)
• Batch replay: python scripts/replay.py --path data/exports/orderbook_snapshots --symbol BTC-USD --speed 1
• Backtest harness: python services/strategy/backtest.py --path <parquet> --symbol BTC-USD --threshold-bps 1.0
• Config-driven example: python services/strategy/backtest.py --config configs/backtest_spread_capture.json (uses backtests/datasets/sample_snapshots.parquet and compares to backtests/golden/spread_capture_sample.json).
• Stream mode: while make run-replay (or live ingest) is running, you can collect ticks directly: python services/strategy/backtest.py --stream-duration 30 --symbol BTC-USD --threshold-bps 0.5.

DEX notes

• The included DEX connector queries Uniswap v3’s public subgraph for the USDC/WETH 0.3% pool and writes a synthetic orderbook for ETH-USD with a tiny ±5 bps half‑spread.
• View cross‑venue spreads for ETH with: http://127.0.0.1:8000/spreads?symbol=ETH-USD (after running make dex, plus make fetch and make kraken for ETH-USD).
• For a DEX-native source without 3rd‑party APIs, add ETH_RPC_URL=... to your .env and run make dex-onchain. This reads the Uniswap v3 pool’s slot0 directly and computes price.

Single‑port mode

• Add SINGLE_PORT_MODE=1 to .env (or run make run-single).
• The execution and ingest APIs will be available under /api/execution and /api/ingest on the main UI port (:8000).