🤖 Lazy-load AI SDK providers to reduce startup time #233
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ammar-agent
force-pushed
the
tree-shake
branch
from
30faaa1 to
aa4062f
Compare
**Problem:** App startup time was 6-13s due to eagerly loading all AI SDK providers (@ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google) at module load time. **Solution:** Implemented lazy loading strategy for AI SDK dependencies: Phase 1: Lazy-load provider-specific web search tools - Made getToolsForModel() async with dynamic imports - Providers only loaded when creating tools for that specific model Phase 2: Lazy-load provider model creation in aiService - Converted createModel() to async with dynamic imports - Anthropic and OpenAI providers loaded on-demand Phase 3: Removed unused Google provider - Google was only used for web_search tool (not chat) - Removed @ai-sdk/google dependency entirely **Regression Prevention:** Added two CI checks to prevent future regressions: 1. check_eager_imports.sh - Detects eager AI SDK imports in critical startup files (main.ts, config.ts, preload.ts) 2. check_bundle_size.sh - Fails if dist/main.js exceeds 20KB (currently 15KB, indicates eager imports if it grows) Both integrated into `make static-check` for CI enforcement. **Impact:** - Expected 50-60% reduction in startup time (6-13s → 3-6s target) - Bundle size remains at 15KB (within 20KB limit) - First message has ~200-300ms delay for provider load (acceptable) - All unit and integration tests pass **Files changed:** - src/utils/tools/tools.ts - Async tools with lazy provider imports - src/services/aiService.ts - Lazy provider creation - src/utils/ai/providerFactory.ts - NEW: Centralized provider factory - scripts/check_eager_imports.sh - NEW: Static import analysis - scripts/check_bundle_size.sh - NEW: Bundle size monitoring - Makefile - Added startup performance checks to static-check - eslint.config.mjs - Allow dynamic imports for AI SDK lazy loading - src/main.ts - Added documentation about lazy loading requirements Related: #231
ammar-agent
force-pushed
the
tree-shake
branch
from
aa4062f to
b620c0d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #231
Problem
Startup time is 6-13s because all AI SDK providers load eagerly at module initialization, even though only one provider is used per session.
Solution
Lazy-load AI SDK dependencies — providers now load on-demand using dynamic imports:
Regression Prevention
Added CI checks to prevent future regressions:
check_eager_imports.sh— fails if AI SDK imported eagerly in startup-critical filescheck_bundle_size.sh— fails if dist/main.js exceeds 20KB (currently 15KB)Both integrated into
make static-checkfor automatic enforcement.Impact
Generated with
cmux