diff --git a/README.md b/README.md index 7e9b406e..7c0f8053 100644 --- a/README.md +++ b/README.md @@ -410,6 +410,15 @@ You can always go back: +
+Telemetry – OpenSpec collects anonymous usage stats (opt-out: OPENSPEC_TELEMETRY=0) + +We collect only command names and version to understand usage patterns. No arguments, paths, content, or PII. Automatically disabled in CI. + +**Opt-out:** `export OPENSPEC_TELEMETRY=0` or `export DO_NOT_TRACK=1` + +
+ ## Contributing - Install dependencies: `pnpm install` diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/.openspec.yaml b/openspec/changes/archive/2026-01-09-add-posthog-analytics/.openspec.yaml new file mode 100644 index 00000000..68bc29d0 --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-01-10 diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/design.md b/openspec/changes/archive/2026-01-09-add-posthog-analytics/design.md new file mode 100644 index 00000000..eb5b3496 --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/design.md @@ -0,0 +1,175 @@ +## Context + +OpenSpec needs usage analytics to understand adoption and inform product decisions. PostHog provides a privacy-conscious analytics platform suitable for open source projects. + +## Goals / Non-Goals + +**Goals:** +- Track daily/weekly/monthly active usage +- Understand command usage patterns +- Keep implementation minimal and privacy-respecting +- Enable opt-out with minimal friction + +**Non-Goals:** +- Detailed error tracking or diagnostics +- User identification or profiling +- Complex event hierarchies +- Full CLI command for telemetry management (env var sufficient for now) + +## Decisions + +### Opt-Out Model + +**Decision:** Telemetry enabled by default, opt-out via environment variable. + +```bash +OPENSPEC_TELEMETRY=0 # Disable telemetry +DO_NOT_TRACK=1 # Industry standard, also respected +``` + +Auto-disabled when `CI=true` is detected. + +**Rationale:** +- Opt-in typically yields ~3% participation—not enough for meaningful data +- Understanding usage patterns requires statistically significant sample sizes +- Environment variable opt-out is simple and immediate +- Respecting `DO_NOT_TRACK` follows industry convention + +**Alternatives considered:** +- Opt-in only - Insufficient data for product decisions +- Config file setting - More complex, env var sufficient for MVP +- Full `openspec telemetry` command - Can add later if users request + +### Event Design + +**Decision:** Single event type with minimal properties. + +```typescript +{ + event: 'command_executed', + properties: { + command: 'init', // Command name only + version: '1.2.3' // OpenSpec version + } +} +``` + +**Rationale:** +- Answers the core questions: how much usage, which commands are popular +- PostHog derives DAU/WAU/MAU from anonymous user counts over time +- No arguments, paths, or content—clean privacy story +- Easy to explain in disclosure notice + +**Not tracked:** +- Command arguments +- File paths or contents +- Error messages or stack traces +- Project names or spec content +- IP addresses (`$ip: null` explicitly set) + +### Anonymous ID + +**Decision:** Random UUID, lazily generated on first telemetry send, stored in global config. + +```typescript +// ~/.config/openspec/config.json +{ + "telemetry": { + "anonymousId": "f47ac10b-58cc-4372-a567-0e02b2c3d479" + } +} +``` + +**Rationale:** +- Random UUID has no relation to the person—can't be reversed +- Stored in config so same user = same ID across sessions (needed for DAU/WAU/MAU) +- Lazy generation means no ID created if user opts out before first command +- User can delete config to reset identity + +**Alternatives considered:** +- Machine-derived hash (hostname, MAC) - Feels invasive, fingerprint-like +- Per-session UUID - Breaks user counting metrics entirely + +### SDK Configuration + +**Decision:** PostHog Node SDK with immediate flush, shutdown on exit. + +```typescript +const posthog = new PostHog(API_KEY, { + flushAt: 1, // Send immediately, don't batch + flushInterval: 0 // No timer-based flushing +}); + +// Before CLI exits +await posthog.shutdown(); +``` + +**Rationale:** +- CLI processes are short-lived; batching would lose events +- `flushAt: 1` ensures each event sends immediately +- `shutdown()` guarantees flush before process exit +- Adds ~100-300ms to exit—negligible for typical CLI workflows + +**Error handling:** +- Network failures silently ignored (telemetry shouldn't break CLI) +- `shutdown()` wrapped in try/catch + +### Hook Location + +**Decision:** Commander.js `preAction` and `postAction` hooks. + +```typescript +program + .hook('preAction', (thisCommand) => { + maybeShowTelemetryNotice(); + trackCommand(thisCommand.name(), VERSION); + }) + .hook('postAction', async () => { + await shutdown(); + }); +``` + +**Rationale:** +- Centralized—one place for all telemetry logic +- Automatic—new commands get tracked without code changes +- Clean separation—command handlers don't know about telemetry + +**Subcommand handling:** +- Track full command path for nested commands (e.g., `change:apply`) + +### First-Run Notice + +**Decision:** One-liner on first command ever, stored "seen" flag in config. + +``` +Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0 +``` + +**Rationale:** +- First command (not just `init`) ensures notice is always seen +- Non-blocking—no prompt, just informational +- One-liner is visible but not intrusive +- Storing "seen" in config prevents repeated display + +**Config after first run:** +```json +{ + "telemetry": { + "anonymousId": "...", + "noticeSeen": true + } +} +``` + +## Risks / Trade-offs + +| Risk | Mitigation | +|------|------------| +| Users prefer opt-in | Clear disclosure, trivial opt-out, transparent about what's collected | +| GDPR concerns | No personal data, no IP, user can delete config | +| Slows CLI exit by ~200ms | Negligible for most workflows; can optimize if needed | +| PostHog outage affects CLI | Fire-and-forget with timeout; failures are silent | + +## Open Questions + +None—design is intentionally minimal. Future enhancements (dedicated command, workflow tracking) can be added based on user feedback. diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/proposal.md b/openspec/changes/archive/2026-01-09-add-posthog-analytics/proposal.md new file mode 100644 index 00000000..6b026514 --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/proposal.md @@ -0,0 +1,37 @@ +## Why + +OpenSpec currently has no visibility into how the tool is being used. Without analytics, we cannot: +- Understand which commands and features are most valuable to users +- Measure adoption and usage patterns +- Make data-driven decisions about product development + +Adding PostHog analytics enables product insights while respecting user privacy through transparent, opt-out telemetry. + +## What Changes + +- Add PostHog Node.js SDK as a dependency +- Implement telemetry system with environment variable opt-out +- Track command usage (command name and version only) +- Show first-run notice informing users about telemetry +- Store anonymous ID in global config (`~/.config/openspec/config.json`) +- Respect `DO_NOT_TRACK` and `OPENSPEC_TELEMETRY=0` environment variables +- Auto-disable in CI environments + +## Capabilities + +### New Capabilities + +- `telemetry`: Anonymous usage analytics using PostHog. Covers command tracking, opt-out controls, and first-run disclosure notice. + +### Modified Capabilities + +- `global-config`: Add telemetry state storage (anonymous ID, notice seen flag) + +## Impact + +- **Dependencies**: Add `posthog-node` package +- **Privacy**: Opt-out via env var, no personal data collected, clear disclosure +- **Configuration**: New global config fields for telemetry state +- **Network**: Async event sending with flush on exit (~100-300ms added) +- **CI/CD**: Telemetry auto-disabled when `CI=true` +- **Documentation**: Update README with telemetry disclosure diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/global-config/spec.md b/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/global-config/spec.md new file mode 100644 index 00000000..77782dc0 --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/global-config/spec.md @@ -0,0 +1,21 @@ +## MODIFIED Requirements + +### Requirement: Global configuration storage +The system SHALL store global configuration in `~/.config/openspec/config.json`, including telemetry state with `anonymousId` and `noticeSeen` fields. + +#### Scenario: Initial config creation +- **WHEN** no global config file exists +- **AND** the first telemetry event is about to be sent +- **THEN** the system creates `~/.config/openspec/config.json` with telemetry configuration + +#### Scenario: Telemetry config structure +- **WHEN** reading or writing telemetry configuration +- **THEN** the config contains a `telemetry` object with `anonymousId` (string UUID) and `noticeSeen` (boolean) fields + +#### Scenario: Config file format +- **WHEN** storing configuration +- **THEN** the system writes valid JSON that can be read and modified by users + +#### Scenario: Existing config preservation +- **WHEN** adding telemetry fields to an existing config file +- **THEN** the system preserves all existing configuration fields diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/telemetry/spec.md b/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/telemetry/spec.md new file mode 100644 index 00000000..c47176fc --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/telemetry/spec.md @@ -0,0 +1,116 @@ +## ADDED Requirements + +### Requirement: Command execution tracking +The system SHALL send a `command_executed` event to PostHog when any CLI command executes, including only the command name and OpenSpec version as properties. + +#### Scenario: Standard command execution +- **WHEN** a user runs any openspec command +- **THEN** the system sends a `command_executed` event with `command` and `version` properties + +#### Scenario: Subcommand execution +- **WHEN** a user runs a nested command like `openspec change apply` +- **THEN** the system sends a `command_executed` event with the full command path (e.g., `change:apply`) + +### Requirement: Privacy-preserving event design +The system SHALL NOT include command arguments, file paths, project names, spec content, error messages, or IP addresses in telemetry events. + +#### Scenario: Command with arguments +- **WHEN** a user runs `openspec init my-project --force` +- **THEN** the telemetry event contains only `command: "init"` and `version: ""` without arguments + +#### Scenario: IP address exclusion +- **WHEN** the system sends a telemetry event +- **THEN** the event explicitly sets `$ip: null` to prevent IP tracking + +### Requirement: Environment variable opt-out +The system SHALL disable telemetry when `OPENSPEC_TELEMETRY=0` or `DO_NOT_TRACK=1` environment variables are set. + +#### Scenario: OPENSPEC_TELEMETRY opt-out +- **WHEN** `OPENSPEC_TELEMETRY=0` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: DO_NOT_TRACK opt-out +- **WHEN** `DO_NOT_TRACK=1` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: Environment variable takes precedence +- **WHEN** the user has previously used the CLI (config exists) +- **AND** the user sets `OPENSPEC_TELEMETRY=0` +- **THEN** telemetry is disabled regardless of config state + +### Requirement: CI environment auto-disable +The system SHALL automatically disable telemetry when `CI=true` environment variable is detected. + +#### Scenario: CI environment detection +- **WHEN** `CI=true` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: CI with explicit enable +- **WHEN** `CI=true` is set +- **AND** `OPENSPEC_TELEMETRY=1` is explicitly set +- **THEN** telemetry remains disabled (CI takes precedence for privacy) + +### Requirement: First-run telemetry notice +The system SHALL display a one-line telemetry disclosure notice on the first command execution, before any telemetry is sent. + +#### Scenario: First command execution +- **WHEN** a user runs their first openspec command +- **AND** telemetry is enabled +- **THEN** the system displays: "Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0" + +#### Scenario: Subsequent command execution +- **WHEN** a user has already seen the notice (noticeSeen: true in config) +- **THEN** the system does not display the notice + +#### Scenario: Notice before telemetry +- **WHEN** displaying the first-run notice +- **THEN** the notice appears before any telemetry event is sent + +### Requirement: Anonymous user identification +The system SHALL generate a random UUID as an anonymous identifier on first telemetry send, stored in global config. + +#### Scenario: First telemetry event +- **WHEN** the first telemetry event is sent +- **AND** no anonymousId exists in config +- **THEN** the system generates a random UUID v4 and stores it in config + +#### Scenario: Persistent identity +- **WHEN** a user runs multiple commands across sessions +- **THEN** the same anonymousId is used for all events + +#### Scenario: Lazy generation with opt-out +- **WHEN** a user opts out before running any command +- **THEN** no anonymousId is ever generated or stored + +### Requirement: Immediate event sending +The system SHALL send telemetry events immediately without batching, using `flushAt: 1` and `flushInterval: 0` configuration. + +#### Scenario: Event transmission timing +- **WHEN** a command executes +- **THEN** the telemetry event is sent immediately, not queued for batch transmission + +### Requirement: Graceful shutdown +The system SHALL call `posthog.shutdown()` before CLI exit to ensure pending events are flushed. + +#### Scenario: Normal exit +- **WHEN** a command completes successfully +- **THEN** the system awaits `shutdown()` before exiting + +#### Scenario: Error exit +- **WHEN** a command fails with an error +- **THEN** the system still awaits `shutdown()` before exiting + +### Requirement: Silent failure handling +The system SHALL silently ignore telemetry failures without affecting CLI functionality. + +#### Scenario: Network failure +- **WHEN** the telemetry request fails due to network error +- **THEN** the CLI command completes normally without error message + +#### Scenario: PostHog outage +- **WHEN** PostHog service is unavailable +- **THEN** the CLI command completes normally without error message + +#### Scenario: Shutdown failure +- **WHEN** `shutdown()` fails or times out +- **THEN** the CLI exits normally without error message diff --git a/openspec/changes/archive/2026-01-09-add-posthog-analytics/tasks.md b/openspec/changes/archive/2026-01-09-add-posthog-analytics/tasks.md new file mode 100644 index 00000000..972ffe9f --- /dev/null +++ b/openspec/changes/archive/2026-01-09-add-posthog-analytics/tasks.md @@ -0,0 +1,47 @@ +## 1. Setup + +- [x] 1.1 Add `posthog-node` package as a dependency +- [x] 1.2 Create `src/telemetry/` module directory +- [x] 1.3 Add PostHog API key configuration (environment variable or embedded) + +## 2. Global Config + +- [x] 2.1 Create or extend global config module for `~/.config/openspec/config.json` +- [x] 2.2 Implement read/write functions that preserve existing config fields +- [x] 2.3 Define telemetry config structure (`anonymousId`, `noticeSeen`) + +## 3. Core Telemetry Module + +- [x] 3.1 Implement `isTelemetryEnabled()` checking `OPENSPEC_TELEMETRY`, `DO_NOT_TRACK`, and `CI` env vars +- [x] 3.2 Implement `getOrCreateAnonymousId()` with lazy UUID generation +- [x] 3.3 Initialize PostHog client with `flushAt: 1` and `flushInterval: 0` +- [x] 3.4 Implement `trackCommand(commandName, version)` with `$ip: null` +- [x] 3.5 Implement `shutdown()` with try/catch for silent failure handling + +## 4. First-Run Notice + +- [x] 4.1 Implement `maybeShowTelemetryNotice()` function +- [x] 4.2 Check `noticeSeen` flag before displaying notice +- [x] 4.3 Display notice text: "Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0" +- [x] 4.4 Update `noticeSeen` in config after first display + +## 5. CLI Integration + +- [x] 5.1 Add Commander.js `preAction` hook to show notice and track command +- [x] 5.2 Add Commander.js `postAction` hook to call shutdown +- [x] 5.3 Handle subcommand path extraction (e.g., `change:apply`) + +## 6. Testing + +- [x] 6.1 Test opt-out via `OPENSPEC_TELEMETRY=0` +- [x] 6.2 Test opt-out via `DO_NOT_TRACK=1` +- [x] 6.3 Test auto-disable in CI environment +- [x] 6.4 Test first-run notice display and noticeSeen persistence +- [x] 6.5 Test anonymous ID generation and persistence +- [x] 6.6 Test silent failure on network error (mock PostHog) + +## 7. Documentation + +- [x] 7.1 Add telemetry disclosure section to README +- [x] 7.2 Document opt-out methods (`OPENSPEC_TELEMETRY=0`, `DO_NOT_TRACK=1`) +- [x] 7.3 Document what data is collected and not collected diff --git a/openspec/specs/global-config/spec.md b/openspec/specs/global-config/spec.md index b8538aad..9411fb69 100644 --- a/openspec/specs/global-config/spec.md +++ b/openspec/specs/global-config/spec.md @@ -4,6 +4,26 @@ This spec defines how OpenSpec resolves, reads, and writes user-level global configuration. It governs the `src/core/global-config.ts` module, which provides the foundation for storing user preferences, feature flags, and settings that persist across projects. The spec ensures cross-platform compatibility by following XDG Base Directory Specification with platform-specific fallbacks, and guarantees forward/backward compatibility through schema evolution rules. ## Requirements +### Requirement: Global configuration storage +The system SHALL store global configuration in `~/.config/openspec/config.json`, including telemetry state with `anonymousId` and `noticeSeen` fields. + +#### Scenario: Initial config creation +- **WHEN** no global config file exists +- **AND** the first telemetry event is about to be sent +- **THEN** the system creates `~/.config/openspec/config.json` with telemetry configuration + +#### Scenario: Telemetry config structure +- **WHEN** reading or writing telemetry configuration +- **THEN** the config contains a `telemetry` object with `anonymousId` (string UUID) and `noticeSeen` (boolean) fields + +#### Scenario: Config file format +- **WHEN** storing configuration +- **THEN** the system writes valid JSON that can be read and modified by users + +#### Scenario: Existing config preservation +- **WHEN** adding telemetry fields to an existing config file +- **THEN** the system preserves all existing configuration fields + ### Requirement: Global Config Directory Path The system SHALL resolve the global configuration directory path following XDG Base Directory Specification with platform-specific fallbacks. diff --git a/openspec/specs/telemetry/spec.md b/openspec/specs/telemetry/spec.md new file mode 100644 index 00000000..07dc851f --- /dev/null +++ b/openspec/specs/telemetry/spec.md @@ -0,0 +1,122 @@ +# telemetry Specification + +## Purpose + +This spec defines how OpenSpec collects anonymous usage telemetry to help improve the tool. It governs the `src/telemetry/` module, which handles PostHog integration, privacy-preserving event design, user opt-out mechanisms, and first-run notice display. The spec ensures telemetry is minimal, transparent, and respects user privacy. + +## Requirements + +### Requirement: Command execution tracking +The system SHALL send a `command_executed` event to PostHog when any CLI command executes, including only the command name and OpenSpec version as properties. + +#### Scenario: Standard command execution +- **WHEN** a user runs any openspec command +- **THEN** the system sends a `command_executed` event with `command` and `version` properties + +#### Scenario: Subcommand execution +- **WHEN** a user runs a nested command like `openspec change apply` +- **THEN** the system sends a `command_executed` event with the full command path (e.g., `change:apply`) + +### Requirement: Privacy-preserving event design +The system SHALL NOT include command arguments, file paths, project names, spec content, error messages, or IP addresses in telemetry events. + +#### Scenario: Command with arguments +- **WHEN** a user runs `openspec init my-project --force` +- **THEN** the telemetry event contains only `command: "init"` and `version: ""` without arguments + +#### Scenario: IP address exclusion +- **WHEN** the system sends a telemetry event +- **THEN** the event explicitly sets `$ip: null` to prevent IP tracking + +### Requirement: Environment variable opt-out +The system SHALL disable telemetry when `OPENSPEC_TELEMETRY=0` or `DO_NOT_TRACK=1` environment variables are set. + +#### Scenario: OPENSPEC_TELEMETRY opt-out +- **WHEN** `OPENSPEC_TELEMETRY=0` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: DO_NOT_TRACK opt-out +- **WHEN** `DO_NOT_TRACK=1` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: Environment variable takes precedence +- **WHEN** the user has previously used the CLI (config exists) +- **AND** the user sets `OPENSPEC_TELEMETRY=0` +- **THEN** telemetry is disabled regardless of config state + +### Requirement: CI environment auto-disable +The system SHALL automatically disable telemetry when `CI=true` environment variable is detected. + +#### Scenario: CI environment detection +- **WHEN** `CI=true` is set in the environment +- **THEN** the system sends no telemetry events + +#### Scenario: CI with explicit enable +- **WHEN** `CI=true` is set +- **AND** `OPENSPEC_TELEMETRY=1` is explicitly set +- **THEN** telemetry remains disabled (CI takes precedence for privacy) + +### Requirement: First-run telemetry notice +The system SHALL display a one-line telemetry disclosure notice on the first command execution, before any telemetry is sent. + +#### Scenario: First command execution +- **WHEN** a user runs their first openspec command +- **AND** telemetry is enabled +- **THEN** the system displays: "Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0" + +#### Scenario: Subsequent command execution +- **WHEN** a user has already seen the notice (noticeSeen: true in config) +- **THEN** the system does not display the notice + +#### Scenario: Notice before telemetry +- **WHEN** displaying the first-run notice +- **THEN** the notice appears before any telemetry event is sent + +### Requirement: Anonymous user identification +The system SHALL generate a random UUID as an anonymous identifier on first telemetry send, stored in global config. + +#### Scenario: First telemetry event +- **WHEN** the first telemetry event is sent +- **AND** no anonymousId exists in config +- **THEN** the system generates a random UUID v4 and stores it in config + +#### Scenario: Persistent identity +- **WHEN** a user runs multiple commands across sessions +- **THEN** the same anonymousId is used for all events + +#### Scenario: Lazy generation with opt-out +- **WHEN** a user opts out before running any command +- **THEN** no anonymousId is ever generated or stored + +### Requirement: Immediate event sending +The system SHALL send telemetry events immediately without batching, using `flushAt: 1` and `flushInterval: 0` configuration. + +#### Scenario: Event transmission timing +- **WHEN** a command executes +- **THEN** the telemetry event is sent immediately, not queued for batch transmission + +### Requirement: Graceful shutdown +The system SHALL call `posthog.shutdown()` before CLI exit to ensure pending events are flushed. + +#### Scenario: Normal exit +- **WHEN** a command completes successfully +- **THEN** the system awaits `shutdown()` before exiting + +#### Scenario: Error exit +- **WHEN** a command fails with an error +- **THEN** the system still awaits `shutdown()` before exiting + +### Requirement: Silent failure handling +The system SHALL silently ignore telemetry failures without affecting CLI functionality. + +#### Scenario: Network failure +- **WHEN** the telemetry request fails due to network error +- **THEN** the CLI command completes normally without error message + +#### Scenario: PostHog outage +- **WHEN** PostHog service is unavailable +- **THEN** the CLI command completes normally without error message + +#### Scenario: Shutdown failure +- **WHEN** `shutdown()` fails or times out +- **THEN** the CLI exits normally without error message diff --git a/package.json b/package.json index a7f366de..d9491bcd 100644 --- a/package.json +++ b/package.json @@ -76,6 +76,7 @@ "commander": "^14.0.0", "fast-glob": "^3.3.3", "ora": "^8.2.0", + "posthog-node": "^5.20.0", "yaml": "^2.8.2", "zod": "^4.0.17" } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 032d0d88..870e9d76 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -26,6 +26,9 @@ importers: ora: specifier: ^8.2.0 version: 8.2.0 + posthog-node: + specifier: ^5.20.0 + version: 5.20.0 yaml: specifier: ^2.8.2 version: 2.8.2 @@ -484,6 +487,9 @@ packages: '@polka/url@1.0.0-next.29': resolution: {integrity: sha512-wwQAWhWSuHaag8c4q/KN/vCoeOJYshAIvMQwD4GpSb3OiZklFfvAgmj0VCBBImRpuF/aFgIRzllXlVX93Jevww==} + '@posthog/core@1.9.1': + resolution: {integrity: sha512-kRb1ch2dhQjsAapZmu6V66551IF2LnCbc1rnrQqnR7ArooVyJN9KOPXre16AJ3ObJz2eTfuP7x25BMyS2Y5Exw==} + '@rollup/rollup-android-arm-eabi@4.46.2': resolution: {integrity: sha512-Zj3Hl6sN34xJtMv7Anwb5Gu01yujyE/cLBDB2gnHTAHaWS1Z38L7kuSG+oAh0giZMqG060f/YBStXtMH6FvPMA==} cpu: [arm] @@ -1284,6 +1290,10 @@ packages: resolution: {integrity: sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==} engines: {node: ^10 || ^12 || >=14} + posthog-node@5.20.0: + resolution: {integrity: sha512-LkR5KfrvEQTnUtNKN97VxFB00KcYG1Iz8iKg8r0e/i7f1eQhg1WSZO+Jp1B4bvtHCmdpIE4HwYbvCCzFoCyjVg==} + engines: {node: '>=20'} + prelude-ls@1.2.1: resolution: {integrity: sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==} engines: {node: '>= 0.8.0'} @@ -2038,6 +2048,10 @@ snapshots: '@polka/url@1.0.0-next.29': {} + '@posthog/core@1.9.1': + dependencies: + cross-spawn: 7.0.6 + '@rollup/rollup-android-arm-eabi@4.46.2': optional: true @@ -2808,6 +2822,10 @@ snapshots: picocolors: 1.1.1 source-map-js: 1.2.1 + posthog-node@5.20.0: + dependencies: + '@posthog/core': 1.9.1 + prelude-ls@1.2.1: {} prettier@2.8.8: {} diff --git a/src/cli/index.ts b/src/cli/index.ts index 6dd2ac29..be2620f0 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -15,11 +15,32 @@ import { ShowCommand } from '../commands/show.js'; import { CompletionCommand } from '../commands/completion.js'; import { registerConfigCommand } from '../commands/config.js'; import { registerArtifactWorkflowCommands } from '../commands/artifact-workflow.js'; +import { maybeShowTelemetryNotice, trackCommand, shutdown } from '../telemetry/index.js'; const program = new Command(); const require = createRequire(import.meta.url); const { version } = require('../../package.json'); +/** + * Get the full command path for nested commands. + * For example: 'change show' -> 'change:show' + */ +function getCommandPath(command: Command): string { + const names: string[] = []; + let current: Command | null = command; + + while (current) { + const name = current.name(); + // Skip the root 'openspec' command + if (name && name !== 'openspec') { + names.unshift(name); + } + current = current.parent; + } + + return names.join(':') || 'openspec'; +} + program .name('openspec') .description('AI-native system for spec-driven development') @@ -28,12 +49,24 @@ program // Global options program.option('--no-color', 'Disable color output'); -// Apply global flags before any command runs -program.hook('preAction', (thisCommand) => { +// Apply global flags and telemetry before any command runs +program.hook('preAction', async (thisCommand) => { const opts = thisCommand.opts(); if (opts.color === false) { process.env.NO_COLOR = '1'; } + + // Show first-run telemetry notice (if not seen) + await maybeShowTelemetryNotice(); + + // Track command execution + const commandPath = getCommandPath(thisCommand); + await trackCommand(commandPath, version); +}); + +// Shutdown telemetry after command completes +program.hook('postAction', async () => { + await shutdown(); }); const availableToolIds = AI_TOOLS.filter((tool) => tool.available).map((tool) => tool.value); diff --git a/src/telemetry/config.ts b/src/telemetry/config.ts new file mode 100644 index 00000000..a79704f0 --- /dev/null +++ b/src/telemetry/config.ts @@ -0,0 +1,85 @@ +/** + * Global configuration for telemetry state. + * Stores anonymous ID and notice-seen flag in ~/.config/openspec/config.json + */ +import { promises as fs } from 'fs'; +import path from 'path'; +import os from 'os'; + +export interface TelemetryConfig { + anonymousId?: string; + noticeSeen?: boolean; +} + +export interface GlobalConfig { + telemetry?: TelemetryConfig; + [key: string]: unknown; // Preserve other fields +} + +/** + * Get the path to the global config file. + * Uses ~/.config/openspec/config.json on all platforms. + */ +export function getConfigPath(): string { + const configDir = path.join(os.homedir(), '.config', 'openspec'); + return path.join(configDir, 'config.json'); +} + +/** + * Read the global config file. + * Returns an empty object if the file doesn't exist. + */ +export async function readConfig(): Promise { + const configPath = getConfigPath(); + try { + const content = await fs.readFile(configPath, 'utf-8'); + return JSON.parse(content) as GlobalConfig; + } catch (error: unknown) { + if ((error as NodeJS.ErrnoException).code === 'ENOENT') { + return {}; + } + // If parse fails or other error, return empty config + return {}; + } +} + +/** + * Write to the global config file. + * Preserves existing fields and merges in new values. + */ +export async function writeConfig(updates: Partial): Promise { + const configPath = getConfigPath(); + const configDir = path.dirname(configPath); + + // Ensure directory exists + await fs.mkdir(configDir, { recursive: true }); + + // Read existing config and merge + const existing = await readConfig(); + const merged = { ...existing, ...updates }; + + // Deep merge for telemetry object + if (updates.telemetry && existing.telemetry) { + merged.telemetry = { ...existing.telemetry, ...updates.telemetry }; + } + + await fs.writeFile(configPath, JSON.stringify(merged, null, 2) + '\n'); +} + +/** + * Get the telemetry config section. + */ +export async function getTelemetryConfig(): Promise { + const config = await readConfig(); + return config.telemetry ?? {}; +} + +/** + * Update the telemetry config section. + */ +export async function updateTelemetryConfig(updates: Partial): Promise { + const existing = await getTelemetryConfig(); + await writeConfig({ + telemetry: { ...existing, ...updates }, + }); +} diff --git a/src/telemetry/index.ts b/src/telemetry/index.ts new file mode 100644 index 00000000..753db89f --- /dev/null +++ b/src/telemetry/index.ts @@ -0,0 +1,161 @@ +/** + * Telemetry module for anonymous usage analytics. + * + * Privacy-first design: + * - Only tracks command name and version + * - No arguments, file paths, or content + * - Opt-out via OPENSPEC_TELEMETRY=0 or DO_NOT_TRACK=1 + * - Auto-disabled in CI environments + * - Anonymous ID is a random UUID with no relation to the user + */ +import { PostHog } from 'posthog-node'; +import { randomUUID } from 'crypto'; +import { getTelemetryConfig, updateTelemetryConfig } from './config.js'; + +// PostHog API key - public key for client-side analytics +// This is safe to embed as it only allows sending events, not reading data +const POSTHOG_API_KEY = 'phc_Hthu8YvaIJ9QaFKyTG4TbVwkbd5ktcAFzVTKeMmoW2g'; +// Using reverse proxy to avoid ad blockers and keep traffic on our domain +const POSTHOG_HOST = 'https://edge.openspec.dev'; + +let posthogClient: PostHog | null = null; +let anonymousId: string | null = null; + +/** + * Check if telemetry is enabled. + * + * Disabled when: + * - OPENSPEC_TELEMETRY=0 + * - DO_NOT_TRACK=1 + * - CI=true (any CI environment) + */ +export function isTelemetryEnabled(): boolean { + // Check explicit opt-out + if (process.env.OPENSPEC_TELEMETRY === '0') { + return false; + } + + // Respect DO_NOT_TRACK standard + if (process.env.DO_NOT_TRACK === '1') { + return false; + } + + // Auto-disable in CI environments + if (process.env.CI === 'true') { + return false; + } + + return true; +} + +/** + * Get or create the anonymous user ID. + * Lazily generates a UUID on first call and persists it. + */ +export async function getOrCreateAnonymousId(): Promise { + // Return cached value if available + if (anonymousId) { + return anonymousId; + } + + // Try to load from config + const config = await getTelemetryConfig(); + if (config.anonymousId) { + anonymousId = config.anonymousId; + return anonymousId; + } + + // Generate new UUID and persist + anonymousId = randomUUID(); + await updateTelemetryConfig({ anonymousId }); + return anonymousId; +} + +/** + * Get the PostHog client instance. + * Creates it on first call with CLI-optimized settings. + */ +function getClient(): PostHog { + if (!posthogClient) { + posthogClient = new PostHog(POSTHOG_API_KEY, { + host: POSTHOG_HOST, + flushAt: 1, // Send immediately, don't batch + flushInterval: 0, // No timer-based flushing + }); + } + return posthogClient; +} + +/** + * Track a command execution. + * + * @param commandName - The command name (e.g., 'init', 'change:apply') + * @param version - The OpenSpec version + */ +export async function trackCommand(commandName: string, version: string): Promise { + if (!isTelemetryEnabled()) { + return; + } + + try { + const userId = await getOrCreateAnonymousId(); + const client = getClient(); + + client.capture({ + distinctId: userId, + event: 'command_executed', + properties: { + command: commandName, + version: version, + surface: 'cli', + $ip: null, // Explicitly disable IP tracking + }, + }); + } catch { + // Silent failure - telemetry should never break CLI + } +} + +/** + * Show first-run telemetry notice if not already seen. + */ +export async function maybeShowTelemetryNotice(): Promise { + if (!isTelemetryEnabled()) { + return; + } + + try { + const config = await getTelemetryConfig(); + if (config.noticeSeen) { + return; + } + + // Display notice + console.log( + 'Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0' + ); + + // Mark as seen + await updateTelemetryConfig({ noticeSeen: true }); + } catch { + // Silent failure - telemetry should never break CLI + } +} + +/** + * Shutdown the PostHog client and flush pending events. + * Call this before CLI exit. + */ +export async function shutdown(): Promise { + if (!posthogClient) { + return; + } + + try { + await posthogClient.shutdown(); + } catch { + // Silent failure - telemetry should never break CLI exit + } finally { + posthogClient = null; + } +} diff --git a/test/telemetry/config.test.ts b/test/telemetry/config.test.ts new file mode 100644 index 00000000..b4db4c7e --- /dev/null +++ b/test/telemetry/config.test.ts @@ -0,0 +1,180 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import * as os from 'node:os'; + +import { + getConfigPath, + readConfig, + writeConfig, + getTelemetryConfig, + updateTelemetryConfig, +} from '../../src/telemetry/config.js'; + +describe('telemetry/config', () => { + let tempDir: string; + let originalHome: string | undefined; + + beforeEach(() => { + // Create temp directory for tests + tempDir = path.join(os.tmpdir(), `openspec-telemetry-test-${Date.now()}`); + fs.mkdirSync(tempDir, { recursive: true }); + + // Mock HOME to point to temp dir + originalHome = process.env.HOME; + process.env.HOME = tempDir; + }); + + afterEach(() => { + // Restore HOME + process.env.HOME = originalHome; + + // Clean up temp directory + fs.rmSync(tempDir, { recursive: true, force: true }); + }); + + describe('getConfigPath', () => { + it('should return path to config.json in .config/openspec', () => { + const result = getConfigPath(); + expect(result).toBe(path.join(tempDir, '.config', 'openspec', 'config.json')); + }); + }); + + describe('readConfig', () => { + it('should return empty object when config file does not exist', async () => { + const config = await readConfig(); + expect(config).toEqual({}); + }); + + it('should load valid config from file', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, JSON.stringify({ + telemetry: { anonymousId: 'test-id', noticeSeen: true } + })); + + const config = await readConfig(); + expect(config.telemetry).toEqual({ anonymousId: 'test-id', noticeSeen: true }); + }); + + it('should return empty object for invalid JSON', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, '{ invalid json }'); + + const config = await readConfig(); + expect(config).toEqual({}); + }); + }); + + describe('writeConfig', () => { + it('should create directory if it does not exist', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + + await writeConfig({ telemetry: { noticeSeen: true } }); + + expect(fs.existsSync(configDir)).toBe(true); + }); + + it('should write config to file', async () => { + const configPath = path.join(tempDir, '.config', 'openspec', 'config.json'); + + await writeConfig({ telemetry: { anonymousId: 'test-123' } }); + + const content = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(content); + expect(parsed.telemetry.anonymousId).toBe('test-123'); + }); + + it('should preserve existing fields when updating', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + // Create initial config with other fields + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, JSON.stringify({ + existingField: 'preserved', + telemetry: { anonymousId: 'old-id' } + })); + + // Update telemetry + await writeConfig({ telemetry: { noticeSeen: true } }); + + const content = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(content); + expect(parsed.existingField).toBe('preserved'); + expect(parsed.telemetry.noticeSeen).toBe(true); + }); + + it('should deep merge telemetry fields', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + // Create initial config + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, JSON.stringify({ + telemetry: { anonymousId: 'existing-id' } + })); + + // Update with noticeSeen only + await writeConfig({ telemetry: { noticeSeen: true } }); + + const content = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(content); + expect(parsed.telemetry.anonymousId).toBe('existing-id'); + expect(parsed.telemetry.noticeSeen).toBe(true); + }); + }); + + describe('getTelemetryConfig', () => { + it('should return empty object when no config exists', async () => { + const config = await getTelemetryConfig(); + expect(config).toEqual({}); + }); + + it('should return telemetry section from config', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, JSON.stringify({ + telemetry: { anonymousId: 'my-id', noticeSeen: false } + })); + + const config = await getTelemetryConfig(); + expect(config).toEqual({ anonymousId: 'my-id', noticeSeen: false }); + }); + }); + + describe('updateTelemetryConfig', () => { + it('should create telemetry config when none exists', async () => { + await updateTelemetryConfig({ anonymousId: 'new-id' }); + + const configPath = path.join(tempDir, '.config', 'openspec', 'config.json'); + const content = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(content); + expect(parsed.telemetry.anonymousId).toBe('new-id'); + }); + + it('should merge with existing telemetry config', async () => { + const configDir = path.join(tempDir, '.config', 'openspec'); + const configPath = path.join(configDir, 'config.json'); + + fs.mkdirSync(configDir, { recursive: true }); + fs.writeFileSync(configPath, JSON.stringify({ + telemetry: { anonymousId: 'existing-id' } + })); + + await updateTelemetryConfig({ noticeSeen: true }); + + const content = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(content); + expect(parsed.telemetry.anonymousId).toBe('existing-id'); + expect(parsed.telemetry.noticeSeen).toBe(true); + }); + }); +}); diff --git a/test/telemetry/index.test.ts b/test/telemetry/index.test.ts new file mode 100644 index 00000000..ab41dab2 --- /dev/null +++ b/test/telemetry/index.test.ts @@ -0,0 +1,135 @@ +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'; +import * as fs from 'node:fs'; +import * as path from 'node:path'; +import * as os from 'node:os'; +import { randomUUID } from 'node:crypto'; + +// Mock posthog-node before importing the module +vi.mock('posthog-node', () => { + return { + PostHog: vi.fn().mockImplementation(() => ({ + capture: vi.fn(), + shutdown: vi.fn().mockResolvedValue(undefined), + })), + }; +}); + +// Import after mocking +import { isTelemetryEnabled, maybeShowTelemetryNotice, shutdown, trackCommand } from '../../src/telemetry/index.js'; +import { PostHog } from 'posthog-node'; + +describe('telemetry/index', () => { + let tempDir: string; + let originalEnv: NodeJS.ProcessEnv; + let consoleLogSpy: ReturnType; + + beforeEach(() => { + // Create unique temp directory for each test using UUID + tempDir = path.join(os.tmpdir(), `openspec-telemetry-test-${randomUUID()}`); + fs.mkdirSync(tempDir, { recursive: true }); + + // Save original env + originalEnv = { ...process.env }; + + // Mock HOME to point to temp dir + process.env.HOME = tempDir; + + // Clear all mocks + vi.clearAllMocks(); + + // Spy on console.log for notice tests + consoleLogSpy = vi.spyOn(console, 'log').mockImplementation(() => {}); + }); + + afterEach(() => { + // Restore original env + process.env = originalEnv; + + // Clean up temp directory + try { + fs.rmSync(tempDir, { recursive: true, force: true }); + } catch { + // Ignore cleanup errors + } + + // Restore all mocks + vi.restoreAllMocks(); + }); + + describe('isTelemetryEnabled', () => { + it('should return false when OPENSPEC_TELEMETRY=0', () => { + process.env.OPENSPEC_TELEMETRY = '0'; + expect(isTelemetryEnabled()).toBe(false); + }); + + it('should return false when DO_NOT_TRACK=1', () => { + process.env.DO_NOT_TRACK = '1'; + expect(isTelemetryEnabled()).toBe(false); + }); + + it('should return false when CI=true', () => { + process.env.CI = 'true'; + expect(isTelemetryEnabled()).toBe(false); + }); + + it('should return true when no opt-out is set', () => { + delete process.env.OPENSPEC_TELEMETRY; + delete process.env.DO_NOT_TRACK; + delete process.env.CI; + expect(isTelemetryEnabled()).toBe(true); + }); + + it('should prioritize OPENSPEC_TELEMETRY=0 over other settings', () => { + process.env.OPENSPEC_TELEMETRY = '0'; + delete process.env.DO_NOT_TRACK; + delete process.env.CI; + expect(isTelemetryEnabled()).toBe(false); + }); + }); + + describe('maybeShowTelemetryNotice', () => { + it('should not show notice when telemetry is disabled', async () => { + process.env.OPENSPEC_TELEMETRY = '0'; + + await maybeShowTelemetryNotice(); + + expect(consoleLogSpy).not.toHaveBeenCalled(); + }); + }); + + describe('trackCommand', () => { + it('should not track when telemetry is disabled', async () => { + process.env.OPENSPEC_TELEMETRY = '0'; + + await trackCommand('test', '1.0.0'); + + expect(PostHog).not.toHaveBeenCalled(); + }); + + it('should track when telemetry is enabled', async () => { + delete process.env.OPENSPEC_TELEMETRY; + delete process.env.DO_NOT_TRACK; + delete process.env.CI; + + await trackCommand('test', '1.0.0'); + + expect(PostHog).toHaveBeenCalled(); + }); + }); + + describe('shutdown', () => { + it('should not throw when no client exists', async () => { + await expect(shutdown()).resolves.not.toThrow(); + }); + + it('should handle shutdown errors silently', async () => { + const mockPostHog = { + capture: vi.fn(), + shutdown: vi.fn().mockRejectedValue(new Error('Network error')), + }; + (PostHog as any).mockImplementation(() => mockPostHog); + + await expect(shutdown()).resolves.not.toThrow(); + }); + }); +});