geoprobe: telemetry agent extensions — probe measurement and offset delivery

[nikw9944]

RFC Reference: rfc16-geolocation-verification.md — Phase 1, item 1 (Telemetry agent extensions)
Tracking issue: #2893

Goal

Extend the existing telemetry agent to measure RTT to child geoProbes via TWAMP, generate signed Offset structures, and deliver them via UDP.

Scope

• New CLI flag --additional-child-probes (comma-separated host:port list) on the telemetry agent binary (controlplane/telemetry/cmd/telemetry/main.go)
• New module controlplane/telemetry/internal/geoprobe/pinger.go: periodic TWAMP measurement to configured probes, reusing existing twamplight.Sender interface from tools/twamp/pkg/light/
• New module controlplane/telemetry/internal/geoprobe/publisher.go: generates signed Offset (using DZD's metrics_publisher_pk keypair, lat/lng from device config) and sends via UDP to each probe
• Integration into collector.go: add a new goroutine alongside the existing pinger/submitter goroutines, gated on --additional-child-probes being non-empty
• Unit tests for the probe pinger and publisher modules

Integration points

• collector.go (lines ~83-168): Add goroutine for geoprobe measurement loop, similar to existing pinger goroutine pattern
• cmd/telemetry/main.go: Add CLI flags, pass config to collector
• Reuses twamplight.NewSender for TWAMP probing (same interface as existing device probing)
• Reuses existing solana.PrivateKey (loaded from --keypair) for offset signing

Files to change

• controlplane/telemetry/cmd/telemetry/main.go — new flags, wire up geoprobe config
• controlplane/telemetry/internal/telemetry/collector.go — new goroutine for probe measurement
• controlplane/telemetry/internal/telemetry/config.go — new config fields

Files to create

• controlplane/telemetry/internal/geoprobe/pinger.go
• controlplane/telemetry/internal/geoprobe/publisher.go
• controlplane/telemetry/internal/geoprobe/pinger_test.go
• controlplane/telemetry/internal/geoprobe/publisher_test.go

Estimated scope

Medium (~300–500 lines)

Dependencies

Depends on #2898 (shared geoprobe library).

[nikw9944]

Approved Work Plan (Revision 2)

Estimated Scope: Medium

Work Plan: Issue #2899 - Geoprobe Telemetry Agent Extensions (Revision 2)

Issue: #2899
Branch: ben-dz/doublezero-2899
RFC: RFC16 - Geolocation Verification
Tracking Issue: #2893
Dependencies: #2898 (shared geoprobe library - available on feature/geolocation-start branch)

Summary

Extend the existing telemetry agent to measure RTT to child geoProbes via TWAMP, generate signed LocationOffset structures, and deliver them via UDP. This is Phase 1, item 1 of the geolocation verification system described in RFC16.

The telemetry agent currently measures device-to-device latency using TWAMP. This work adds parallel functionality to measure latency to configured geoProbes (bare metal servers that act as intermediaries for geolocation verification). The agent will:

1. Maintain a unified list of child probes populated from multiple sources (initially CLI flag, later onchain configuration)
2. Accept a CLI flag --additional-child-probes with comma-separated host:port values to populate the child probes list
3. Support dynamic probe list updates at runtime (new probes can be added while the agent is running)
4. Periodically probe these geoProbes using TWAMP (reusing existing twamplight.Sender infrastructure)
5. Generate signed LocationOffset datagrams containing lat/lng (from device config), RTT measurements, and Ed25519 signatures
6. Send these offsets via UDP to each probe for caching and further processing

All runtime code execution (except parsing the CLI argument) will be gated on a non-empty child probes list, ensuring zero overhead when the feature is disabled.

Approach

Design Decisions

1. Reuse Existing Patterns: Follow the established collector pattern with separate pinger and publisher goroutines, similar to the existing device telemetry flow (pinger → buffer → submitter).

2. Dynamic Probe List with Channel Updates: Use a channel-based architecture to support adding new probes at runtime. The main geoprobe goroutine listens for probe updates and dynamically adjusts the probe list without requiring agent restart.

3. Configuration via CLI Flag: Use --additional-child-probes flag for POC testing without requiring onchain probe configuration. This enables rapid testing in Phase 1 before Phase 2 adds onchain discovery.

4. Parallel Goroutine Architecture: Add a new goroutine in collector.go that runs independently alongside existing telemetry goroutines, gated on the presence of child probes in the initial list.

5. Dependency on geoprobe Library: The implementation assumes the geoprobe library from geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 is merged (or we merge it into our branch). The library provides:

   • LocationOffset struct and serialization
   • OffsetSigner for Ed25519 signing
   • SendOffset() for UDP transport

6. Lat/Lng Caching: Retrieve latitude and longitude from the Device account onchain once at startup. Cache the values for 24 hours. If retrieval fails, retry on next measurement cycle. If lat/lng are missing, skip offset generation (non-fatal error).

7. Slot Retrieval Fallback: Retrieve current slot from ledger. If retrieval fails, use the last successfully retrieved slot as long as it was obtained within the last 5 minutes. If no recent slot is available, skip offset generation for that measurement cycle (non-fatal error).

8. Minimal RTT Selection: Use the minimum RTT observed during the measurement interval (matching existing telemetry behavior).

9. UDP Socket Management: Create a single UDP connection per probe, reused for all offset sends to that probe. Dynamically create new sockets when new probes are added.

Implementation Structure

The implementation follows the existing telemetry agent architecture:

collector.go
    ├─> reflector (existing)
    ├─> peer discovery (existing)
    ├─> device pinger (existing)
    ├─> device submitter (existing)
    └─> geoprobe measurement loop (NEW)
            ├─> probe list manager (handles dynamic updates)
            ├─> geoprobe/pinger.go (probe RTT measurement)
            └─> geoprobe/publisher.go (offset generation + UDP send)

Dynamic Probe List Architecture

The agent will maintain a dynamic probe list that can be updated at runtime:

Initial Population (Phase 1):

• CLI flag --additional-child-probes provides the initial probe list
• Feature gating: geoprobe goroutine only starts if initial list is non-empty

Runtime Updates:

• A channel (probeUpdateCh chan []ProbeAddress) receives probe list updates
• The geoprobe goroutine listens on this channel and updates its internal state
• New probes are added to the pinger and publisher (new UDP sockets created)
• Removed probes have their resources cleaned up (UDP sockets closed)

Phase 2 (future work):

• Add onchain probe discovery goroutine
• Periodically query serviceability program for assigned child probes
• Merge onchain probes with CLI-provided probes
• Send merged list via probeUpdateCh
• Geoprobe measurement code remains unchanged

This design ensures clean separation between probe source management and probe measurement logic, and supports dynamic updates without agent restart.

Caching Strategy

Lat/Lng Caching:

• Retrieve device lat/lng from serviceability program once at startup
• Cache for 24 hours (configurable via constant)
• On cache expiry, fetch fresh values in background
• If fetch fails, continue using cached values (log warning)
• If lat/lng are never successfully retrieved, skip offset generation (log error)

Slot Caching:

• Attempt to retrieve current slot from ledger on each measurement cycle
• If retrieval fails, check if last successful slot is <5 minutes old
• If yes, use cached slot; if no, skip offset generation for this cycle (log warning)
• This prevents blocking offset generation on transient RPC failures

Key Components

1. CLI Flag Parsing (cmd/telemetry/main.go)

   • Add --additional-child-probes flag accepting comma-separated host:port values
   • Parse into slice of probe addresses
   • Pass to Config.InitialChildProbes (renamed from ChildProbes to clarify it's the startup list)

2. Config Extension (internal/telemetry/config.go)

   • Add InitialChildProbes []ProbeAddress field (startup probe list)
   • ProbeAddress struct: {Host string, Port uint16}
   • Validation: ensure valid host:port format
   • Add comment documenting that this is the initial list; runtime updates happen via channel

3. Collector Integration (internal/telemetry/collector.go)

   • Add probeUpdateCh channel for dynamic probe list updates
   • Add goroutine launch in Run() method, gated on len(c.cfg.InitialChildProbes) > 0
   • Pattern: similar to existing pinger/submitter goroutines
   • Lifecycle: start with wg.Add(1), clean shutdown via context
   • Pass probeUpdateCh to geoprobe measurement loop

4. GeoProbe Pinger (internal/geoprobe/pinger.go)

   • Maintains map of active probes and their TWAMP senders
   • Periodic TWAMP measurement to configured probes
   • Reuses twamplight.NewSender (same interface as device pinger)
   • Records minimum RTT per interval per probe
   • Supports dynamic probe addition/removal:
     • AddProbe(addr ProbeAddress): creates new TWAMP sender
     • RemoveProbe(addr ProbeAddress): cleans up sender resources
   • Returns measurements to publisher

5. GeoProbe Publisher (internal/geoprobe/publisher.go)

   • Lat/Lng Management:
     • Retrieves device lat/lng from serviceability program at startup
     • Caches for 24 hours
     • Implements refreshLatLng() method for background refresh
     • If lat/lng unavailable, skips offset generation (non-fatal)
   • Slot Management:
     • Attempts to retrieve current slot from ledger on each cycle
     • Falls back to cached slot if <5 minutes old
     • If no valid slot, skips offset generation for that cycle (non-fatal)
   • Offset Generation:
     • Creates LocationOffset struct with:
       • Pubkey: metrics_publisher_pk from keypair
       • MeasurementSlot: current or cached slot
       • Lat/Lng: cached device location
       • MeasuredRttNs: minimum RTT from pinger
       • RttNs: same as MeasuredRttNs (no reference chain for DZD)
       • NumReferences: 0 (DZDs are roots of trust)
       • References: empty slice
   • Signs offset using existing keypair via geoprobe.OffsetSigner
   • Sends via UDP using geoprobe.SendOffset()
   • Maintains UDP connection pool (one per probe)
   • Supports dynamic probe addition (creates new UDP connections)

6. GeoProbe Coordinator (internal/geoprobe/coordinator.go)

   • Main control loop for geoprobe measurement
   • Listens on probeUpdateCh for probe list updates
   • Manages pinger and publisher lifecycle
   • Handles probe addition/removal events:
     • New probes: calls pinger.AddProbe() and creates UDP socket
     • Removed probes: calls pinger.RemoveProbe() and closes UDP socket
   • Coordinates measurement cycles between pinger and publisher

7. Unit Tests

   • coordinator_test.go: test dynamic probe addition/removal, channel updates
   • pinger_test.go: test TWAMP probe creation, RTT measurement, error handling, dynamic probe management
   • publisher_test.go: test offset generation, signing, UDP send, lat/lng caching, slot fallback

Files to Change

Modify Existing Files

1. controlplane/telemetry/cmd/telemetry/main.go

   • Add --additional-child-probes flag declaration
   • Parse comma-separated host:port values into []ProbeAddress
   • Populate cfg.InitialChildProbes with parsed values
   • ~30 lines added

2. controlplane/telemetry/internal/telemetry/config.go

   • Add InitialChildProbes []ProbeAddress field (startup probe list)
   • Add ProbeAddress struct definition
   • Update Validate() to check probe address format
   • Add doc comment: "InitialChildProbes is the startup probe list from CLI; runtime updates happen via channel"
   • ~25 lines added

3. controlplane/telemetry/internal/telemetry/collector.go

   • Add probeUpdateCh channel creation
   • Add goroutine launch in Run() for geoprobe coordinator
   • Gate on len(c.cfg.InitialChildProbes) > 0
   • Wire up coordinator with initial probes and update channel
   • ~50 lines added (includes channel setup)

Create New Files

4. controlplane/telemetry/internal/geoprobe/coordinator.go

   • CoordinatorConfig struct (includes initial probes, update channel)
   • Coordinator struct with Run() method
   • Main control loop: listens on probeUpdateCh and measurement ticker
   • Manages pinger and publisher lifecycle
   • Handles dynamic probe addition/removal
   • ~200 lines

5. controlplane/telemetry/internal/geoprobe/pinger.go

   • PingerConfig struct
   • Pinger struct with map of probe addresses to TWAMP senders
   • AddProbe(addr ProbeAddress) method
   • RemoveProbe(addr ProbeAddress) method
   • MeasureAll() method: probes all active targets and returns RTT map
   • ~180 lines (includes dynamic probe management)

6. controlplane/telemetry/internal/geoprobe/publisher.go

   • PublisherConfig struct
   • Publisher struct with lat/lng cache and slot cache
   • refreshLatLng() method (background refresh with 24h TTL)
   • getCurrentSlot() method (with 5min fallback)
   • Publish(rttData map[ProbeAddress]uint64) method
   • UDP connection pool management
   • ~220 lines (includes caching logic)

7. controlplane/telemetry/internal/geoprobe/coordinator_test.go

   • Test initial probe setup
   • Test dynamic probe addition via channel
   • Test dynamic probe removal via channel
   • Test graceful shutdown
   • ~120 lines

8. controlplane/telemetry/internal/geoprobe/pinger_test.go

   • Test TWAMP sender creation for probe targets
   • Test RTT measurement with mock sender
   • Test error handling (timeout, unreachable probe)
   • Test minimum RTT selection over multiple probes
   • Test AddProbe() and RemoveProbe() methods
   • ~100 lines

9. controlplane/telemetry/internal/geoprobe/publisher_test.go

   • Test offset generation with mock device data
   • Test signing with test keypair
   • Test UDP send with mock connection
   • Test lat/lng caching and 24h expiry
   • Test slot caching and 5min fallback
   • Test error handling (missing lat/lng, signing failure, stale slot)
   • ~130 lines

Dependency Files (from #2898)

These files must be available (either merged or cherry-picked):

• controlplane/telemetry/internal/geoprobe/offset.go
• controlplane/telemetry/internal/geoprobe/signer.go
• controlplane/telemetry/internal/geoprobe/transport.go
• controlplane/telemetry/internal/geoprobe/offset_test.go
• controlplane/telemetry/internal/geoprobe/signer_test.go

Risks & Considerations

Technical Risks

1. Dependency on geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898: The geoprobe library must be available. If not merged, we need to cherry-pick or merge the feature/geolocation-start branch.

   • Mitigation: Check if geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 is merged; if not, coordinate with that work or merge the branch.

2. Device Lat/Lng Retrieval: The Device account must have lat/lng fields populated onchain.

   • Mitigation: Verify Device struct in serviceability program has lat/lng fields. If missing, cache retrieval will fail gracefully (non-fatal). Agent logs warning and skips offset generation.

3. UDP Socket Creation: Creating UDP sockets may fail due to network namespace configuration or permissions.

   • Mitigation: Follow existing UDP socket patterns from TWAMP reflector/sender. Test in devnet.

4. Slot Retrieval Failures: RPC calls to get current slot may be slow or fail intermittently.

   • Mitigation: Implement 5-minute slot cache fallback. This allows offset generation to continue during transient RPC failures.

5. Probe Unreachability: Child probes may be unreachable during testing.

   • Mitigation: Add comprehensive error logging. Non-fatal errors should not crash the agent.

6. Dynamic Probe Updates Race Conditions: Concurrent probe addition/removal could cause races.

   • Mitigation: All probe list modifications happen in a single coordinator goroutine. Channel-based updates ensure serialized processing.

Operational Considerations

1. Zero Overhead When Disabled: All runtime code must be gated on len(cfg.InitialChildProbes) > 0 at startup to ensure zero overhead when feature is unused.

2. Backward Compatibility: No changes to existing device telemetry flow. All new code is additive.

3. Resource Usage: Additional goroutines and UDP sockets increase memory/CPU usage.

   • Impact: Minimal - one coordinator goroutine + one goroutine per probe (expected 1-3 probes per device in production).

4. Network Namespace Compatibility: Must work with existing --management-namespace flag for network isolation.

   • Mitigation: Follow existing TWAMP sender patterns which already handle namespaces.

5. Cache Expiry Handling: Lat/lng cache expiry and slot staleness must be handled gracefully.

   • Mitigation: Background refresh for lat/lng (24h TTL). Slot fallback (5min) prevents blocking. Both failures are non-fatal.

6. Future Extensibility: The channel-based architecture enables future work (onchain discovery) to send probe updates without changing the measurement code.

Edge Cases

1. Empty Initial Probe List: If --additional-child-probes is provided but empty, coordinator goroutine never starts (zero overhead).
2. Invalid host:port Format: Validation should fail at startup, not runtime.
3. Probe Resolution Failure: DNS lookup may fail for probe hostnames.
   • Mitigation: Parse and resolve addresses at startup; fail early if invalid.
4. Large RTT Values: RTT may exceed uint64 nanoseconds (unlikely but possible).
   • Mitigation: Use existing RTT handling patterns from device telemetry.
5. Lat/Lng Never Retrieved: If onchain fetch always fails, agent logs errors but continues running. No offsets are generated.
6. Stale Slot Cache: If slot is >5 minutes old, skip offset generation for that cycle (log warning).
7. Duplicate Probe Addresses: If same probe appears in CLI flag multiple times, deduplicate at parse time.
8. Probe Removal While Measurement In Progress: Coordinator handles removal after current measurement cycle completes.

Testing Strategy

Unit Tests

1. Coordinator Tests (coordinator_test.go)

   • Test initial probe setup from config
   • Test dynamic probe addition via channel
   • Test dynamic probe removal via channel
   • Test concurrent updates (rapid add/remove)
   • Test graceful shutdown with active probes

2. Pinger Tests (pinger_test.go)

   • Test TWAMP sender creation for probe targets
   • Test RTT measurement with mock sender
   • Test error handling (timeout, unreachable probe)
   • Test minimum RTT selection over multiple probes
   • Test AddProbe() adds new sender to map
   • Test RemoveProbe() cleans up sender and closes connection

3. Publisher Tests (publisher_test.go)

   • Test offset generation with mock device data
   • Test signing with test keypair
   • Test UDP send with mock connection
   • Test lat/lng caching and 24h TTL expiry
   • Test slot caching and 5min fallback logic
   • Test error handling (missing lat/lng, signing failure, stale slot)
   • Test UDP connection pool management (one per probe)

Integration Tests

The primary integration testing will happen in the local devnet (Phase 1 POC deployment):

1. Manual Devnet Test

   • Deploy telemetry agent with --additional-child-probes flag
   • Point to a test probe server (or simple UDP listener)
   • Verify offset datagrams are received
   • Verify signature is valid
   • Verify lat/lng matches device config
   • Test dynamic probe addition (if onchain discovery is implemented)

2. E2E Test (future work, not in scope for Phase 1)

   • Full DZD → Probe → Target chain test
   • Requires probe agent implementation (geoprobe: geoProbe server (doublezero-geoprobe-agent) #2900)

Build & Lint Validation

1. Run make rust-build (if any Rust changes)
2. Run make go-build to verify compilation
3. Run make go-lint to ensure code quality
4. Run make go-test to run unit tests

Estimated Scope

Medium-Large (~800-900 lines)

• CLI flag parsing: ~30 lines
• Config changes: ~25 lines
• Collector integration: ~50 lines
• Coordinator implementation: ~200 lines
• Pinger implementation: ~180 lines
• Publisher implementation: ~220 lines
• Unit tests: ~350 lines (coordinator: 120, pinger: 100, publisher: 130)
• Total: ~1055 lines

This includes comprehensive error handling, caching logic, dynamic probe management, logging, and testing. The actual implementation may vary based on code reuse from existing telemetry patterns.

Open Questions

1. Is geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 merged? If not, do we need to merge feature/geolocation-start into our branch?

   • Answer: Check PR status and coordinate with team.

2. Do Device accounts have lat/lng fields? Need to verify the serviceability program Device struct.

   • Answer: Review smartcontract/sdk/go/serviceability Device struct.

3. What measurement interval should be used? RFC suggests 10s for DZD→Probe, but should we make it configurable?

   • Decision: Use existing --probe-interval flag (default 10s) for consistency with device telemetry.

4. Should we validate probe reachability at startup? Or allow startup and log errors at runtime?

   • Decision: Allow startup and log errors at runtime (non-fatal). Matches existing peer discovery behavior.

5. Should UDP sockets be pooled or created per-send? Performance vs resource trade-off.

   • Decision: Create one UDP connection per probe, reused for all sends (standard pattern).

6. Should lat/lng cache TTL be configurable? Or hardcode 24 hours?

   • Decision: Hardcode 24h as a constant. Can make configurable in future if needed.

7. Should slot fallback window be configurable? Or hardcode 5 minutes?

   • Decision: Hardcode 5min as a constant. Can make configurable in future if needed.

Implementation Checklist

• Verify geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 geoprobe library is available (merge or cherry-pick if needed)
• Verify Device account has lat/lng fields
• Add --additional-child-probes CLI flag
• Extend Config struct with InitialChildProbes field
• Add probeUpdateCh channel to collector
• Implement geoprobe/coordinator.go (dynamic probe management)
• Implement geoprobe/pinger.go (with AddProbe/RemoveProbe)
• Implement geoprobe/publisher.go (with lat/lng and slot caching)
• Integrate coordinator into collector.go with gated goroutine
• Write unit tests for coordinator (dynamic updates)
• Write unit tests for pinger (including dynamic probe management)
• Write unit tests for publisher (including caching logic)
• Run make go-lint and make go-test
• Test in local devnet with UDP listener
• Verify lat/lng caching works (check logs for 24h refresh)
• Verify slot fallback works (simulate RPC failures)
• Update CLAUDE.md if new patterns are introduced
• Update README.md if CLI flags need documentation

Future Work (Phase 2)

The channel-based architecture enables future work to add onchain probe discovery:

1. Add periodic goroutine that queries serviceability program for assigned child probes
2. Merge onchain probes with CLI-provided --additional-child-probes
3. Send merged list to probeUpdateCh
4. Coordinator handles updates (adds new probes, removes old ones)
5. Measurement code (pinger/publisher) remains unchanged

This ensures clean separation between probe source management (CLI, onchain, etc.) and probe measurement logic. The coordinator abstracts away the complexity of dynamic updates from the measurement components.

Changes from Revision 1

Operator Feedback Addressed:

1. Lat/Lng Caching (24h TTL):

   • Publisher retrieves lat/lng once at startup
   • Caches for 24 hours
   • Background refresh on expiry
   • Non-fatal if retrieval fails (agent continues, logs warning)

2. Slot Fallback (5min window):

   • Attempt to retrieve current slot on each measurement cycle
   • If retrieval fails, use cached slot if <5 minutes old
   • If no valid slot, skip offset generation for that cycle (non-fatal)

3. Dynamic Probe Discovery Support:

   • New coordinator.go component manages dynamic probe list
   • Channel-based updates (probeUpdateCh) support runtime probe addition
   • Pinger and Publisher support adding new probes while running
   • AddProbe() / RemoveProbe() methods in Pinger
   • UDP connection pool in Publisher dynamically grows as probes are added
   • Architecture ready for Phase 2 onchain discovery without code changes

New Components:

• coordinator.go: Manages probe list lifecycle and coordinates pinger/publisher
• coordinator_test.go: Tests dynamic probe updates

Renamed Fields:

• Config.ChildProbes → Config.InitialChildProbes (clarifies this is the startup list)

Updated Estimates:

• Total lines increased from ~605 to ~1055 due to coordinator and caching logic

[ben-dz]

Approved Work Plan (Revision 2)

Estimated Scope: Medium

Work Plan: Issue #2899 - Geoprobe Telemetry Agent Extensions (Revision 2)

Issue: #2899
Branch: ben-dz/doublezero-2899
RFC: RFC16 - Geolocation Verification
Tracking Issue: #2893
Dependencies: #2898 (shared geoprobe library - available on feature/geolocation-start branch)

Summary

Extend the existing telemetry agent to measure RTT to child geoProbes via TWAMP, generate signed LocationOffset structures, and deliver them via UDP. This is Phase 1, item 1 of the geolocation verification system described in RFC16.

The telemetry agent currently measures device-to-device latency using TWAMP. This work adds parallel functionality to measure latency to configured geoProbes (bare metal servers that act as intermediaries for geolocation verification). The agent will:

1. Maintain a unified list of child probes populated from multiple sources (initially CLI flag, later onchain configuration)
2. Accept a CLI flag --additional-child-probes with comma-separated host:port values to populate the child probes list
3. Support dynamic probe list updates at runtime (new probes can be added while the agent is running)
4. Periodically probe these geoProbes using TWAMP (reusing existing twamplight.Sender infrastructure)
5. Generate signed LocationOffset datagrams containing lat/lng (from device config), RTT measurements, and Ed25519 signatures
6. Send these offsets via UDP to each probe for caching and further processing

All runtime code execution (except parsing the CLI argument) will be gated on a non-empty child probes list, ensuring zero overhead when the feature is disabled.

Approach

Design Decisions

1. Reuse Existing Patterns: Follow the established collector pattern with separate pinger and publisher goroutines, similar to the existing device telemetry flow (pinger → buffer → submitter).

2. Dynamic Probe List with Channel Updates: Use a channel-based architecture to support adding new probes at runtime. The main geoprobe goroutine listens for probe updates and dynamically adjusts the probe list without requiring agent restart.

3. Configuration via CLI Flag: Use --additional-child-probes flag for POC testing without requiring onchain probe configuration. This enables rapid testing in Phase 1 before Phase 2 adds onchain discovery.

4. Parallel Goroutine Architecture: Add a new goroutine in collector.go that runs independently alongside existing telemetry goroutines, gated on the presence of child probes in the initial list.

5. Dependency on geoprobe Library: The implementation assumes the geoprobe library from geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 is merged (or we merge it into our branch). The library provides:

   • LocationOffset struct and serialization
   • OffsetSigner for Ed25519 signing
   • SendOffset() for UDP transport

6. Lat/Lng Caching: Retrieve latitude and longitude from the Device account onchain once at startup. Cache the values for 24 hours. If retrieval fails, retry on next measurement cycle. If lat/lng are missing, skip offset generation (non-fatal error).

7. Slot Retrieval Fallback: Retrieve current slot from ledger. If retrieval fails, use the last successfully retrieved slot as long as it was obtained within the last 5 minutes. If no recent slot is available, skip offset generation for that measurement cycle (non-fatal error).

8. Minimal RTT Selection: Use the minimum RTT observed during the measurement interval (matching existing telemetry behavior).

9. UDP Socket Management: Create a single UDP connection per probe, reused for all offset sends to that probe. Dynamically create new sockets when new probes are added.

Implementation Structure

The implementation follows the existing telemetry agent architecture:

collector.go
    ├─> reflector (existing)
    ├─> peer discovery (existing)
    ├─> device pinger (existing)
    ├─> device submitter (existing)
    └─> geoprobe measurement loop (NEW)
            ├─> probe list manager (handles dynamic updates)
            ├─> geoprobe/pinger.go (probe RTT measurement)
            └─> geoprobe/publisher.go (offset generation + UDP send)

Dynamic Probe List Architecture

The agent will maintain a dynamic probe list that can be updated at runtime:

Initial Population (Phase 1):

• CLI flag --additional-child-probes provides the initial probe list
• Feature gating: geoprobe goroutine only starts if initial list is non-empty

Runtime Updates:

• A channel (probeUpdateCh chan []ProbeAddress) receives probe list updates
• The geoprobe goroutine listens on this channel and updates its internal state
• New probes are added to the pinger and publisher (new UDP sockets created)
• Removed probes have their resources cleaned up (UDP sockets closed)

Phase 2 (future work):

• Add onchain probe discovery goroutine
• Periodically query serviceability program for assigned child probes
• Merge onchain probes with CLI-provided probes
• Send merged list via probeUpdateCh
• Geoprobe measurement code remains unchanged

This design ensures clean separation between probe source management and probe measurement logic, and supports dynamic updates without agent restart.

Caching Strategy

Lat/Lng Caching:

• Retrieve device lat/lng from serviceability program once at startup
• Cache for 24 hours (configurable via constant)
• On cache expiry, fetch fresh values in background
• If fetch fails, continue using cached values (log warning)
• If lat/lng are never successfully retrieved, skip offset generation (log error)

Slot Caching:

• Attempt to retrieve current slot from ledger on each measurement cycle
• If retrieval fails, check if last successful slot is <5 minutes old
• If yes, use cached slot; if no, skip offset generation for this cycle (log warning)
• This prevents blocking offset generation on transient RPC failures

Key Components

1. CLI Flag Parsing (cmd/telemetry/main.go)

   • Add --additional-child-probes flag accepting comma-separated host:port values
   • Parse into slice of probe addresses
   • Pass to Config.InitialChildProbes (renamed from ChildProbes to clarify it's the startup list)

2. Config Extension (internal/telemetry/config.go)

   • Add InitialChildProbes []ProbeAddress field (startup probe list)
   • ProbeAddress struct: {Host string, Port uint16}
   • Validation: ensure valid host:port format
   • Add comment documenting that this is the initial list; runtime updates happen via channel

3. Collector Integration (internal/telemetry/collector.go)

   • Add probeUpdateCh channel for dynamic probe list updates
   • Add goroutine launch in Run() method, gated on len(c.cfg.InitialChildProbes) > 0
   • Pattern: similar to existing pinger/submitter goroutines
   • Lifecycle: start with wg.Add(1), clean shutdown via context
   • Pass probeUpdateCh to geoprobe measurement loop

4. GeoProbe Pinger (internal/geoprobe/pinger.go)

   • Maintains map of active probes and their TWAMP senders
   • Periodic TWAMP measurement to configured probes
   • Reuses twamplight.NewSender (same interface as device pinger)
   • Records minimum RTT per interval per probe
   • Supports dynamic probe addition/removal:
     • AddProbe(addr ProbeAddress): creates new TWAMP sender
     • RemoveProbe(addr ProbeAddress): cleans up sender resources
   • Returns measurements to publisher

5. GeoProbe Publisher (internal/geoprobe/publisher.go)

   • Lat/Lng Management:
     • Retrieves device lat/lng from serviceability program at startup
     • Caches for 24 hours
     • Implements refreshLatLng() method for background refresh
     • If lat/lng unavailable, skips offset generation (non-fatal)
   • Slot Management:
     • Attempts to retrieve current slot from ledger on each cycle
     • Falls back to cached slot if <5 minutes old
     • If no valid slot, skips offset generation for that cycle (non-fatal)
   • Offset Generation:
     • Creates LocationOffset struct with:
       • Pubkey: metrics_publisher_pk from keypair
       • MeasurementSlot: current or cached slot
       • Lat/Lng: cached device location
       • MeasuredRttNs: minimum RTT from pinger
       • RttNs: same as MeasuredRttNs (no reference chain for DZD)
       • NumReferences: 0 (DZDs are roots of trust)
       • References: empty slice
   • Signs offset using existing keypair via geoprobe.OffsetSigner
   • Sends via UDP using geoprobe.SendOffset()
   • Maintains UDP connection pool (one per probe)
   • Supports dynamic probe addition (creates new UDP connections)

6. GeoProbe Coordinator (internal/geoprobe/coordinator.go)

   • Main control loop for geoprobe measurement
   • Listens on probeUpdateCh for probe list updates
   • Manages pinger and publisher lifecycle
   • Handles probe addition/removal events:
     • New probes: calls pinger.AddProbe() and creates UDP socket
     • Removed probes: calls pinger.RemoveProbe() and closes UDP socket
   • Coordinates measurement cycles between pinger and publisher

7. Unit Tests

   • coordinator_test.go: test dynamic probe addition/removal, channel updates
   • pinger_test.go: test TWAMP probe creation, RTT measurement, error handling, dynamic probe management
   • publisher_test.go: test offset generation, signing, UDP send, lat/lng caching, slot fallback

Files to Change

Modify Existing Files

1. controlplane/telemetry/cmd/telemetry/main.go

   • Add --additional-child-probes flag declaration
   • Parse comma-separated host:port values into []ProbeAddress
   • Populate cfg.InitialChildProbes with parsed values
   • ~30 lines added

2. controlplane/telemetry/internal/telemetry/config.go

   • Add InitialChildProbes []ProbeAddress field (startup probe list)
   • Add ProbeAddress struct definition
   • Update Validate() to check probe address format
   • Add doc comment: "InitialChildProbes is the startup probe list from CLI; runtime updates happen via channel"
   • ~25 lines added

3. controlplane/telemetry/internal/telemetry/collector.go

   • Add probeUpdateCh channel creation
   • Add goroutine launch in Run() for geoprobe coordinator
   • Gate on len(c.cfg.InitialChildProbes) > 0
   • Wire up coordinator with initial probes and update channel
   • ~50 lines added (includes channel setup)

Create New Files

4. controlplane/telemetry/internal/geoprobe/coordinator.go

   • CoordinatorConfig struct (includes initial probes, update channel)
   • Coordinator struct with Run() method
   • Main control loop: listens on probeUpdateCh and measurement ticker
   • Manages pinger and publisher lifecycle
   • Handles dynamic probe addition/removal
   • ~200 lines

5. controlplane/telemetry/internal/geoprobe/pinger.go

   • PingerConfig struct
   • Pinger struct with map of probe addresses to TWAMP senders
   • AddProbe(addr ProbeAddress) method
   • RemoveProbe(addr ProbeAddress) method
   • MeasureAll() method: probes all active targets and returns RTT map
   • ~180 lines (includes dynamic probe management)

6. controlplane/telemetry/internal/geoprobe/publisher.go

   • PublisherConfig struct
   • Publisher struct with lat/lng cache and slot cache
   • refreshLatLng() method (background refresh with 24h TTL)
   • getCurrentSlot() method (with 5min fallback)
   • Publish(rttData map[ProbeAddress]uint64) method
   • UDP connection pool management
   • ~220 lines (includes caching logic)

7. controlplane/telemetry/internal/geoprobe/coordinator_test.go

   • Test initial probe setup
   • Test dynamic probe addition via channel
   • Test dynamic probe removal via channel
   • Test graceful shutdown
   • ~120 lines

8. controlplane/telemetry/internal/geoprobe/pinger_test.go

   • Test TWAMP sender creation for probe targets
   • Test RTT measurement with mock sender
   • Test error handling (timeout, unreachable probe)
   • Test minimum RTT selection over multiple probes
   • Test AddProbe() and RemoveProbe() methods
   • ~100 lines

9. controlplane/telemetry/internal/geoprobe/publisher_test.go

   • Test offset generation with mock device data
   • Test signing with test keypair
   • Test UDP send with mock connection
   • Test lat/lng caching and 24h expiry
   • Test slot caching and 5min fallback
   • Test error handling (missing lat/lng, signing failure, stale slot)
   • ~130 lines

Dependency Files (from #2898)

These files must be available (either merged or cherry-picked):

• controlplane/telemetry/internal/geoprobe/offset.go
• controlplane/telemetry/internal/geoprobe/signer.go
• controlplane/telemetry/internal/geoprobe/transport.go
• controlplane/telemetry/internal/geoprobe/offset_test.go
• controlplane/telemetry/internal/geoprobe/signer_test.go

Risks & Considerations

Technical Risks

1. Dependency on geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898: The geoprobe library must be available. If not merged, we need to cherry-pick or merge the feature/geolocation-start branch.

   • Mitigation: Check if geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 is merged; if not, coordinate with that work or merge the branch.

2. Device Lat/Lng Retrieval: The Device account must have lat/lng fields populated onchain.

   • Mitigation: Verify Device struct in serviceability program has lat/lng fields. If missing, cache retrieval will fail gracefully (non-fatal). Agent logs warning and skips offset generation.

3. UDP Socket Creation: Creating UDP sockets may fail due to network namespace configuration or permissions.

   • Mitigation: Follow existing UDP socket patterns from TWAMP reflector/sender. Test in devnet.

4. Slot Retrieval Failures: RPC calls to get current slot may be slow or fail intermittently.

   • Mitigation: Implement 5-minute slot cache fallback. This allows offset generation to continue during transient RPC failures.

5. Probe Unreachability: Child probes may be unreachable during testing.

   • Mitigation: Add comprehensive error logging. Non-fatal errors should not crash the agent.

6. Dynamic Probe Updates Race Conditions: Concurrent probe addition/removal could cause races.

   • Mitigation: All probe list modifications happen in a single coordinator goroutine. Channel-based updates ensure serialized processing.

Operational Considerations

1. Zero Overhead When Disabled: All runtime code must be gated on len(cfg.InitialChildProbes) > 0 at startup to ensure zero overhead when feature is unused.

2. Backward Compatibility: No changes to existing device telemetry flow. All new code is additive.

3. Resource Usage: Additional goroutines and UDP sockets increase memory/CPU usage.

   • Impact: Minimal - one coordinator goroutine + one goroutine per probe (expected 1-3 probes per device in production).

4. Network Namespace Compatibility: Must work with existing --management-namespace flag for network isolation.

   • Mitigation: Follow existing TWAMP sender patterns which already handle namespaces.

5. Cache Expiry Handling: Lat/lng cache expiry and slot staleness must be handled gracefully.

   • Mitigation: Background refresh for lat/lng (24h TTL). Slot fallback (5min) prevents blocking. Both failures are non-fatal.

6. Future Extensibility: The channel-based architecture enables future work (onchain discovery) to send probe updates without changing the measurement code.

Edge Cases

1. Empty Initial Probe List: If --additional-child-probes is provided but empty, coordinator goroutine never starts (zero overhead).
2. Invalid host:port Format: Validation should fail at startup, not runtime.
3. Probe Resolution Failure: DNS lookup may fail for probe hostnames.
   • Mitigation: Parse and resolve addresses at startup; fail early if invalid.
4. Large RTT Values: RTT may exceed uint64 nanoseconds (unlikely but possible).
   • Mitigation: Use existing RTT handling patterns from device telemetry.
5. Lat/Lng Never Retrieved: If onchain fetch always fails, agent logs errors but continues running. No offsets are generated.
6. Stale Slot Cache: If slot is >5 minutes old, skip offset generation for that cycle (log warning).
7. Duplicate Probe Addresses: If same probe appears in CLI flag multiple times, deduplicate at parse time.
8. Probe Removal While Measurement In Progress: Coordinator handles removal after current measurement cycle completes.

Testing Strategy

Unit Tests

1. Coordinator Tests (coordinator_test.go)

   • Test initial probe setup from config
   • Test dynamic probe addition via channel
   • Test dynamic probe removal via channel
   • Test concurrent updates (rapid add/remove)
   • Test graceful shutdown with active probes

2. Pinger Tests (pinger_test.go)

   • Test TWAMP sender creation for probe targets
   • Test RTT measurement with mock sender
   • Test error handling (timeout, unreachable probe)
   • Test minimum RTT selection over multiple probes
   • Test AddProbe() adds new sender to map
   • Test RemoveProbe() cleans up sender and closes connection

3. Publisher Tests (publisher_test.go)

   • Test offset generation with mock device data
   • Test signing with test keypair
   • Test UDP send with mock connection
   • Test lat/lng caching and 24h TTL expiry
   • Test slot caching and 5min fallback logic
   • Test error handling (missing lat/lng, signing failure, stale slot)
   • Test UDP connection pool management (one per probe)

Integration Tests

The primary integration testing will happen in the local devnet (Phase 1 POC deployment):

1. Manual Devnet Test

   • Deploy telemetry agent with --additional-child-probes flag
   • Point to a test probe server (or simple UDP listener)
   • Verify offset datagrams are received
   • Verify signature is valid
   • Verify lat/lng matches device config
   • Test dynamic probe addition (if onchain discovery is implemented)

2. E2E Test (future work, not in scope for Phase 1)

   • Full DZD → Probe → Target chain test
   • Requires probe agent implementation (geoprobe: geoProbe server (doublezero-geoprobe-agent) #2900)

Build & Lint Validation

1. Run make rust-build (if any Rust changes)
2. Run make go-build to verify compilation
3. Run make go-lint to ensure code quality
4. Run make go-test to run unit tests

Estimated Scope

Medium-Large (~800-900 lines)

• CLI flag parsing: ~30 lines
• Config changes: ~25 lines
• Collector integration: ~50 lines
• Coordinator implementation: ~200 lines
• Pinger implementation: ~180 lines
• Publisher implementation: ~220 lines
• Unit tests: ~350 lines (coordinator: 120, pinger: 100, publisher: 130)
• Total: ~1055 lines

This includes comprehensive error handling, caching logic, dynamic probe management, logging, and testing. The actual implementation may vary based on code reuse from existing telemetry patterns.

Open Questions

1. Is geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 merged? If not, do we need to merge feature/geolocation-start into our branch?

   • Answer: Check PR status and coordinate with team.

2. Do Device accounts have lat/lng fields? Need to verify the serviceability program Device struct.

   • Answer: Review smartcontract/sdk/go/serviceability Device struct.

3. What measurement interval should be used? RFC suggests 10s for DZD→Probe, but should we make it configurable?

   • Decision: Use existing --probe-interval flag (default 10s) for consistency with device telemetry.

4. Should we validate probe reachability at startup? Or allow startup and log errors at runtime?

   • Decision: Allow startup and log errors at runtime (non-fatal). Matches existing peer discovery behavior.

5. Should UDP sockets be pooled or created per-send? Performance vs resource trade-off.

   • Decision: Create one UDP connection per probe, reused for all sends (standard pattern).

6. Should lat/lng cache TTL be configurable? Or hardcode 24 hours?

   • Decision: Hardcode 24h as a constant. Can make configurable in future if needed.

7. Should slot fallback window be configurable? Or hardcode 5 minutes?

   • Decision: Hardcode 5min as a constant. Can make configurable in future if needed.

Implementation Checklist

• Verify geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 geoprobe library is available (merge or cherry-pick if needed)
• Verify Device account has lat/lng fields
• Add --additional-child-probes CLI flag
• Extend Config struct with InitialChildProbes field
• Add probeUpdateCh channel to collector
• Implement geoprobe/coordinator.go (dynamic probe management)
• Implement geoprobe/pinger.go (with AddProbe/RemoveProbe)
• Implement geoprobe/publisher.go (with lat/lng and slot caching)
• Integrate coordinator into collector.go with gated goroutine
• Write unit tests for coordinator (dynamic updates)
• Write unit tests for pinger (including dynamic probe management)
• Write unit tests for publisher (including caching logic)
• Run make go-lint and make go-test
• Test in local devnet with UDP listener
• Verify lat/lng caching works (check logs for 24h refresh)
• Verify slot fallback works (simulate RPC failures)
• Update CLAUDE.md if new patterns are introduced
• Update README.md if CLI flags need documentation

Future Work (Phase 2)

The channel-based architecture enables future work to add onchain probe discovery:

1. Add periodic goroutine that queries serviceability program for assigned child probes
2. Merge onchain probes with CLI-provided --additional-child-probes
3. Send merged list to probeUpdateCh
4. Coordinator handles updates (adds new probes, removes old ones)
5. Measurement code (pinger/publisher) remains unchanged

This ensures clean separation between probe source management (CLI, onchain, etc.) and probe measurement logic. The coordinator abstracts away the complexity of dynamic updates from the measurement components.

Changes from Revision 1

Operator Feedback Addressed:

1. Lat/Lng Caching (24h TTL):

   • Publisher retrieves lat/lng once at startup
   • Caches for 24 hours
   • Background refresh on expiry
   • Non-fatal if retrieval fails (agent continues, logs warning)

2. Slot Fallback (5min window):

   • Attempt to retrieve current slot on each measurement cycle
   • If retrieval fails, use cached slot if <5 minutes old
   • If no valid slot, skip offset generation for that cycle (non-fatal)

3. Dynamic Probe Discovery Support:

   • New coordinator.go component manages dynamic probe list
   • Channel-based updates (probeUpdateCh) support runtime probe addition
   • Pinger and Publisher support adding new probes while running
   • AddProbe() / RemoveProbe() methods in Pinger
   • UDP connection pool in Publisher dynamically grows as probes are added
   • Architecture ready for Phase 2 onchain discovery without code changes

New Components:

• coordinator.go: Manages probe list lifecycle and coordinates pinger/publisher
• coordinator_test.go: Tests dynamic probe updates

Renamed Fields:

• Config.ChildProbes → Config.InitialChildProbes (clarifies this is the startup list)

Updated Estimates:

• Total lines increased from ~605 to ~1055 due to coordinator and caching logic

[ben-dz]

Approved Work Plan (Revision 3)

Work Plan: Issue #2899 - Geoprobe Telemetry Agent Extensions (Revision 3)

Issue: #2899
Branch: ben-dz/doublezero-2899
RFC: RFC16 - Geolocation Verification
Tracking Issue: #2893
Dependencies: #2898 (shared geoprobe library - merged into current branch)

Summary

Extend the existing telemetry agent to measure RTT to child geoProbes via TWAMP, generate signed LocationOffset structures, and deliver them via UDP. This is Phase 1, item 1 of the geolocation verification system described in RFC16.

The telemetry agent currently measures device-to-device latency using TWAMP. This work adds parallel functionality to measure latency to configured geoProbes (bare metal servers that act as intermediaries for geolocation verification). The agent will:

1. Maintain a unified list of child probes populated from multiple sources (initially CLI flag, later onchain configuration)
2. Accept a CLI flag --additional-child-probes with comma-separated host:port values to populate the child probes list
3. Support dynamic probe list updates at runtime (new probes can be added while the agent is running)
4. Periodically probe these geoProbes using TWAMP (reusing existing twamplight.Sender infrastructure)
5. Generate signed LocationOffset datagrams containing lat/lng (from device config), RTT measurements, and Ed25519 signatures
6. Send these offsets via UDP to each probe for caching and further processing

All runtime code execution (except parsing the CLI argument) will be gated on a non-empty child probes list, ensuring zero overhead when the feature is disabled.

Current Status: Implementation is complete. Security and architecture reviews identified critical and high-priority issues that must be addressed before creating a PR.

Review Findings Summary

Two comprehensive reviews (security and architecture) identified issues requiring fixes before PR creation:

Critical Issues (5)

1. Unbounded goroutine spawning in probe measurements (DoS risk)
2. Unbounded memory allocation in reference chains (DoS risk)
3. Duplicate Publisher type definition (code organization)
4. Missing config validation for geoprobe fields (runtime panics)
5. UDP connection namespace awareness issue (networking)

High Priority Issues (4)

6. ProbeAddress lacks validation method
7. Pinger.MeasureAll error handling insufficient
8. Publisher AddProbe/RemoveProbe inconsistent error semantics
9. Coordinator tests use uninitialized mocks

Medium Priority Issues (3)

10. Insufficient probe address validation (security)
11. DNS resolution blocking (DoS risk)
12. Unbounded goroutines in Publisher.Publish

Low Priority Issues (8 - deferred with documentation)

• Port range validation, UDP encryption, CLI input size limits, signature verification scope, TamperOffset location, empty probe list handling, DNS retry mechanism, secrets management

Operator Decision: Fix all critical, high, and medium issues before creating PR. Document low-priority issues as known limitations.

Approach

Design Decisions

1. Reuse Existing Patterns: Follow the established collector pattern with separate pinger and publisher goroutines, similar to the existing device telemetry flow (pinger → buffer → submitter).

2. Dynamic Probe List with Channel Updates: Use a channel-based architecture to support adding new probes at runtime. The main geoprobe goroutine listens for probe updates and dynamically adjusts the probe list without requiring agent restart.

3. Configuration via CLI Flag: Use --additional-child-probes flag for POC testing without requiring onchain probe configuration. This enables rapid testing in Phase 1 before Phase 2 adds onchain discovery.

4. Parallel Goroutine Architecture: Add a new goroutine in collector.go that runs independently alongside existing telemetry goroutines, gated on the presence of child probes in the initial list.

5. Dependency on geoprobe Library: The implementation uses the geoprobe library from geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 (merged). The library provides:

   • LocationOffset struct and serialization
   • OffsetSigner for Ed25519 signing
   • SendOffset() for UDP transport

6. Lat/Lng Caching: Retrieve latitude and longitude from the Device account onchain once at startup. Cache the values for 24 hours. If retrieval fails, retry on next measurement cycle. If lat/lng are missing, skip offset generation (non-fatal error).

7. Slot Retrieval Fallback: Retrieve current slot from ledger. If retrieval fails, use the last successfully retrieved slot as long as it was obtained within the last 5 minutes. If no recent slot is available, skip offset generation for that measurement cycle (non-fatal error).

8. Minimal RTT Selection: Use the minimum RTT observed during the measurement interval (matching existing telemetry behavior).

9. UDP Socket Management: Create a single UDP connection per probe, reused for all offset sends to that probe. Dynamically create new sockets when new probes are added.

10. Resource Limits (NEW): Implement concurrency limits (100 concurrent probes), reference chain depth limits (5 levels per RFC16), and reference count limits (20 total references) to prevent DoS attacks.

Implementation Structure

The implementation follows the existing telemetry agent architecture:

collector.go
    ├─> reflector (existing)
    ├─> peer discovery (existing)
    ├─> device pinger (existing)
    ├─> device submitter (existing)
    └─> geoprobe measurement loop (NEW)
            ├─> probe list manager (handles dynamic updates)
            ├─> geoprobe/pinger.go (probe RTT measurement with concurrency limits)
            └─> geoprobe/publisher.go (offset generation + UDP send with validation)

Phase 1: Critical Fixes (Must Complete Before PR)

Fix #1: Unbounded Goroutine Spawning in Pinger (HIGH - DoS)

Issue: No limit on concurrent probe measurements in MeasureAll() and MeasureAllWithRetries(). Large probe lists could spawn thousands of goroutines.

Location: pinger.go:118-138 (MeasureAll), pinger.go:162-200 (MeasureAllWithRetries)

Fix Strategy:

• Add constant MaxConcurrentProbes = 100
• Implement worker pool using buffered channel semaphore
• Queue measurements when limit reached
• Update tests to verify limit enforcement

Files Modified:

• controlplane/telemetry/internal/geoprobe/pinger.go (~50 lines added/modified)
• controlplane/telemetry/internal/geoprobe/pinger_test.go (~30 lines added)

Estimated Effort: ~80 lines total

Fix #2: Unbounded Reference Chain Allocation (HIGH - DoS)

Issue: No depth or count limits for reference chains in Unmarshal(). Malicious messages could cause unbounded memory allocation.

Location: offset.go:110-115 (Unmarshal)

Fix Strategy:

• Add constants: MaxReferenceDepth = 5 (per RFC16), MaxTotalReferences = 20
• Add validation in Unmarshal() to reject messages exceeding limits
• Add validation helper to count total references and check depth
• Return clear error messages when limits exceeded
• Update tests to verify limit enforcement

Files Modified:

• controlplane/telemetry/internal/geoprobe/offset.go (~30 lines added)
• controlplane/telemetry/internal/geoprobe/offset_test.go (~40 lines added)

Estimated Effort: ~70 lines total

Fix #3: Duplicate Publisher Definition (CRITICAL - Code Quality)

Issue: Publisher struct and PublisherConfig defined twice - once in coordinator.go:226-248 and again in publisher.go:31-48. Serious code organization issue.

Location: coordinator.go:226-248 and publisher.go:31-48

Fix Strategy:

• Remove duplicate Publisher/PublisherConfig from coordinator.go (lines 226-248)
• Remove old Publisher methods from coordinator.go (lines 448+)
• Update coordinator code to import and use Publisher from publisher.go
• Verify all tests pass after removal
• Verify coordinator correctly uses publisher.go implementation

Files Modified:

• controlplane/telemetry/internal/geoprobe/coordinator.go (~200 lines deleted, ~20 lines modified)
• controlplane/telemetry/internal/geoprobe/coordinator_test.go (~10 lines modified if needed)

Estimated Effort: Net -180 lines (delete duplicates, minimal refactoring)

Fix #4: Config Validation Inconsistency (CRITICAL - API Design)

Issue: telemetry.Config.Validate() doesn't validate geoprobe fields (InitialChildProbes, ServiceabilityProgramClient, RPCClient, Keypair). Runtime panics possible when geoprobe enabled with nil dependencies.

Location: config.go:65-102

Fix Strategy:

• Add validation block in Validate() method:

  if len(c.InitialChildProbes) > 0 {
      if c.ServiceabilityProgramClient == nil {
          return errors.New("serviceability client is required when geoprobe is enabled")
      }
      if c.RPCClient == nil {
          return errors.New("rpc client is required when geoprobe is enabled")
      }
      if c.Keypair == nil {
          return errors.New("keypair is required when geoprobe is enabled")
      }
  }

• Add unit tests for validation

Files Modified:

• controlplane/telemetry/internal/telemetry/config.go (~20 lines added)
• controlplane/telemetry/internal/telemetry/config_test.go (~30 lines added)

Estimated Effort: ~50 lines total

Fix #5: Namespace Awareness (Resolved by Fix #3)

Issue: Old Publisher.AddProbe() in coordinator.go doesn't use ManagementNamespace.

Resolution: Automatically resolved by removing duplicate Publisher from coordinator.go (Fix #3). The publisher.go implementation correctly handles namespaces.

No additional work required.

Phase 2: High Priority Fixes (Must Complete Before PR)

Fix #6: ProbeAddress Lacks Validation (HIGH - API Design)

Issue: No Validate() method to check Host/Port after construction. Invalid ProbeAddress instances could cause errors downstream.

Location: address.go:10-18

Fix Strategy:

• Add Validate() method to ProbeAddress:

  func (p ProbeAddress) Validate() error {
      if p.Host == "" {
          return errors.New("host cannot be empty")
      }
      if p.Port == 0 {
          return errors.New("port cannot be zero")
      }
      return nil
  }

• Call Validate() in ParseProbeAddresses after construction
• Add tests

Files Modified:

• controlplane/telemetry/internal/geoprobe/address.go (~10 lines added)
• controlplane/telemetry/internal/geoprobe/address_test.go (~20 lines added)

Estimated Effort: ~30 lines total

Fix #7: Pinger.MeasureAll Error Handling (HIGH - Error Handling)

Issue: MeasureAll() always returns (map, nil) even when probes fail. Can't distinguish "no probes" vs "all failed" vs "some succeeded".

Location: pinger.go:102-142

Fix Strategy:

• Return error when all probes fail
• Add structured logging with success/failure counts
• Consider returning partial results with error
• Update tests

Files Modified:

• controlplane/telemetry/internal/geoprobe/pinger.go (~20 lines modified)
• controlplane/telemetry/internal/geoprobe/pinger_test.go (~20 lines added)

Estimated Effort: ~40 lines total

Fix #8: Inconsistent AddProbe/RemoveProbe Semantics (HIGH - Code Quality)

Issue: AddProbe() returns error if probe exists, RemoveProbe() returns error if probe doesn't exist. Asymmetric and confusing.

Location: publisher.go:80-138

Fix Strategy:

• Make both idempotent:
  • AddProbe(): return nil if probe already exists (log debug message)
  • RemoveProbe(): return nil if probe doesn't exist (log debug message)
• Update tests to verify idempotency
• Document idempotent behavior in comments

Files Modified:

• controlplane/telemetry/internal/geoprobe/publisher.go (~10 lines modified)
• controlplane/telemetry/internal/geoprobe/publisher_test.go (~15 lines added)

Estimated Effort: ~25 lines total

Fix #9: Uninitialized Mocks in Tests (HIGH - Testability)

Issue: Coordinator tests use zero-value mock clients (&serviceability.Client{}, &solanarpc.Client{}). Tests calling Publisher methods will panic.

Location: coordinator_test.go:36-44

Fix Strategy:

• Implement proper mock interfaces or use testify/mock
• Create mock implementations that return sensible test data
• Update all coordinator tests to use proper mocks
• Verify tests exercise real code paths

Files Modified:

• controlplane/telemetry/internal/geoprobe/coordinator_test.go (~50 lines added/modified)
• Possibly add controlplane/telemetry/internal/geoprobe/mocks_test.go (~50 lines)

Estimated Effort: ~100 lines total

Phase 3: Medium Priority Security Fixes (Must Complete Before PR)

Fix #10: Insufficient Address Validation (MEDIUM - Security)

Issue: No validation to reject localhost, loopback, link-local, or private IPs. Could enable attacks on internal infrastructure.

Location: address.go:20-62

Fix Strategy:

• Add validation functions:
  • isPrivateIP(): check 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
  • isLoopback(): check 127.0.0.0/8, ::1
  • isLinkLocal(): check 169.254.0.0/16
  • isReserved(): check 0.0.0.0/8, 240.0.0.0/4
• Add validation in ParseProbeAddresses to reject these by default
• Log warnings when rejecting addresses
• Add comprehensive tests

Files Modified:

• controlplane/telemetry/internal/geoprobe/address.go (~80 lines added)
• controlplane/telemetry/internal/geoprobe/address_test.go (~60 lines added)

Estimated Effort: ~140 lines total

Fix #11: DNS Resolution Blocking (MEDIUM - DoS)

Issue: Synchronous net.ResolveUDPAddr() can block for extended periods if DNS servers slow/unresponsive.

Location: pinger.go:52, publisher.go:106

Fix Strategy:

• Create custom resolver with timeout:

  resolver := &net.Resolver{}
  ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
  defer cancel()
  addrs, err := resolver.LookupHost(ctx, host)

• Replace net.ResolveUDPAddr() calls with timeout-based resolution
• Handle timeout errors gracefully
• Add tests with mock resolver

Files Modified:

• controlplane/telemetry/internal/geoprobe/pinger.go (~30 lines modified)
• controlplane/telemetry/internal/geoprobe/publisher.go (~30 lines modified)
• controlplane/telemetry/internal/geoprobe/pinger_test.go (~20 lines added)
• controlplane/telemetry/internal/geoprobe/publisher_test.go (~20 lines added)

Estimated Effort: ~100 lines total

Fix #12: Unbounded Goroutines in Publisher (MEDIUM - DoS)

Issue: Same as pinger - no upper bound on goroutines spawned during Publish() cycle.

Location: publisher.go:159-195

Fix Strategy:

• Reuse worker pool pattern from Fix controlplane: open source controller/agent #1
• Share semaphore or create separate limit if needed
• Queue publish operations when limit reached
• Update tests

Files Modified:

• controlplane/telemetry/internal/geoprobe/publisher.go (~20 lines modified)
• controlplane/telemetry/internal/geoprobe/publisher_test.go (~20 lines added)

Estimated Effort: ~40 lines total (if reusing Fix #1 pattern)

Phase 4: Documentation and Final Validation

Documentation Updates

1. Create docs/security-considerations-geoprobe.md

   • Document known limitations (UDP unencrypted, keypair in memory, etc.)
   • Document security model (who signs, who verifies)
   • Document expected deployment environment (trusted networks)
   • List deferred low-priority issues as known limitations
   • ~100 lines

2. Update CHANGELOG.md

   • Add entry for geoprobe feature
   • Use /changelog skill

Final Validation Checklist

• Run make go-build - verify compilation
• Run make go-lint - ensure code quality
• Run make go-test - all tests pass
• Test in local devnet with UDP listener
• Verify concurrency limits work (test with large probe list)
• Verify reference chain limits work (test deep chains)
• Verify address validation rejects private IPs
• Verify DNS timeout handling works
• Verify lat/lng caching works (check logs for 24h refresh)
• Verify slot fallback works (simulate RPC failures)

Files to Change - Complete List

Modified for Critical Fixes (Phase 1)

1. controlplane/telemetry/internal/geoprobe/pinger.go - Add concurrency limits (~50 lines)
2. controlplane/telemetry/internal/geoprobe/pinger_test.go - Test concurrency limits (~30 lines)
3. controlplane/telemetry/internal/geoprobe/offset.go - Add reference limits (~30 lines)
4. controlplane/telemetry/internal/geoprobe/offset_test.go - Test reference limits (~40 lines)
5. controlplane/telemetry/internal/geoprobe/coordinator.go - Remove duplicates (net -180 lines)
6. controlplane/telemetry/internal/telemetry/config.go - Add validation (~20 lines)
7. controlplane/telemetry/internal/telemetry/config_test.go - Test validation (~30 lines)

Phase 1 Total: ~20 lines net (280 added - 200 deleted - 60 test lines)

Modified for High Priority Fixes (Phase 2)

8. controlplane/telemetry/internal/geoprobe/address.go - Add Validate() (~10 lines)
9. controlplane/telemetry/internal/geoprobe/address_test.go - Test Validate() (~20 lines)
10. controlplane/telemetry/internal/geoprobe/pinger.go - Fix error handling (~20 lines)
11. controlplane/telemetry/internal/geoprobe/pinger_test.go - Test error handling (~20 lines)
12. controlplane/telemetry/internal/geoprobe/publisher.go - Fix idempotency (~10 lines)
13. controlplane/telemetry/internal/geoprobe/publisher_test.go - Test idempotency (~15 lines)
14. controlplane/telemetry/internal/geoprobe/coordinator_test.go - Proper mocks (~50 lines)
15. controlplane/telemetry/internal/geoprobe/mocks_test.go - Mock implementations (~50 lines, NEW)

Phase 2 Total: ~195 lines

Modified for Medium Priority Fixes (Phase 3)

16. controlplane/telemetry/internal/geoprobe/address.go - Address validation (~80 lines)
17. controlplane/telemetry/internal/geoprobe/address_test.go - Test address validation (~60 lines)
18. controlplane/telemetry/internal/geoprobe/pinger.go - DNS timeout (~30 lines)
19. controlplane/telemetry/internal/geoprobe/pinger_test.go - Test DNS timeout (~20 lines)
20. controlplane/telemetry/internal/geoprobe/publisher.go - DNS timeout + concurrency (~50 lines)
21. controlplane/telemetry/internal/geoprobe/publisher_test.go - Test both (~40 lines)

Phase 3 Total: ~280 lines

Documentation (Phase 4)

22. docs/security-considerations-geoprobe.md - Security documentation (~100 lines, NEW)
23. CHANGELOG.md - Feature entry (~5 lines)

Phase 4 Total: ~105 lines

Grand Total

• Original Implementation: ~1055 lines (complete)
• Review Fixes: ~600 lines (495 code + 105 docs)
• Net After Deletions: ~1475 lines (1055 + 600 - 180 duplicate deletion)

Implementation Checklist (Revision 3)

Pre-Existing Work (Complete)

• Verify geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 geoprobe library is available
• Verify Device account has lat/lng fields
• Add --additional-child-probes CLI flag
• Extend Config struct with InitialChildProbes field
• Add probeUpdateCh channel to collector
• Implement geoprobe/coordinator.go (dynamic probe management)
• Implement geoprobe/pinger.go (with AddProbe/RemoveProbe)
• Implement geoprobe/publisher.go (with lat/lng and slot caching)
• Integrate coordinator into collector.go with gated goroutine
• Write unit tests for coordinator (dynamic updates)
• Write unit tests for pinger (including dynamic probe management)
• Write unit tests for publisher (including caching logic)
• Initial make go-lint and make go-test validation
• Security review completed
• Architecture review completed

Phase 1: Critical Fixes

• Fix controlplane: open source controller/agent #1: Implement concurrency limits for probe measurements (pinger.go + tests)
• Fix activator: open source activator #2: Add reference chain depth/count limits (offset.go + tests)
• Fix client: open source doublezero client #3: Remove duplicate Publisher definitions from coordinator.go
• Fix #4: Add geoprobe field validation to config.go + tests
• Validate Phase 1: Run make go-lint and make go-test

Phase 2: High Priority Fixes

• Fix build(deps): bump golang.org/x/net from 0.31.0 to 0.33.0 in the go_modules group across 1 directory #6: Add ProbeAddress.Validate() method + tests
• Fix build(deps): bump the cargo group across 3 directories with 1 update #7: Enhance Pinger.MeasureAll error handling + tests
• Fix dependabot: create dependabot configuration #8: Make Publisher AddProbe/RemoveProbe idempotent + tests
• Fix actions: add missing go.{mod,sum} path #9: Implement proper mocks in coordinator_test.go
• Validate Phase 2: Run make go-lint and make go-test

Phase 3: Medium Priority Security Fixes

• Fix dependabot: use directories keyword #10: Add address validation (reject private/localhost IPs) + tests
• Fix client: pass config file to goreleaser in action #11: Add timeout-based DNS resolution + tests
• Fix #12: Add concurrency limiting to Publisher.Publish + tests
• Validate Phase 3: Run make go-lint and make go-test

Phase 4: Documentation and Final Validation

• Create security considerations documentation
• Update CHANGELOG.md using /changelog skill
• Run full test suite: make go-test
• Run linter: make go-lint
• Test in local devnet with UDP listener
• Verify concurrency limits (large probe list)
• Verify reference chain limits (deep chains)
• Verify address validation (try invalid addresses)
• Verify DNS timeout handling
• Verify lat/lng caching (check logs for 24h refresh)
• Verify slot fallback (simulate RPC failures)
• Generate PR description using /pr-text skill
• Create PR using gh pr create

Deferred Issues (Low Priority - Document Only)

The following issues will be documented in security-considerations-geoprobe.md as known limitations for Phase 1:

1. Port Range Validation: Currently rejects port 0, but doesn't validate against expected range (e.g., 10000-65535). Document expected port ranges.

2. UDP Encryption: LocationOffset messages sent over UDP without encryption. While signed (preventing tampering), contents (lat/lng, RTT, signatures, pubkeys) visible to network observers. Document requirement for trusted networks or future DTLS/WireGuard implementation.

3. CLI Flag Input Size: No size limit on --additional-child-probes flag. Operator-controlled, so low risk. Could validate reasonable length (e.g., 10KB) in future.

4. Signature Verification Scope: Verification functions implemented but not called in DZD code (DZD only sends, never receives). Document security model: DZD signs and sends, child probes verify.

5. TamperOffset Location: Function marked test-only but in production code. Move to _test.go or add //go:build test constraint in future.

6. Empty Probe List Handling: Empty lists silently accepted. Add warning logs in future to help with operational debugging.

7. DNS Retry Mechanism: No retry for failed DNS lookups. Transient DNS issues permanently exclude probes. Document limitation; add retry with exponential backoff in Phase 2.

8. Secrets Management: Keypair stored in memory for process lifetime. Document limitation and future KMS integration plans.

Testing Strategy

Unit Tests (Updated for Fixes)

All existing tests remain valid. New tests required:

1. Pinger Tests (additions):

   • Test concurrency limits (Fix controlplane: open source controller/agent #1)
   • Test enhanced error handling (Fix build(deps): bump the cargo group across 3 directories with 1 update #7)
   • Test DNS timeout handling (Fix client: pass config file to goreleaser in action #11)

2. Publisher Tests (additions):

   • Test idempotent AddProbe/RemoveProbe (Fix dependabot: create dependabot configuration #8)
   • Test concurrency limits (Fix #12)
   • Test DNS timeout handling (Fix client: pass config file to goreleaser in action #11)

3. Offset Tests (additions):

   • Test reference chain depth limits (Fix activator: open source activator #2)
   • Test reference count limits (Fix activator: open source activator #2)

4. Address Tests (additions):

   • Test ProbeAddress.Validate() (Fix build(deps): bump golang.org/x/net from 0.31.0 to 0.33.0 in the go_modules group across 1 directory #6)
   • Test address validation - reject private IPs (Fix dependabot: use directories keyword #10)
   • Test address validation - reject localhost/loopback (Fix dependabot: use directories keyword #10)
   • Test address validation - reject link-local (Fix dependabot: use directories keyword #10)

5. Config Tests (additions):

   • Test geoprobe field validation (Fix #4)

6. Coordinator Tests (updates):

   • Update to use proper mocks (Fix actions: add missing go.{mod,sum} path #9)

Integration Tests

Manual devnet testing with additional validation:

1. Existing Tests:

   • Deploy with --additional-child-probes flag
   • Verify offset datagrams received
   • Verify signature valid
   • Verify lat/lng matches device config

2. New Tests for Fixes:

   • Test with large probe lists (100+) - verify concurrency limits work
   • Test with invalid addresses (private IPs, localhost) - verify rejection
   • Test with slow DNS servers - verify timeout handling
   • Test with deep reference chains - verify rejection (if receiving offsets)

Build & Lint Validation

Run after each phase:

1. make go-build - verify compilation
2. make go-lint - ensure code quality
3. make go-test - run unit tests
4. Fix any issues before proceeding to next phase

Open Questions (New for Revision 3)

Resolved Questions

1. Is geoprobe: shared library — LocationOffset types, signing, and UDP transport #2898 merged? Resolved - library is merged
2. Do Device accounts have lat/lng fields? Resolved - verified
3. What measurement interval? Resolved - use existing --probe-interval flag (10s)
4. Validate probe reachability at startup? Resolved - allow startup, log errors at runtime
5. UDP socket pooling? Resolved - one connection per probe, reused
6. Lat/lng cache TTL configurable? Resolved - hardcode 24h
7. Slot fallback window configurable? Resolved - hardcode 5min

New Questions for Fixes

8. Concurrency limit value?

   • Question: Security review recommends 100-1000. What's appropriate for Phase 1?
   • Proposed Answer: Start with 100 as constant. This handles expected Phase 1 scale (1-10 probes per device) with large safety margin. Make configurable via CLI flag in Phase 2 if needed.

9. Max reference chain depth?

   • Question: RFC16 mentions 5 levels. Should this be configurable?
   • Proposed Answer: Hardcode 5 as constant per RFC16. Make configurable via constant or config in Phase 2 if testing reveals need.

10. Max total references count?

    • Question: Security review suggests 10-20. What's appropriate?
    • Proposed Answer: Start with 20 as constant. This allows 5 levels with up to 4 references per level (20 total). Adjust in Phase 2 based on real-world usage.

11. Address validation - opt-in or opt-out?

    • Question: Reject private IPs by default or require explicit allowlist?
    • Proposed Answer: Reject private/localhost/loopback/link-local by default (secure by default). Log clear warnings. Add --allow-private-probes flag in Phase 2 for local testing/devnet if needed. For Phase 1 devnet testing, use public IPs or document workaround.

12. DNS timeout duration?

    • Question: What timeout value for DNS resolution?
    • Proposed Answer: 5 seconds (matches security review suggestion). Balances between allowing slow DNS and preventing indefinite blocking.

13. Worker pool implementation?

    • Question: Use buffered channel semaphore or external worker pool library?
    • Proposed Answer: Use buffered channel semaphore (simple, standard Go pattern). External libraries add dependencies and complexity.

Future Work (Phase 2)

Onchain Probe Discovery

1. Add periodic goroutine querying serviceability program for assigned child probes
2. Merge onchain probes with CLI-provided --additional-child-probes
3. Send merged list via probeUpdateCh
4. Coordinator handles updates (adds new probes, removes old ones)
5. Measurement code (pinger/publisher) remains unchanged

Configuration Enhancements

• Make concurrency limits configurable via CLI flag (--max-concurrent-probes)
• Make reference chain limits configurable (--max-reference-depth, --max-total-references)
• Add --allow-private-probes flag for local testing/devnet
• Add --dns-timeout flag for DNS resolution timeout

Observability Improvements

• Implement metrics for probe success/failure rates
• Add operator dashboard for probe health monitoring
• Add structured logging with probe performance stats
• Add alerting for probe failures

Security Enhancements

• Consider DTLS or WireGuard encryption for UDP traffic
• Implement KMS integration for keypair management
• Add mTLS for probe authentication
• Implement rate limiting on receiving side (if DZD accepts incoming offsets)

Reliability Improvements

• Add retry mechanism for failed DNS resolution (exponential backoff)
• Implement probe health checks and automatic removal
• Add circuit breaker pattern for consistently failing probes
• Implement graceful degradation when probe count exceeds limits

Code Quality

• Extract CachedValue[T] generic type for Publisher caching logic
• Standardize "probe" terminology in logs and comments
• Move TamperOffset to _test.go or add build constraints
• Consider IPv6 support in ProbeAddress.String()
• Add integration test for complete measurement cycle

Changes from Revision 2

Operator Feedback Addressed:

The operator chose to fix critical and high findings before creating the PR (option 1). Revision 3 adds comprehensive implementation plan for addressing all review findings.

Major Changes:

1. New "Review Findings Summary" Section: Clear prioritization of all 17 identified issues (5 critical, 4 high, 3 medium, 5 low deferred)

2. Four-Phase Implementation Plan:

   • Phase 1: Critical issues (DoS protection, code duplication, config validation)
   • Phase 2: High priority issues (API design, error handling, testability)
   • Phase 3: Medium priority security issues (input validation, DNS timeouts)
   • Phase 4: Documentation and final validation

3. Detailed Fix Specifications: Each fix includes:

   • Issue description
   • Exact file locations
   • Concrete fix strategy with code examples
   • Estimated effort (lines of code)
   • Test requirements

4. Updated Scope Estimates:

   • Original work: ~1055 lines (complete)
   • Review fixes: ~600 lines (495 code + 105 docs)
   • Net total: ~1475 lines (after 180-line deletion)

5. Deferred Issues Section: Clearly documents 8 low-priority issues to be addressed through documentation rather than code changes

6. Updated Testing Strategy: Specifies new tests required for each fix category

7. New Open Questions: Addresses fix-specific questions (concurrency limits, reference chain depth, address validation strategy, DNS timeouts, worker pool implementation)

8. Enhanced Future Work: Expanded with specific Phase 2 improvements based on review findings

Structural Improvements:

• More concise summary of approach (moved detailed specs to fix sections)
• Clear separation between completed work and required fixes
• Explicit validation checkpoints after each phase
• Comprehensive file change list with line counts
• Updated checklist reflecting new work phases

Key Decisions Documented:

• Concurrency limit: 100 probes (hardcoded constant)
• Reference chain depth: 5 levels (per RFC16)
• Total references: 20 maximum
• DNS timeout: 5 seconds
• Address validation: Reject private IPs by default
• Worker pool: Buffered channel semaphore (no external dependencies)

Estimated Timeline:

• Phase 1 (Critical): ~220 lines net (~20 code + 200 test - 200 deleted)
• Phase 2 (High): ~195 lines
• Phase 3 (Medium): ~280 lines
• Phase 4 (Docs): ~105 lines
• Total new work: ~600 lines

This revision transforms the plan from "implementation complete" to "implementation complete, now fixing review findings before PR creation".

[ben-dz]

Approved Work Plan (Revision 4)

Work Plan: Issue #2899 - Geoprobe Telemetry Agent Extensions

Issue: #2899 - geoprobe: telemetry agent extensions — probe measurement and offset delivery
Branch: ben-dz/doublezero-2899
Status: Implementation complete, ready for PR creation

Revision 4 - PR Creation

Current State

All implementation work has been completed and staged:

• 2 commits already on branch:
  • d4a5ce1d Create base modules to be used by geolocation services
  • 5e71724b Update formatting & overly verbose comments
• Additional security hardening changes staged (4,204 lines added across 15 files)
• All components have comprehensive test coverage

Remaining Tasks

1. Commit staged changes - Create a final commit with the security hardening work
2. Generate PR description - Use /pr-text skill to create PR description following project conventions
3. Create pull request - Use gh pr create to open PR against main branch

PR Content Summary

The PR will include:

• Base geoprobe infrastructure (LocationOffset, OffsetSigner, UDP transport)
• Coordinator for managing geoprobe lifecycle
• Pinger for TWAMP-based RTT measurements with retry support
• Publisher for delivering signed LocationOffset structures via UDP
• Security hardening:
  • ProbeAddress validation (reject private/loopback/link-local/reserved IPs)
  • Concurrency limiting (100 max concurrent operations)
  • Reference chain validation (MaxReferenceDepth=5, MaxTotalReferences=20)
  • DNS resolution timeout (5 seconds)
  • Configuration validation for geoprobe dependencies
  • Idempotent AddProbe/RemoveProbe operations
  • Mock interfaces for ServiceabilityClient and RPCClient
• Comprehensive unit tests for all components
• CHANGELOG.md updates
• Security considerations documentation

Implementation Plan

Step 1: Commit Staged Changes

Create a commit with descriptive message following project style (lowercase, component prefix).

Step 2: Generate PR Description

Use the /pr-text skill to generate a PR description following project conventions:

• Title format: telemetry/geoprobe: <short description>
• Summary bullets ordered by importance
• Focus on "what" and "why", not implementation details
• Include "Testing Verification" section
• No "Generated with Claude Code" footer

Step 3: Push and Create PR

Push branch to origin and create PR using gh pr create with the generated title and description.

Success Criteria

• All changes committed with descriptive commit message following project style
• Branch pushed to remote repository
• PR created with proper title and description
• PR resolves issue geoprobe: telemetry agent extensions — probe measurement and offset delivery #2899

---

STATUS: READY FOR EXECUTION - All implementation complete, just need to commit and create PR.