Skip to content

Refactor the gateway integration test suite into a dedicated ros2_medkit_integration_tests package#227

Open
bburda wants to merge 17 commits intomainfrom
feat/improve-integration-tests
Open

Refactor the gateway integration test suite into a dedicated ros2_medkit_integration_tests package#227
bburda wants to merge 17 commits intomainfrom
feat/improve-integration-tests

Conversation

@bburda
Copy link
Collaborator

@bburda bburda commented Feb 19, 2026 •
edited
Loading

Pull Request

Summary

Refactor the gateway integration test suite into a dedicated ros2_medkit_integration_tests package with a shared Python test library (ros2_medkit_test_utils), two-tier test organization (features + scenarios), CMake labels, and structural fixes for flaky tests.

What changed:

  • New package ros2_medkit_integration_tests - contains all integration tests, demo nodes, launch files, config YAMLs, and a shared Python test library
  • Shared test library ros2_medkit_test_utils (4 modules) - eliminates ~1,200 lines of duplicated code: get_coverage_env() (9 copies), health polling (9 variants), demo node launch boilerplate, HTTP helpers
  • Monolithic test_integration.test.py (4,990 lines, 141 tests) split into 16 feature test files (~90 tests) and 10 scenario test files (~50 tests) - each file launches its own gateway with only the demo nodes it needs
  • CMake labels (integration;feature, integration;scenario) enable colcon test --ctest-args -L feature filtering
  • Gateway package cleaned up - removed integration test files, demo node build targets, and test-only dependencies; unit tests (26 GTest targets) stay unchanged

Key library modules:

  • constants.py - shared API paths, ports, timeouts (replaces 50+ duplicated lines)
  • coverage.py - single get_coverage_env() (replaces 9 copies, ~270 lines)
  • launch_helpers.py - create_test_launch() factory (replaces ~500 lines of inline launch boilerplate)
  • gateway_test_case.py - GatewayTestCase base class with health polling, HTTP helpers, discovery waiters, assertion helpers

Issue

Type

  • Bug fix
  • New feature or tests
  • Breaking change
  • Documentation only

Testing

Build verification:

source /opt/ros/jazzy/setup.bash
colcon build --symlink-install
# All 7 packages build successfully

Gateway unit tests (unchanged):

colcon test --packages-select ros2_medkit_gateway
colcon test-result --verbose
# 26/26 GTest targets pass

Integration test registration:

cd build/ros2_medkit_integration_tests
ctest -N                  # 26 tests registered (16 feature + 10 scenario)
ctest -N -L feature       # 16 feature tests
ctest -N -L scenario      # 10 scenario tests
ctest -N -L integration   # 26 total

Run integration tests:

colcon test --packages-select ros2_medkit_integration_tests
colcon test-result --verbose

Coverage still works from separate package - get_coverage_env() resolves ros2_medkit_gateway's build dir via ament_index_python, gateway binary writes .gcda to GCOV_PREFIX, CI lcov --capture --directory build scans the entire build directory.

Issue #222 (flaky test_101): The race condition is resolved structurally - test_scenario_action_lifecycle.test.py replaces the problematic test_100 + test_101 sequence. Each test creates its own execution (no concurrent action goals from previous tests).

Issue #139 (multi-discovery-mode): Two new scenario files (test_scenario_discovery_manifest.test.py, test_scenario_discovery_hybrid.test.py) test manifest-only and hybrid modes. Shared assertion helpers (assert_entity_exists, assert_entity_has_capabilities, etc.) in GatewayTestCase support mode-specific validation. Runtime mode is tested implicitly by all feature tests (default mode).

Checklist

  • Breaking changes are clearly described (and announced in docs / changelog if needed)
  • Tests were added or updated if needed
  • Docs were updated if behavior or public API changed

@bburda bburda self-assigned this Feb 19, 2026
Copilot AI review requested due to automatic review settings February 19, 2026 16:43
Copilot AI reviewed Feb 19, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request extracts integration tests and demo nodes from ros2_medkit_gateway into a new dedicated package ros2_medkit_integration_tests. The refactoring improves test organization by separating integration tests into two categories: scenario tests (end-to-end stories) and feature tests (atomic, independent tests). A new ros2_medkit_test_utils Python package provides shared utilities including launch helpers, base test classes, and constants.

Changes:

  • Created new ros2_medkit_integration_tests package with demo nodes, test utilities, and reorganized tests
  • Moved 9 demo node C++ files from ros2_medkit_gateway/test/demo_nodes/ to the new package
  • Split monolithic integration tests into 23 smaller test files organized by scenario vs. feature
  • Removed integration test dependencies from ros2_medkit_gateway package.xml and CMakeLists.txt
  • Updated launch files and test references to use new package names

Reviewed changes

Copilot reviewed 39 out of 49 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/ros2_medkit_integration_tests/package.xml New package metadata with test and demo node dependencies
src/ros2_medkit_integration_tests/CMakeLists.txt Build config for demo nodes and test registration with proper timeouts
src/ros2_medkit_integration_tests/setup.cfg Python package setup for test utilities
src/ros2_medkit_integration_tests/ros2_medkit_test_utils/* Shared test utilities: constants, coverage helpers, launch factories, base test class
src/ros2_medkit_integration_tests/test/scenarios/* 9 scenario test files (end-to-end stories, 300s timeout)
src/ros2_medkit_integration_tests/test/features/* 14 feature test files (atomic tests, 120s timeout)
src/ros2_medkit_integration_tests/demo_nodes/* 9 demo node C++ files moved from gateway package
src/ros2_medkit_integration_tests/launch/demo_nodes.launch.py Launch file updated to reference new package
src/ros2_medkit_gateway/package.xml Removed test dependencies (pytest, launch_testing, etc.)
src/ros2_medkit_gateway/CMakeLists.txt Removed integration tests and demo node build targets

@bburda bburda requested a review from mfaferek93 February 19, 2026 16:48
@bburda bburda changed the title Feat/improve integration tests Refactor the gateway integration test suite into a dedicated ros2_medkit_integration_tests package Feb 19, 2026
@bburda bburda marked this pull request as draft February 19, 2026 17:10
@bburda bburda force-pushed the feat/improve-integration-tests branch from ae470cf to 8050710 Compare February 19, 2026 21:16
@bburda bburda marked this pull request as ready for review February 19, 2026 22:26
@bburda bburda force-pushed the feat/improve-integration-tests branch from 8050710 to 86cd8f4 Compare February 20, 2026 08:56
bburda added 17 commits February 20, 2026 13:46
@bburda
feat: add ros2_medkit_integration_tests package skeleton
adb4e60 
New ament_cmake package for integration tests and demo nodes.
Includes CMakeLists.txt with file-glob test discovery and
integration;feature / integration;scenario CMake labels.

Refs #139, #222
@bburda
feat: add ros2_medkit_test_utils shared test library
2e70673 
Four modules replacing ~1200 lines of duplicated code:
- constants.py: API paths, ports, timeouts
- coverage.py: unified get_coverage_env()
- launch_helpers.py: factory for gateway/demo/fault_manager nodes
- gateway_test_case.py: base class with health polling, HTTP helpers,
  wait helpers, and discovery assertions

Refs #139, #222
@bburda
feat: move demo nodes and launch file to integration test package
3210366 
9 demo C++ executables and demo_nodes.launch.py moved from
ros2_medkit_gateway. Package references updated to
ros2_medkit_integration_tests.

Refs #139, #222
@bburda
feat: add test_health feature test — validates new package E2E
482a42e 
First integration test in the new package. Verifies gateway health,
root, version, and docs endpoints. Confirms test_utils library,
demo node launching, and coverage collection all work.

Refs #139, #222
@bburda
feat: migrate feature tests from test_integration.test.py
c28be74 
11 feature test files covering entity listing, data read/write,
operations, configuration, faults, HATEOAS, routing, bulk data,
SSE, and snapshots. All tests use GatewayTestCase base class
and descriptive test names.

Refs #139, #222
@bburda
feat: migrate auth, CORS, TLS, and heuristic discovery feature tests
d8be319 
Migrated from gateway package. Using shared test_utils for
coverage, constants, and launch helpers. Custom configs preserved
for auth (JWT), CORS (dual gateway), TLS (certs), and heuristic
(app_strategy params).

Refs #139, #222
@bburda
feat: add scenario tests — self-contained E2E stories
4485fef 
6 scenario files: action lifecycle, config management, fault
lifecycle, fault inspection, bulk data download, data publish.
Each scenario is self-contained with descriptive docstrings.
Fixes #222 by eliminating concurrent action goal race condition.

Fixes #222
Refs #139
@bburda
feat: add discovery mode scenario tests (manifest, hybrid)
5af7675 
Replaces test_discovery_manifest and test_discovery_hybrid from
gateway package with scenario-style tests using shared assertion
helpers. Each scenario validates entity structure, capabilities,
and mode-specific behavior.

Refs #139
@bburda
feat: migrate bulk data upload and subscriptions scenario tests
744085a 
Migrated from gateway package with scenario-style structure.
Using shared test_utils for launch helpers and base test case.

Refs #139
@bburda
refactor: remove integration tests and demo nodes from gateway
3c60bff 
Integration tests and demo nodes moved to
ros2_medkit_integration_tests package. Gateway retains only
production code and GTest unit tests.

Refs #139, #222
@bburda
fix: resolve CI failures in integration tests
d8139ae 
- Add 'calibration_service' and 'long_calibration_action' aliases to
  DEMO_NODE_REGISTRY (fixes KeyError in test_entity_routing and
  test_operations_api)
- Fix camelCase 'faultCode' -> snake_case 'fault_code' in
  test_scenario_fault_lifecycle to match actual API response
- Replace strict assertExitCodes with SIGTERM-tolerant exit code check
  across all 24 test files (exit code -15 is expected during
  launch_testing shutdown)
- Add polling loop in test_scenario_discovery_hybrid test_15 to wait
  for runtime linking (apps become online asynchronously after nodes
  start)
@bburda
fix: graceful shutdown for all demo nodes
a960b05 
- long_calibration_action: replace detached thread with joinable thread
  and atomic shutdown flag; add try-catch for goal handle interactions;
  use set_terminate to handle rclcpp_action race during SIGINT shutdown
- All timer-based demo nodes: cancel timers in destructor to prevent
  callbacks firing during node destruction (fixes SIGSEGV on Humble)
- Revert gateway-only exit code guard: all processes must exit cleanly
- Fix test_discovery_heuristic exit code check (was silently passing)
@bburda
docs: add README, design doc, and fix stale references for integratio…
2d24884 
…n tests package

- Add README.md with package structure, test templates, and GatewayTestCase API
- Add design/index.rst with PlantUML architecture diagram and test catalog
- Add symlink and toctree entry in docs/design/
- Add package description in docs/introduction.rst
- Fix demo_nodes.launch.py package reference in getting_started.rst
- Update devcontainer.rst test filtering for new package structure
@bburda
fix: resolve CI failures and exclude vendored code from coverage
032e44d 
- Add REQUIRED_APPS to test_hateoas discovery wait so temp_sensor is
  guaranteed to be discovered before tests run (fixes 404 on all platforms)
- Increase fault_manager test timeout from 1s to 3s for Humble's slower
  DDS service discovery
- Accept either "not available" or "timed out" error in service
  availability tests (wait_for_service behavior varies across distros)
- Exclude vendored/ from coverage in both codecov.yml and CI lcov filter
@bburda
fix: use unique node names in fault_manager tests for Humble DDS compat
5d061a6 
On Humble (CycloneDDS), reusing the same node name across sequential
GTest cases causes stale DDS participant discovery state. This corrupts
service responses in the second test when the first test did not create
a service. Using unique node names per test eliminates the collision.

Fixes test_fault_manager on Humble CI (GetSnapshotsSuccessWithValidJson).
@bburda
feat: migrate rate limiting integration test to integration tests pac…
88e0dab 
…kage
@bburda
chore: remove stale "Migrated from" comments from integration tests
f0d0c3f
@bburda bburda force-pushed the feat/improve-integration-tests branch from 86cd8f4 to f0d0c3f Compare February 20, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@mfaferek93 mfaferek93 Awaiting requested review from mfaferek93

At least 1 approving review is required to merge this pull request.

Assignees

@bburda bburda

Labels

None yet

Projects

None yet

Milestone

No milestone

1 participant

@bburda

Comments