Large Format Tests (test chaining) in AATH

[nodlesh]

• Proposal to modify test structure in AATH (Aries Agent Test Harness) for improved efficiency and developer-friendliness
• Current approach: Independent tests that set up agents, connect, perform actions, and disconnect
• Suggested approach: Multiple agents connecting and performing a series of exchanges without restarting

Considerations and Potential Improvements:

• Connections are already reused across features when possible
• Agents don't restart for the whole run set unless necessary
• Implement pre- and post-test suite steps at the feature level
• Check for existing credentials before issuing new ones
• Create a large format test combining multiple smaller scenarios as a smoke test
• Use data tables within steps instead of example tables for scenarios to reduce execution time

Potential Challenges:

• Increased interdependence between tests may lead to more brittle systems
• Larger test scenarios may complicate root cause analysis for failures
• Need to balance between test isolation and execution speed

Additional Suggestions:

• Implement tagged smoke tests for PR validations
• Run full test suite nightly to maintain comprehensive coverage