cleanup instance is a no-op - AgenticAppId/AgenticUserId are never populated in generated config

[pratapladhani]

Description

a365 cleanup instance reads AgenticAppId and AgenticUserId from a365.generated.config.json to find the identity SP and agentic user to delete. However, these fields are never populated by any a365 command (setup, publish, deploy). They are always empty/null, so cleanup instance silently does nothing.

Steps to Reproduce

1. a365 setup all --skip-infrastructure --verbose
2. a365 publish --verbose
3. Admin activates blueprint, user creates instance
4. Examine a365.generated.config.json — AgenticAppId and AgenticUserId are not present
5. Run a365 cleanup instance — says "Starting instance cleanup..." then immediately completes without deleting anything

Expected Behavior

One of:

• Option A (Recommended): Deprecate this command. Since create-instance was already deprecated ("bypassed required registration steps"), and instances are now created through M365 Admin Center + Teams, the CLI never has the opportunity to write instance IDs to local config. cleanup instance should be deprecated and instance cleanup folded into cleanup blueprint (with cascade deletion or warnings).
• Option B: Discover instances dynamically using the blueprint ID. The AgentBlueprintId is already populated in a365.generated.config.json. Use it to query for child instances via Graph API — e.g., GET /beta/servicePrincipals/microsoft.graph.agentIdentity filtered by the blueprint's parent app, then resolve agentic users via identityParentId. This would make the command functional without requiring any new config fields.
• Option C: a365 publish or some other command should populate AgenticAppId and AgenticUserId in the generated config after instance creation is detected.

Actual Behavior

cleanup instance guards every deletion behind if (!string.IsNullOrWhiteSpace(config.AgenticAppId)) and if (!string.IsNullOrWhiteSpace(config.AgenticUserId)). Since these are never populated, all deletion blocks are skipped. The command completes "successfully" without cleaning up anything.

Code Reference

CleanupCommand.cs lines ~355-366:

// Delete agent identity application
if (!string.IsNullOrWhiteSpace(config.AgenticAppId))  // Always empty -> skipped
{
    logger.LogInformation("Deleting agent identity application...");
    await executor.ExecuteAsync("az", $"ad app delete --id {config.AgenticAppId}", ...);
}

// Delete agent user
if (!string.IsNullOrWhiteSpace(config.AgenticUserId))  // Always empty -> skipped
{
    logger.LogInformation("Deleting agent user...");
    await executor.ExecuteAsync("az", $"ad user delete --id {config.AgenticUserId}", ...);
}

Impact

Users believe they have cleaned up their instances but orphaned resources remain in Entra ID. Combined with the cleanup blueprint issue (which also doesn't clean up instances), there is no CLI path to properly clean up agent instances.

Additional Context

The create-instance command was deprecated per official docs: "bypassed required registration steps." Instance creation now happens exclusively through M365 Admin Center + Teams. Since the CLI no longer manages the instance lifecycle (create -> cleanup), this command is effectively dead code. Deprecating it and moving instance cleanup responsibility into cleanup blueprint (or a new cleanup all enhancement) would align the CLI with the current platform design.

[github-actions]

[AUTO-TRIAGE] Team Assistant Triage

This issue has been automatically analyzed and triaged.

AI Decision Reasoning: Type: Classified as 'bug' because the issue describes a command (cleanup instance) that fails to perform its intended function due to missing configuration fields, resulting in unexpected behavior.
Priority: Elevated to P1 due to security concern: The issue highlights a security concern where orphaned resources (agent identities and users) remain in Entra ID due to the command being non-functional, potentially leading to unauthorized access or resource misuse if these orphaned entities are exploited.
Copilot: Security issues require human review and should not be auto-fixed
Assignment: Security issue assigned to security lead. The issue highlights a security concern where orphaned resources (agent identities and users) remain in Entra ID due to the command being non-functional, potentially leading to unauthorized access or resource misuse if these orphaned entities are exploited.
Labels: Applied labels bug, P1, security based on issue type and priority

Labels and assignee have been applied based on this analysis.

[github-actions]

[AUTO-TRIAGE] Results

Classification: bug
Priority: P1
Confidence: 95.0%

Recommended Assignee: @pontemonti

Analysis:
type rationale: Classified as 'bug' because the issue describes a command (cleanup instance) that fails to perform its intended function due to missing configuration fields, resulting in unexpected behavior.

priority rationale: Elevated to P1 due to security concern: The issue highlights a security concern where orphaned resources (agent identities and users) remain in Entra ID due to the command being non-functional, potentially leading to unauthorized access or resource misuse if these orphaned entities are exploited.

copilot rationale: Security issues require human review and should not be auto-fixed

assignment rationale: Security issue assigned to security lead. The issue highlights a security concern where orphaned resources (agent identities and users) remain in Entra ID due to the command being non-functional, potentially leading to unauthorized access or resource misuse if these orphaned entities are exploited.

labels rationale: Applied labels bug, P1, security based on issue type and priority

[SUGGESTION] Suggested Approach

1. Deprecate the cleanup instance command entirely as suggested in Option A. Update the CLI command registry in src/Microsoft.Agents.A365.DevTools.Cli/Commands to remove cleanup instance and consolidate its functionality into cleanup blueprint. Modify the documentation in docs/commands.md to reflect this change and provide guidance on cascading deletion or warnings for instances tied to blueprints.
2. Implement Option B to dynamically discover instances using the AgentBlueprintId. In src/Microsoft.Agents.A365.DevTools.Cli/Services/CleanupService.cs, add logic to query the Graph API for service principals filtered by the blueprint's parent app ID. Use the identityParentId to resolve agentic users. Ensure proper error handling and logging for API calls, and add unit tests in src/Tests/Microsoft.Agents.A365.DevTools.Cli.Tests/Services/CleanupServiceTests.cs to validate this behavior.
3. If Option C is chosen, modify the publish command logic in src/Microsoft.Agents.A365.DevTools.Cli/Commands/PublishCommand.cs to detect instance creation and populate AgenticAppId and AgenticUserId in a365.generated.config.json. Use the System.Text.Json library to write these fields dynamically after instance creation. Add integration tests in src/Tests/Microsoft.Agents.A365.DevTools.Cli.Tests/Integration/PublishCommandIntegrationTests.cs to verify the config file updates.
4. Update the a365.generated.config.json schema documentation in docs/configuration.md to clarify the purpose of AgenticAppId and AgenticUserId fields. If these fields are deprecated, explicitly mark them as such and provide alternative workflows for instance cleanup using blueprint IDs.
5. Enhance logging in cleanup instance to provide more actionable feedback when AgenticAppId and AgenticUserId are null or empty. Modify src/Microsoft.Agents.A365.DevTools.Cli/Services/CleanupService.cs to include detailed warnings or errors in the logs, which can help users understand why no cleanup actions were performed.

---

Generated by TeamAssistant Auto-Triage powered by GitHub Models

[github-actions]

SLA Escalation Alert

This P1 issue has exceeded its SLA threshold.

Metric	Value
Priority	P1
SLA Threshold	48 hours
Time Since Update	55.4 hours
Current Assignees	@pontemonti

Action: This issue is already assigned to a Tech Lead but remains unresolved.

Escalating to Engineering Manager @tmlsousa for visibility.

Tech Leads: @sellakumaran, @pontemonti

---

Generated by AutoTriage SLA Escalation

[github-actions]

SLA Escalation Alert

This P1 issue has exceeded its SLA threshold.

Metric	Value
Priority	P1
SLA Threshold	48 hours
Time Since Update	58.6 hours
Current Assignees	@sellakumaran

Action: This issue is already assigned to a Tech Lead but remains unresolved.

Escalating to Engineering Manager @ajmfehr for visibility.

Tech Leads: @sellakumaran

---

Generated by AutoTriage SLA Escalation