Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter out future purge dates resources #1910

Merged
merged 1 commit into from
Aug 14, 2021

Conversation

weshaggard
Copy link
Member

Most (>8k) of the deleted key vaults have a future purge date and we fail to delete them and we end up timing out so I'm filtering so that we only attempt to delete ones with past due purge dates.

@weshaggard weshaggard requested review from benbp and heaths August 13, 2021 23:55
@weshaggard weshaggard requested a review from a team as a code owner August 13, 2021 23:55
@weshaggard
Copy link
Member Author

/azp run automation - live-test-resource-cleanup

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@weshaggard weshaggard changed the title Filter out fugure purge dates resources Filter out future purge dates resources Aug 13, 2021
@weshaggard
Copy link
Member Author

/azp run automation - live-test-resource-cleanup

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@weshaggard
Copy link
Member Author

@heaths I'm merging this for now to give the cleanup automation a change to finish as it keeps timing out currently and even locally I seem to keep hitting hangs or timeouts on certain resources. If you know of a better way to filter what we can and cannot delete yet let me know.

@weshaggard weshaggard merged commit e2ebf99 into Azure:main Aug 14, 2021
$purgeableResources = Get-PurgeableResources
$allPurgeCount = $purgeableResources.Count

# Filter down to the ones that we can actually perge.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"purge"

/cc @weshaggard

$allPurgeCount = $purgeableResources.Count

# Filter down to the ones that we can actually perge.
$purgeableResources = $purgeableResources.Where({ $purgeDate = $_.ScheduledPurgeDate -as [DateTime]; (!$purgeDate -or $now -gt $purgeDate) })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now how you filter them. You need to check EnableSoftDelete. You've also not purged the vaults early enough. The default is 90d, so you're effectively leaving hundreds of vaults un-purged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no point in purging vaults in the future. Once the purge date is hit, they are purged automatically.

@heaths
Copy link
Member

heaths commented Aug 16, 2021

Most (>8k) of the deleted key vaults have a future purge date and we fail to delete them and we end up timing out so I'm filtering so that we only attempt to delete ones with past due purge dates.

That's not why some fail to delete. The ones that fail to delete is because EnableSoftDelete is null or false. As I explained in discussions with my PR, the default for EnableSoftDelete was false, but now is true and cannot be disabled anymore for new vaults. That's why I try to delete them anyway.

Vaults and managed HSMs are automatically deleted on the purge dates. That's what that property is for. By not deleting vaults and managed HSMs until after their purge date, you've effectively obviated the whole point of this clean-up exercise.

heaths added a commit that referenced this pull request Aug 16, 2021
Reverts #1910. Vaults and managed HSMs are automatically purged on their purge date. The point was to purge them daily to preserve capacity. The default purge date is +90 days.
ghost pushed a commit that referenced this pull request Aug 18, 2021
Reverts #1910. Vaults and managed HSMs are automatically purged on their purge date. The point was to purge them daily to preserve capacity. The default purge date is +90 days.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants