Bug: Liquidation auction fails to run

[rowgraus]

Describe the bug

Have observed this bug twice now.

Set up Vaults to liquidate based on a decline in price
At liquidation time, UI shows vaults entering liquidating state. Dashboard shows correct amount of collateral available for auction
However, no updates are available about the current auction price from the CLI or in the dashboard
Bids that should have been picked up weren't

Seems like some issue in handoff between vault manager and auction but not clear.

Both times, vaults successfully liquidated (and auction ran) at the next scheduled time.

[rowgraus]

We've seen this twice but aren't fully confident it can be repro-ed on command

[Chris-Hibbert]

The timing is handled separately by the VaultDirector and the Auction. They run off the same schedule, but set up their timers independently. The only hand-off between them is in sending the collateral, and if the dashboard shows collateral available then vaults sent it and auction received it.

bids are supposed to be accepted by the auction regardless of the auction state. If the auction is active and the bid is in the money, then it'll be settled, otherwise (whether priced below current or no auction active) it should get added to the book.

Additional details that would be useful include

• any recent change to auction timing?
• What did nextAuctionStart show? Did it change when the auction should have started?
• I have an unreviewed PR (feat(auction): auction publish all bids off-chain #7618) ready for post-vaults that publishes the bids as received. Do you want to prioritize it sooner?

[Chris-Hibbert]

From @otoole-brendan:

"schedule": {
  "activeStartTime": "2023-05-10T19:00:02.000Z",
  "nextStartTime": "2023-05-10T19:10:02.000Z",
  "nextDescendingStepTime": "2023-05-10T19:00:02.000Z"
 },
 "book0": {
  "startPrice": "9.99 IST/ATOM",
  "startProceedsGoal": "206.025 IST",
  "startCollateral": "30 ATOM",
  "collateralAvailable": "30 ATOM"
 },
 "params": {
  "DiscountStep": "5.00%",
  "ClockStep": "00:00:20",
  "LowestRate": "45.00%"

right before xx:50

right after xx:50

Scheduling Parameters:

• Start Frequency: 600
• Price Lock Period: 180
• Clock Step: 20
• Start Delay: 2

[Chris-Hibbert]

My current suspicion is that this is due to clock jitter. If the auction wakeup action is called later than intended, it jumps to the next auction rather than start an auction late. The reasoning was that a big time discontinuity would invalidate the current prices, but that isn't applicable for small amounts of jitter.

My suggestion is that we notice that it's late, and choose a margin to allow staring late. I don't have a characterization of how much jitter we should expect normally.