Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial inter liquidation bidding CLI #7132

Merged
merged 4 commits into from
Mar 28, 2023
Merged

initial inter liquidation bidding CLI #7132

merged 4 commits into from
Mar 28, 2023

Conversation

dckc
Copy link
Member

@dckc dckc commented Mar 8, 2023

refs: #6930

Description

Add an inter command with sub-commands for inter bid by-price etc.
Cancel (exit) functionality is to come in a follow-on PR.

  • feat(inter-protcol): inter CLI with liquidation bidding, reserve
  • feat(inter-protocol): clientSupport for bidding, reserve
  • feat(agoric-cli): support explicit net, env, stdout authority

Security Considerations

Uses "want" in the same way that the contract does; that is: in a way that suggests, but does not actually provide, offer safety.

Scaling Considerations

The expected userbase is small-ish. Consequences of using it at a larger scale have not been explored.

Documentation Considerations

Brief descriptions of commands and options are provided. test/snapshots/test-inter-cli.js.md
should provide a handy reference as a basis for more thorough end-user docs (#6889).

Some effort went into reasonable diagnostics, distinguished from crashes / stack traces.

Testing Considerations

Unit tests of several different sorts are provided, though coverage is incomplete.

I did some manual testing with a local chain; @raphdev was able to reproduce some of my results.

@datadog-full-agoric
Copy link

datadog-full-agoric bot commented Mar 8, 2023

Datadog Report

Branch report: dc-bid-cli
Commit report: cc3e4b4

agoric-sdk: 11 Failed (0 Known Flaky), 0 New Flaky, 2195 Passed, 22 Skipped, 18m 22.63s Wall Time

❌ Failed Tests (11)

This report shows up to 5 failed tests.

  • Error: unexpected [undefined] - agoric.agoric-cli - Details

    Expand for error
     at NonNullish (packages/assert/src/assert.js:70:10)
       at getNetworkConfig (packages/agoric-cli/src/lib/rpc.js:38:21)
       at packages/agoric-cli/src/lib/rpc.js:42:36
       at async Promise.all (index 0)
       ---
         name: Error
         message: 'unexpected "[undefined]"'
         at: |-
           NonNullish (packages/assert/src/assert.js:70:10)
           getNetworkConfig (packages/agoric-cli/src/lib/rpc.js:38:21)
     ...
    
  • Error: unexpected [undefined] - agoric.agoric-cli - Details

    Expand for error
     at NonNullish (packages/assert/src/assert.js:70:10)
       at getNetworkConfig (packages/agoric-cli/src/lib/rpc.js:38:21)
       at packages/agoric-cli/src/lib/rpc.js:42:36
       ---
         name: Error
         message: 'unexpected "[undefined]"'
         at: |-
           NonNullish (packages/assert/src/assert.js:70:10)
           getNetworkConfig (packages/agoric-cli/src/lib/rpc.js:38:21)
           packages/agoric-cli/src/lib/rpc.js:42:36
     ...
    
  • test/test-main.js exited with a non-zero exit code: 1 - agoric.agoric-cli - Details

    Expand for error
     1..6
     # tests 6
     # pass 6
     # fail 2
     
    
  • test/test-main.js exited with a non-zero exit code: 1 - agoric.agoric-cli - Details

    Expand for error
     1..6
     # tests 6
     # pass 6
     # fail 2
     
    
  • make › before hook - agoric.cosmic-swingset - Details

    Expand for error
     ---
         name: AssertionError
         message: Rejected promise returned by test
         values:
           'Rejected promise returned by test. Reason:': |-
             Error {
               message: 'exit 2 from: make scenario2-setup',
             }
         at: 'ChildProcess.<anonymous> (file://test/scenario2.js:17:18)'
       ...
     ...
    

@dckc dckc force-pushed the dc-boot-check branch 6 times, most recently from e706981 to 478dc43 Compare March 14, 2023 20:11
Base automatically changed from dc-boot-check to master March 14, 2023 20:53
@datadog-full-agoric
Copy link

datadog-full-agoric bot commented Mar 15, 2023

Datadog Report

Branch report: dc-bid-cli
Commit report: 5fb6660

agoric-sdk: 1 Failed (0 Known Flaky), 0 New Flaky, 2918 Passed, 31 Skipped, 13m 19.03s Wall Time

❌ Failed Tests (1)

  • bootstrapTests › vaults-integration › bid for liquidation - agoric.vats - Details

    Expand for error
     ---
         name: AssertionError
         message: Rejected promise returned by test
         values:
           'Rejected promise returned by test. Reason:': |-
             Error {
               message: 'Invalid numeric data: "[NaN]"',
             }
         at: >-
           parseRatio (packages/zoe/src/contractSupport/ratio.js:373:15)
     ...
    

@dckc dckc changed the base branch from master to 6641-auctioneer March 16, 2023 21:59
Base automatically changed from 6641-auctioneer to master March 17, 2023 00:51
@dckc dckc force-pushed the dc-bid-cli branch 2 times, most recently from 5ad081f to 302c5d5 Compare March 21, 2023 02:00
@dckc
Copy link
Member Author

dckc commented Mar 21, 2023

Run auctions every 2 min for testing

These are the edits I make to tweak the parameters:

$ git stash show -p 3
diff --git a/packages/inter-protocol/src/proposals/econ-behaviors.js b/packages/inter-protocol/src/proposals/econ-behaviors.js
index d44510ac4..ddf76a9ce 100644
--- a/packages/inter-protocol/src/proposals/econ-behaviors.js
+++ b/packages/inter-protocol/src/proposals/econ-behaviors.js
@@ -519,8 +519,8 @@ export const startAuctioneer = async (
   },
   {
     auctionParams = {
-      startFreq: 3600n,
-      clockStep: 3n * 60n,
+      startFreq: 2n * 60n, // 3600n,
+      clockStep: 40n, // 3n * 60n,
       startingRate: 10500n,
       lowestRate: 4500n,
       discountStep: 500n,

Tracing Liquidation and Auctions

edits I use ``` diff --git a/packages/inter-protocol/src/auction/auctionBook.js b/packages/inter-protocol/src/auction/auctionBook.js index dec1e039b..80c28734e 100644 --- a/packages/inter-protocol/src/auction/auctionBook.js +++ b/packages/inter-protocol/src/auction/auctionBook.js @@ -48,7 +48,7 @@ const DEFAULT_DECIMALS = 9; * added in the appropriate place and settled when the price reaches that level. */

-const trace = makeTracer('AucBook', false);
+const trace = makeTracer('AucBook', true); // @@@@@@

/**

  • @typedef {{
    diff --git a/packages/inter-protocol/src/auction/auctioneer.js b/packages/inter-protocol/src/auction/auctioneer.js
    index c859cc361..b9a4a8cef 100644
    --- a/packages/inter-protocol/src/auction/auctioneer.js
    +++ b/packages/inter-protocol/src/auction/auctioneer.js
    @@ -29,7 +29,7 @@ import { AuctionState } from './util.js';

const { Fail, quote: q } = assert;

-const trace = makeTracer('Auction', false);
+const trace = makeTracer('Auction', true); // @@

/**
*
diff --git a/packages/inter-protocol/src/proposals/econ-behaviors.js b/packages/inter-protocol/src/proposals/econ-behaviors.js
index d44510ac4..ddf76a9ce 100644
--- a/packages/inter-protocol/src/proposals/econ-behaviors.js
+++ b/packages/inter-protocol/src/proposals/econ-behaviors.js
@@ -519,8 +519,8 @@ export const startAuctioneer = async (
},
{
auctionParams = {

  •  startFreq: 3600n,
    
  •  clockStep: 3n * 60n,
    
  •  startFreq: 2n * 60n, // 3600n,
    
  •  clockStep: 40n, // 3n * 60n,
     startingRate: 10500n,
     lowestRate: 4500n,
     discountStep: 500n,
    

diff --git a/packages/inter-protocol/src/vaultFactory/liquidation.js b/packages/inter-protocol/src/vaultFactory/liquidation.js
index 30361e725..fde3336c0 100644
--- a/packages/inter-protocol/src/vaultFactory/liquidation.js
+++ b/packages/inter-protocol/src/vaultFactory/liquidation.js
@@ -9,7 +9,7 @@ import { makeScalarMapStore } from '@agoric/store';

import { AUCTION_START_DELAY, PRICE_LOCK_PERIOD } from '../auction/params.js';

-const trace = makeTracer('LIQ', false);
+const trace = makeTracer('LIQ', true); // @@

/** @typedef {import('@agoric/time/src/types').TimerService} TimerService /
/
* @typedef {import('@agoric/time/src/types').TimerWaker} TimerWaker */
diff --git a/packages/inter-protocol/src/vaultFactory/orderedVaultStore.js b/packages/inter-protocol/src/vaultFactory/orderedVaultStore.js
index 7fd83d612..c15c6da7f 100644
--- a/packages/inter-protocol/src/vaultFactory/orderedVaultStore.js
+++ b/packages/inter-protocol/src/vaultFactory/orderedVaultStore.js
@@ -26,6 +26,7 @@ export const makeOrderedVaultStore = store => {
const debt = vault.getNormalizedDebt();
const collateral = vault.getCollateralAmount();
const key = toVaultKey(debt, collateral, vaultId);

  • console.log('addVault', { debt, collateral, vaultId, key });
    store.init(key, vault);
    return key;
    };
    diff --git a/packages/inter-protocol/src/vaultFactory/vault.js b/packages/inter-protocol/src/vaultFactory/vault.js
    index 9eb9f472d..2da92a50f 100644
    --- a/packages/inter-protocol/src/vaultFactory/vault.js
    +++ b/packages/inter-protocol/src/vaultFactory/vault.js
    @@ -750,6 +750,7 @@ export const prepareVault = (baggage, marshaller, zcf) => {
    const { facets } = this;

      const { helper } = facets;
    
  •      trace(this.state.idInManager, '.liquidating() @@@');
         helper.assignPhase(Phase.LIQUIDATING);
         helper.updateUiState();
       },
    

diff --git a/packages/inter-protocol/src/vaultFactory/vaultManager.js b/packages/inter-protocol/src/vaultFactory/vaultManager.js
index e224e6144..d55ac5201 100644
--- a/packages/inter-protocol/src/vaultFactory/vaultManager.js
+++ b/packages/inter-protocol/src/vaultFactory/vaultManager.js
@@ -66,7 +66,7 @@ import { Phase, prepareVault } from './vault.js';

const { details: X, Fail } = assert;

-const trace = makeTracer('VM', false);
+const trace = makeTracer('VM', true); // @@@@

/** @typedef {import('./storeUtils.js').NormalizedDebt} NormalizedDebt /
/
* @typedef {import('@agoric/time/src/types').RelativeTime} RelativeTime */

</details> 

@dckc dckc force-pushed the dc-bid-cli branch 4 times, most recently from c6b4aa3 to f8d0b76 Compare March 24, 2023 17:37
@codecov
Copy link

codecov bot commented Mar 24, 2023

Codecov Report

Merging #7132 (395a5af) into master (0985679) will decrease coverage by 8.45%.
The diff coverage is 82.90%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7132      +/-   ##
==========================================
- Coverage   79.35%   70.91%   -8.45%     
==========================================
  Files         396      450      +54     
  Lines       74796    86004   +11208     
  Branches        3        3              
==========================================
+ Hits        59355    60990    +1635     
- Misses      15440    24948    +9508     
- Partials        1       66      +65     
Impacted Files Coverage Δ
packages/inter-protocol/src/clientSupport.js 52.02% <73.97%> (+4.55%) ⬆️
packages/agoric-cli/src/lib/rpc.js 89.69% <74.41%> (+54.46%) ⬆️
packages/agoric-cli/src/commands/inter.js 84.55% <84.55%> (ø)
packages/agoric-cli/src/lib/format.js 54.40% <90.32%> (+14.63%) ⬆️
packages/agoric-cli/src/lib/wallet.js 71.01% <92.30%> (+16.32%) ⬆️
packages/agoric-cli/src/lib/chain.js 54.54% <100.00%> (+22.36%) ⬆️

... and 53 files with indirect coverage changes

Impacted file tree graph

@datadog-full-agoric
Copy link

datadog-full-agoric bot commented Mar 24, 2023

Datadog Report

Branch report: dc-bid-cli
Commit report: d7f0ee4

agoric-sdk: 0 Failed, 0 New Flaky, 3832 Passed, 57 Skipped, 28m 5.17s Wall Time

@dckc dckc marked this pull request as ready for review March 24, 2023 17:52
@dckc dckc changed the title toward bidding cli (WIP) initial inter liquidation bidding CLI Mar 24, 2023
Copy link
Contributor

@Chris-Hibbert Chris-Hibbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only looked closely at the portion that dealt specifically with adding bids and deposits. It looks fine to me, though I added some comments and questions.

The framework is opaque to me, and I didn't look closely at the rest.

callPipe: [['makeAddCollateralInvitation', []]],
},
proposal: { give },
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be optional offerArgs that look like { toRaise: istAmount }, coming in #6952. Do you want to add it now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know it's coming, but I don't quite understand it well enough to add it without being able to test it yet.

Comment on lines 41 to 43
const discount = r =>
100 - (Number(r.numerator.value) / Number(r.denominator.value)) * 100;
return { amount, record, price, discount };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discounts can be more or less than 100. Will this correctly distinguish 105% of oracle price from 95% and 5%?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the idea, yes...

There was a unit test in the clientSupport layer, where the discount is a number in the -1 to 1 range, that tested that -0.10 gives an offerBidScaling of makeRatio(11n, IST, 10n). But adding tests for the 3 cases you ask about showed that things could be tightened up.

Now I have tests for all 4 cases:

  const discounts = [
    { cliArg: 0.05, offerBidScaling: makeRatio(95n, ist.brand, 100n) },
    { cliArg: 0.95, offerBidScaling: makeRatio(5n, ist.brand, 100n) },
    { cliArg: -0.05, offerBidScaling: makeRatio(105n, ist.brand, 100n) },
    { cliArg: -0.1, offerBidScaling: makeRatio(110n, ist.brand, 100n) },
  ];

And I checked at the command line:

---discount=5 gives offerBidScaling of makeRatio(95n, IST, 100n).
---discount=-5 gives offerBidScaling of makeRatio(105n, IST, 100n).
---discount=95 gives offerBidScaling of makeRatio(5n, IST, 100n).

and --discount=950 gives an error:

error: option '--discount [percent]' argument '950' is invalid. must be between -100 and 100

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this transformation with discountRate in the contract is better ergonomically but I expect confusion in reading the two.

consider calling this +/- expression relativeDiscount

)
.requiredOption(
'--discount [number]',
'bid discount (%)',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'bid discount (%)',
'bid discount or markup (%)',

.description('Print an offer to bid collateral by price.')
.requiredOption('--price [number]', 'bid price', Number)
.requiredOption('--giveCurrency [number]', 'Currency to give', Number)
.requiredOption('--wantCollateral [number]', 'bid price', Number)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offer safety is available and sensible on a bid by price. Is it worth adding a separate option to let users separately specify their offer-safety want? The want in the offerArgs specifies the price, but doesn't provide offer safety.

(Offer safety is also available on bids by discount, though the meaning is more subtle. I haven't thought through the implications, but I know Zoe would enforce it if it was specified.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. I didn't realize. Let's do that in a follow-on PR, along with cancel / exit.

@@ -0,0 +1,111 @@
# Snapshot report for `test/test-inter-cli.js`

The actual snapshot is saved in `test-inter-cli.js.snap`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the actual snapshot be checked in or .gitignored?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My read of the ava snapshot testing docs is that you must check in the .snap file, as that's what determines whether the test passes or fails, and you should check in the .md file so that you can get a text representation of what changed.


> Command usage:

`Usage: inter [options] [command]␊
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to prefer the "␊" marks in the .md and its rendering? How much work to remove them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried .replace(/\r/g, '') but I guess that would be CRs, not LFs. hm.

@dckc
Copy link
Member Author

dckc commented Mar 25, 2023

@turadg I moved the inter command into agops. PTAL.

Copy link
Member

@turadg turadg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff! None of my suggestions are blocking and I see no automerge tag so I'll approve and leave to your discretion.

@@ -0,0 +1,376 @@
// @ts-check
import { CommanderError, InvalidArgumentError } from 'commander';
import { M, matches } from '@agoric/store';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regrettable to have cli depend on store. consider a comment here,

// TODO remove dep after https://github.com/Agoric/agoric-sdk/issues/7090

@@ -1,20 +1,23 @@
#!/usr/bin/env node
/* eslint-disable @jessie.js/no-nested-await */
/* global process */
/* global fetch */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you preferred import process to the global. similarly you could have,

import fetch from 'node-fetch';

though I trust the global slightly more than the package import.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node-fetch seems to be a separate package.

The fetch global doesn't seem to be available from a node built-in package the way process is.

Copy link
Member

@turadg turadg Mar 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node-fetch seems to be a separate package.

Yes, it implements the window.fetch API. But it's not as separate as it may seem because this shim is loaded in the bin scripts in this package. IOW node-fetch is what you're really getting here anyway. Better for it to be explicit, imo, to avoid the confusion demonstrated in this thread.

stderr: process.stderr,
createCommand,
execFileSync,
clock: () => Date.now(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this common to name the time getter clock? now() is more familiar to me fwiw

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed. will change to now

Comment on lines 49 to 56
await program.parseAsync(process.argv).catch(err => {
if (err instanceof CommanderError) {
console.error(err.message);
} else {
console.error(err); // CRASH! show stack trace
}
process.exit(1);
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we do have top-level await, consider using regular try/catch

Suggested change
await program.parseAsync(process.argv).catch(err => {
if (err instanceof CommanderError) {
console.error(err.message);
} else {
console.error(err); // CRASH! show stack trace
}
process.exit(1);
});
try {
await program.parseAsync(process.argv);
} catch (err) {
if (err instanceof CommanderError) {
console.error(err.message);
} else {
console.error(err); // CRASH! show stack trace
}
process.exit(1);
};

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

top-level await feels weird to me... but it clearly does exactly the same thing. What I want is for main programs to just export main and have the runtime call it, passing in all I/O access explicitly.

I guess that's why we're building endo. Meanwhile, when in Rome...

Comment on lines 109 to 115
{ env, stdout, stderr, clock, execFileSync, createCommand },
{ fetch },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you usually put io powers in one param?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I usually do. In this case, it seemed like network access and process stuff fit in distinct piles. Or something.

Comment on lines +293 to +301
const { body, slots } = JSON.parse(txt);
const record = m.unserialize({ body, slots });
coalescer.update(record);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider a feature like,

Suggested change
const { body, slots } = JSON.parse(txt);
const record = m.unserialize({ body, slots });
coalescer.update(record);
coalescer.updateSerialized(txt);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

making the coalescer aware of marshaling looks kinda awkward

value,
} = amt;
const { brand, value } = amt;
// @ts-expect-error XXX BoardRemote
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not type AssetDescriptor?

Copy link
Member Author

@dckc dckc Mar 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand. AssetDescriptor is used on line 36.

export const networkConfig =
'AGORIC_NET' in process.env && process.env.AGORIC_NET !== 'local'
? await fromAgoricNet(NonNullish(process.env.AGORIC_NET))
export const getNetworkConfig = async env =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export const getNetworkConfig = async env =>
/** @returns {MinimalNetworkConfig} */
export const getNetworkConfig = async env =>

const url = (path = 'published', { kind = 'children', height = 0 } = {}) =>
`/abci_query?path=%22/custom/vstorage/${kind}/${path}%22&height=${height}`;

const readStorage = (path = 'published', { kind = 'children', height = 0 }) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@dckc dckc added the automerge:rebase Automatically rebase updates, then merge label Mar 27, 2023
@turadg turadg mentioned this pull request Mar 27, 2023
dckc added 4 commits March 28, 2023 03:34
 - agoric-cli: support explicit net, env, stdout authority
 - update makeAmountFormatter() use of BoardRemote; refine types
 - nicer diagnostics from vstorage
 - prune debug logging from readFully
 - thread invitationBrand thru coalesceWalletState
 - show stack trace for bugs only
 - some unit test coverage with mock network etc.
 - docs snapshot test
@mergify mergify bot merged commit 7268232 into master Mar 28, 2023
@mergify mergify bot deleted the dc-bid-cli branch March 28, 2023 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge:rebase Automatically rebase updates, then merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants