Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE - just commentary and questions as I read #4553

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions packages/run-protocol/bundles/install-on-chain.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
// @ts-check

// Even though this is in the bundles/ directory, it is a source file.
// However, because it is in that directory, it seems exempt from eslint.
// As a result, it looks like several uncought type errors have crept
// in over time. All seem to be failures to pick up ambient types.

import { E } from '@agoric/eventual-send';

import '@agoric/governance/exported.js';
import '../exported.js';
import '../src/vaultFactory/type-imports.js';
import '@agoric/vats';

import { Far } from '@endo/far';
import { PROTOCOL_FEE_KEY, POOL_FEE_KEY } from '../src/vpool-xyk-amm/params.js';
Expand Down
3 changes: 3 additions & 0 deletions packages/run-protocol/globals.d.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
// Why is there one of these in run-protocol?
// Note that vscode does its red underline thing on "function" below.

interface VatData {
makeKind: function;
makeDurableKind: function;
Expand Down
7 changes: 6 additions & 1 deletion packages/run-protocol/jsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,10 @@
"strictNullChecks": true,
"moduleResolution": "node",
},
"include": ["src/**/*.js", "test/**/*.js", "exported.js", "globals.d.ts"],
"include": [
"src/**/*.js",
"test/**/*.js",
"exported.js",
"globals.d.ts",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is reformatted only because I tried an experiment of adding `"bundles/install-on-chain.js" to this list, so it would no longer be exempt from linting.

],
}
92 changes: 65 additions & 27 deletions packages/run-protocol/src/collect.js
Original file line number Diff line number Diff line change
@@ -1,29 +1,67 @@
// @ts-check

const { entries, fromEntries, keys, values } = Object;

export const Collect = {
/**
* @param {Record<string, V>} obj
* @param {(v: V) => U} f
* @returns {Record<string, U>}
* @template V
* @template U
*/
mapValues: (obj, f) => fromEntries(entries(obj).map(([p, v]) => [p, f(v)])),
/**
* @param {X[]} xs
* @param {Y[]} ys
* @returns {[X, Y][]}
* @template X
* @template Y
*/
zip: (xs, ys) => xs.map((x, i) => [x, ys[i]]),
/**
* @param {Record<string, ERef<V>>} obj
* @returns {Promise<Record<string, V>>}
* @template V
*/
allValues: async obj =>
fromEntries(Collect.zip(keys(obj), await Promise.all(values(obj)))),
};
import { deeplyFulfilled } from '@endo/marshal';

// const { entries, fromEntries, keys, values } = Object;
const { entries, fromEntries } = Object;

// Why the `Collect` object? In any case, it was missing a `harden` or `Far`.
// But why not just export the useful functions directly?

// Huh. I didn't know you could put the @template at the bottom.
// See `objectMap` in objArrayConversion.js
// Read the qualifiers in the comment!
/**
* @param {Record<string, V>} obj
* @param {(v: V) => U} f
* @returns {Record<string, U>}
* @template V
* @template U
*/
export const mapValues = (obj, f) =>
harden(fromEntries(entries(obj).map(([p, v]) => [p, f(v)])));
harden(mapValues);

/**
* @param {X[]} xs
* @param {Y[]} ys
* @returns {[X, Y][]}
* @template X
* @template Y
*/
export const zip = (xs, ys) => harden(xs.map((x, i) => [x, ys[i]]));
harden(zip);

// Are you sure you only want one level?
// See deeplyFulfilled in deeplyFulfilled.js
// /**
// * @param {Record<string, ERef<V>>} obj
// * @returns {Promise<Record<string, V>>}
// * @template V
// */
// export const allValues = async obj => {
// // Get both keys and values before the `await`, so they're consistent
// // (assuming that `obj` is not a proxy, which we don't validate)
// const ks = keys(obj);
// const vs = await Promise.all(values(obj));
// return harden(fromEntries(zip(ks, vs)));
// };
// harden(allValues);

// This should have worked, but was passed something non-hardened
// export const allValues = deeplyFulfilled;
// So I tried the following instead
export const allValues = obj => deeplyFulfilled(harden(obj));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left this uncommented for now, and lines 37-49 commented above, so the resulting failures will appear under CI. See comment below for the bug causing these failures.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this harden masks, rather than fixes, bugs elsewhere. The passing around of those unhardened objects may have already caused problems before they reach here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deeplyFulfilled loses the types of its args. So when I know I'm dealing with a Record<string, T>, I much prefer allValues.

I can work on the harden aspects as well, though.

harden(allValues);

// This didn't work either. Investigating why led to a
// missing `harden` in endo, in the last line of `bundleZipBase64`.
// ```js
// return { endoZipBase64, moduleFormat: 'endoZipBase64' };
// ```
// should be
// ```js
// return harden({ endoZipBase64, moduleFormat: 'endoZipBase64' });
// ```
// With this fixed, the second replacement of `allValues` above does
// work.
16 changes: 9 additions & 7 deletions packages/run-protocol/src/econ-behaviors.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import '../exported.js';

import { PROTOCOL_FEE_KEY, POOL_FEE_KEY } from './vpool-xyk-amm/params.js';

import { Collect } from './collect.js';
import { allValues, mapValues } from './collect.js';

const { entries, keys } = Object;

Expand Down Expand Up @@ -50,8 +50,8 @@ export const startEconomicCommittee = async (
const bundles = await governanceBundles;
keys(bundles).forEach(key => assert(shared.contract[key]));

const installations = await Collect.allValues(
Collect.mapValues(bundles, bundle => E(zoe).install(bundle)),
const installations = await allValues(
mapValues(bundles, bundle => E(zoe).install(bundle)),
);

const [installAdmin, instanceAdmin] = await collectNameAdmins(
Expand Down Expand Up @@ -167,6 +167,7 @@ export const setupAmm = async ({
E(instanceAdmin).update('ammGovernor', g.instance),
]);
};
harden(setupAmm);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not surprised to see lots of functions exported but not hardened. Fortunately it is not very dangerous. But it is a hazard we need to avoid. It leaves mutable top-level state, such that two parties importing the same function from the same module instance can now communicate without calling the function. This isn't POLA and breaks reasoning by separation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is a hazard we need to avoid

Could missing harden be detected automatically? i.e. are there consistent rules for when to use harden or not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there are consistent rules:

all objects made by literal expressions (object literals, array literals, the many forms of function literals) must be tamper-proofed with harden before they can be aliased or escape from their static context of origin -- https://github.com/endojs/Jessie#must-freeze-api-surface-before-use

Automatic detection is an open research problem, IIUC. agoric-labs/jessica#35

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • a) One abstract rule is that an object should not escape the abstraction it is born in without first being hardened.
  • b) A related rule is that module instances should be pure --- they should have no mutable top level state.

On #b, We have tried several times to write down static rules that would enforce these abstract rules. I think @michaelfig has grammatical restrictions for pure Jessie modules somewhere. @michaelfig ?

On #a, a conservative rule that works almost all the time is to just put a harden(...) or Far(..., ...) around the outer literal (object literal, array literal, function literal, class literal, regexp literal) where the object is first born. Then, no one gets to see the object in its pre-hardened state.

Occasionally, we do have valid code where we locally mutate the object before hardening it, but still harden it before it might escape. To enforce that statically would involve treating all variables holding the object as "owning" variables in the reference-capability sense. Or as linear or affine variables or something. The key is that we statically know that the object hasn't escaped via an alias before it's been hardened. We have sketched static rules for this but never got very far. Fortunately, the need for such delayed hardening happens even more rarely than I expected. We can almost always simply follow the conservative harden-the-literal rule above.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On #b, a conservative rule is to only have top level const declarations of hardened things. Again, this conservative rule works almost always.

For exported functions, we follow a pattern of a slightly delayed harden:

export const foo = x => x * x;
harden(foo);

This still hardens foo before anything could observe foo in its unhardened state.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dckc . I answered before I saw your response.

Copy link
Member

@turadg turadg Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would an imperfect but worthwhile rule be that named exports from a module have a corresponding harden? We could pretty easily lint for that.

E..g,
FAIL

export const sayHi = () => {
  console.log("hi");
};

PASS

export const sayHi = () => {
  console.log("hi");
};
harden(sayHi);

// eslint-disable-next-line endo/hardened-exports
export const foo = {};

EDIT: I also was typing while @erights was. If I'm reading "this conservative rule works almost always" right then the answer is yes that we could have a lint rule for that.

When it doesn't work how obvious is it? Would someone writing the module know to be able to work around it or suppress the lint? Or would might they ship something overly hardened and break downstream. (With or without the lint we need developers to know when to harden, so I don't suppose linting could be any worse and it should help both educate devs and help catch things they missed.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may need to accept harden or Far there for the rule.


/**
* @param {BootstrapPowers & {
Expand Down Expand Up @@ -222,7 +223,7 @@ export const startVaultFactory = async (
);

const bundles = await vaultBundles;
const installations = await Collect.allValues({
const installations = await allValues({
VaultFactory: E(zoe).install(bundles.VaultFactory),
liquidate: E(zoe).install(bundles.liquidate),
});
Expand Down Expand Up @@ -307,12 +308,13 @@ export const startVaultFactory = async (
),
]);
};
harden(startVaultFactory);

/** @param { BootstrapPowers } powers */
export const configureVaultFactoryUI = async ({
consume: { agoricNames, nameAdmins, board, zoe },
}) => {
const installs = await Collect.allValues({
const installs = await allValues({
amm: E(agoricNames).lookup('installation', 'amm'),
vaultFactory: E(agoricNames).lookup('installation', 'VaultFactory'),
contractGovernor: E(agoricNames).lookup('installation', 'contractGovernor'),
Expand All @@ -325,11 +327,11 @@ export const configureVaultFactoryUI = async ({
'binaryVoteCounter',
),
});
const instances = await Collect.allValues({
const instances = await allValues({
amm: E(agoricNames).lookup('instance', 'amm'),
vaultFactory: E(agoricNames).lookup('instance', 'VaultFactory'),
});
const central = await Collect.allValues({
const central = await allValues({
brand: E(agoricNames).lookup('brand', CENTRAL_ISSUER_NAME),
issuer: E(agoricNames).lookup('issuer', CENTRAL_ISSUER_NAME),
});
Expand Down
3 changes: 3 additions & 0 deletions packages/run-protocol/src/vaultFactory/burn.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ import '@agoric/zoe/exported.js';

import { E } from '@agoric/eventual-send';

// No burning going on here. Why is this file named burn.js ?
// Should this be moved to the contractSupport library of helpers?

/**
* @param {ContractFacet} zcf
* @param {ZCFMint} zcfMint
Expand Down
2 changes: 2 additions & 0 deletions packages/run-protocol/src/vaultFactory/collectRewardFees.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
// @ts-check

// Should this be moved to the contractSupport library of helpers?

/**
* @param {ContractFacet} zcf
* @param {ZCFSeat} feeSeat
Expand Down
6 changes: 6 additions & 0 deletions packages/run-protocol/src/vaultFactory/interest.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ const makeResult = (latestInterestUpdate, interest, newDebt) => ({
newDebt,
});

// What about leap years?
// https://www.grc.nasa.gov/www/k-12/Numbers/Math/Mathematical_Thinking/calendar_calculations.htm
// says the average Gregorian year is 365.2425
// whereas the astronomical figure (tropical year) is 365.2422
// No mention made of the occasional leap second!
// Whether we care depends on what we do with it.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll admit this is overly pedantic and probably doesn't matter at all. But I am curious if, for our use, there is an unambiguous right answer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had wondered the same thing. It's a negligible difference and for loans the regulations allow treating interest in a leap year as any other.
https://www.consumerfinance.gov/rules-policy/regulations/1030/interp-7/#:~:text=Leap%20year.,earn%20interest%20for%20February%2029.

  1. Leap year. Institutions may apply a daily rate of 1/366 or 1/365 of the interest rate for 366 days in a leap year, if the account will earn interest for February 29.

export const SECONDS_PER_YEAR = 60n * 60n * 24n * 365n;
const BASIS_POINTS = 10000;
// single digit APR is less than a basis point per day.
Expand Down
10 changes: 5 additions & 5 deletions packages/run-protocol/test/amm/vpool-xyk-amm/setup.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
makeNameAdmins,
makePromiseSpace,
} from '@agoric/vats/src/core/utils.js';
import { Collect } from '../../../src/collect.js';
import { allValues } from '../../../src/collect.js';
import {
setupAmm,
startEconomicCommittee,
Expand Down Expand Up @@ -76,12 +76,12 @@ export const setupAMMBootstrap = async (
produce.nameAdmins.resolve(nameAdmins);

/** @type {Record<string, Promise<{moduleFormat: string}>>} */
const governanceBundlePs = {
const governanceBundlePs = harden({
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an example of a missing harden bug that's masked by the last version of allValues, where it auto-hardens its argument. The second version of allValues which is just an alias for deeplyFulfilled would fail immediately when applied to this unhardened record, quickly revealing rather than masking the bug.

contractGovernor: contractGovernorBundleP,
committee: committeeBundleP,
binaryVoteCounter: voteCounterBundleP,
};
const bundles = await Collect.allValues(governanceBundlePs);
});
const bundles = await allValues(governanceBundlePs);
produce.governanceBundles.resolve(bundles);

return { produce, consume };
Expand Down Expand Up @@ -122,7 +122,7 @@ export const setupAmmServices = async (
]);

const agoricNames = consume.agoricNames;
const installs = await Collect.allValues({
const installs = await allValues({
amm: E(agoricNames).lookup('installation', 'amm'),
governor: E(agoricNames).lookup('installation', 'contractGovernor'),
electorate: E(agoricNames).lookup('installation', 'committee'),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import {
} from '../../src/vaultFactory/params.js';
import { startVaultFactory } from '../../src/econ-behaviors.js';
import '../../src/vaultFactory/types.js';
import { Collect } from '../../src/collect.js';
import { allValues } from '../../src/collect.js';

const contractRoots = {
faucet: './faucet.js',
Expand Down Expand Up @@ -252,14 +252,14 @@ async function setupServices(
produce.priceAuthority.resolve(priceAuthority);

produce.feeMintAccess.resolve(feeMintAccess);
const vaultBundles = await Collect.allValues({
const vaultBundles = await allValues({
VaultFactory: bundlePs.VaultFactory,
liquidate: bundlePs.liquidate,
});
produce.vaultBundles.resolve(vaultBundles);
await startVaultFactory({ consume, produce }, { loanParams: loanTiming });
const agoricNames = consume.agoricNames;
const installs = await Collect.allValues({
const installs = await allValues({
vaultFactory: E(agoricNames).lookup('installation', 'VaultFactory'),
liquidate: E(agoricNames).lookup('installation', 'liquidate'),
});
Expand Down
5 changes: 3 additions & 2 deletions packages/zoe/src/objArrayConversion.js
Original file line number Diff line number Diff line change
Expand Up @@ -62,18 +62,19 @@ export const assertSubset = (whole, part) => {
* value.
* * No matter whether the original property was an accessor, writable,
* or configurable, all the properties of the returned object will be
* writable, configurable, data properties.
* non-writable, non-configurable, own data properties.
* * No matter what the original object may have inherited from, and
* no matter whether it was a special kind of object such as an array,
* the returned object will always be a plain object inheriting directly
* from `Object.prototype` and whose state is only these new mapped
* own properties.
* * The new object is hardened.
* * The caller-provided mapping function can map the entry to a new entry
* with a different key, in which case the new object will have those
* keys as its property names rather than the original's.
*
* Typical usage will be to apply `objectMap` to a pass-by-copy record, i.e.,
* and object for which `passStyleOf(original) === 'copyRecord'`. For these,
* an object for which `passStyleOf(original) === 'copyRecord'`. For these,
* none of these edge cases arise. If the mapping does not introduce
* symbol-named properties, then the result, once hardened, will also be a
* pass-by-copy record.
Expand Down