| fip | title | author | discussions-to | status | type | category | created | requires |
|---|---|---|---|---|---|---|---|---|
0045 |
De-couple verified registry from markets |
Alex North (@anorth), Zenground0 (@zenground0) |
Draft |
Technical |
Core |
2022-08-12 |
FIP-0034 |
Represents Filecoin Plus DataCap allocations separately from the built-in market actor. Adds explicit term (duration) limits to DataCap allocations. Simplifies the quality-adjusted power calculation to remove power spreading out over a sector's lifetime. Supports extending the terms of DataCap allocations independently of market storage deals. Represents DataCap as a fungible token. These changes provide a foundation for user-programmed market actors to broker Filecoin Plus verified deals in the future.
The current implementation of Filecoin Plus verified deals is deeply coupled to the built-in market actor, which is treated as trusted code. The built-in market is inefficient, limited in functionality and can only improve very slowly through network upgrades. The architectural coupling prevents any user-programmed actors from replicating, improving, or varying deal-related functionality, especially with any relation to Filecoin Plus.
This proposal decouples the representation of Filecoin Plus DataCap allocations from the built-in market actor's deals, and the process of claiming it from the built-in market's processes. The built-in market actor remains a delegate, acting on behalf of verified clients, but only in a way that will be accessible to user-programmed market actors too. This removes some of the necessary trust vested in the built-in market actor's code.
Along the way, this proposal implements the ability to extend DataCap terms beyond the market's deal terms. It also simplifies the quality adjusted power calculation so as to support the future ability to update the data in a sector regardless of its previous content. This proposal gets half-way toward full support for user-programmed market actors.
The core motivation behind this proposal is to enable the development of markets and market-like actors on the FVM, once user-programability is supported. All negotiation between data clients and storage providers is currently limited by the built-in market actor's limited abilities, and improvements to it are thus limited to FIPs and network upgrades. Features like deal extension, multi-sector deals, re-negotiation, deal transfer, capacity deals, auction markets, repair markets, insurance, derivatives etc. should all be possible in user-programmed actors. None should require a network-wide upgrade to realise.
The built-in market actor currently occupies an exclusive position as the sole broker of Filecoin Plus DataCap, tightly coupled with the verified registry and storage miner actors. In order to enable the development of useful storage-related applications and features, we need to break the tight coupling between the built-in actors, expose composable on-chain primitives, and make the capabilities of the built-in market actor available to user-programmable contracts. This proposal addresses the capability to broker of Filecoin Plus DataCap.
Breaking coupling between the built-in market actor, verified registry actor, and miner actors will eventually remove the storage market from consensus-critical risk surface, and move subsequent market development into the realm of smart-contract upgrades, rather than network-wide consensus protocol upgrades. Minimising coupling is a key principle in software and systems engineering. Decoupling the verified data rewards from storage deals provides great opportunities for simplifying Filecoin’s core mechanics in order to provide a better platform for development.
One benefit we can immediately realise is simplifying the calculation of quality-adjusted power. The current mechanism, which is coupled to deal terms, imposes significant limits on the flexibility of sector storage, essentially rendering it write-once. Decoupling these neatly resolves the calculation difficulties to give full flexibility, and is much simpler than the Filecoin Plus premium proposal that also attempted this.
Terminology
- Allocation: DataCap allocated by a client to a specific piece of data.
- Claim: a provider's assertion they are storing all or part of an allocation
- Term: period of time for which a DataCap allocation or claim is valid or active.
The Filecoin Plus verified registry actor is extended to record allocations of DataCap to specific pieces of data to be stored by a provider. An allocation is independent of any deal with the built-in storage market (or any future market).
struct Allocation {
// The verified client which allocated the DataCap.
Client: ActorID
// The provider (miner actor) which may claim the allocation.
Provider: ActorID
// Identifier of the data to be committed.
Data: CID
// The (padded) size of data.
Size: uint64
// The minimum duration which the provider must commit to storing the piece to avoid
// early-termination penalties (epochs).
TermMinimum: int64
// The maximum period for which a provider can earn quality-adjusted power
// for the piece (epochs).
TermMaximum: int64
// The latest epoch by which a provider must commit data before the allocation expires.
Expiration: int64
// Whether the provider can commit the data in multiple pieces.
// This is initially un-used, but supports allocations larger than a full sector in the future.
AllowRanges: bool
}
An allocation specifies a range of terms for which the provider may commit the data, between some minimum and maximum. An allocation's maximum term must be at least as large as its minimum term. Due to the implementation of term enforcement (details below), clients should leave a large buffer between the minimum and maximum term to make the allocation practical for a provider.
Network policy limits the maximum value for an allocation's expiration, so that an allocation cannot occupy non-useful DataCap indefinitely.
Parameters
MinimumVerifiedAllocationTerm: The minimum allowed value forTermMinimumis 6 months.MaximumVerifiedAllocationTerm: The maximum allowed value forTermMaximumis 5 years (can be increased by a future FIP).MaximumVerifiedAllocationExpiration: The maximum difference betweenExpirationand the epoch at which an allocation is created is 60 days.
Allocations are stored in the verified registry actor's state, grouped by client address. Nesting by client promotes more efficient lookups at scale, and supports per-client iteration for revocation.
struct State {
// ... Existing verified registry state ...
// Allocations indexed by client, then by ID.
Allocations: HAMT[Address]HAMT[AllocationID]Allocation
// Sequence number for allocation IDs.
// The value `0` is reserved for "no allocation"
NextAllocationId: AllocationID // uint64
}
An allocation is created by a verified client by transferring DataCap tokens (see below)
to the verified registry.
The Allocation metadata must be provided as transfer metadata.
The verified registry checks the same preconditions as the existing UseBytes method.
The size of the allocation must match the number of tokens received.
An allocation may be removed after its expiration epoch has passed (by anyone).
When removed, the DataCap tokens are transferred back to the client.
The verified registry provides a new method to process removals, replacing RestoreBytes.
struct RemoveExpiredAllocationsParams {
Client: ActorID // client to clean up (need not be the caller)
AllocationIDs: []AllocationID // empty for "all", or specify a set
}
struct RemoveExpiredAllocationsResult {
// Ids of the allocations that were either specified by the caller or discovered to be expired.
Considered: []AllocationID
// Results for each processed allocation.
Results: {
// Count of successful claims.
SuccessCount: uint64
// Index and failure code for each unsuccessful claim.
Failures: []{index: uint64, code: ExitCode}
},
// The amount of datacap reclaimed for the client.
DatacapRecovered: DataCap
}
// Removes all expired allocations for a client.
// The DataCap for cleaned-up allocations is returned to the client.
// Note that un-expired allocations cannot be removed.
fn RemoveExpiredAllocations(params: RemoveExpiredAllocationsParams) -> RemoveExpiredAllocationsResult
Allocations are also removed when claimed by a provider (see below).
The existing RemoveVerifiedClientDataCap method is changed to invoke a
privileged method of the DataCap token actor to burn DataCap tokens held on a client's balance.
Only the verified registry actor is permitted to invoke this method.
The verified registry actor is further extended to record each storage provider's commitment to a DataCap allocation, called a claim. A claim represents a provider's obligation to store a piece of data, and corresponding benefit of incentivized storage power.
struct Claim {
// The provider storing the data (from allocation).
Provider: ActorID
// The client which allocated the DataCap (from allocation).
Client: ActorID
// Identifier of the data committed (from allocation).
Data: CID
// The (padded) size of data (from allocation).
Size: uint64
// The minimum period which the provider must commit to storing the piece (from allocation).
TermMinimum: int64
// The maximum period for which the provider can earn QA-power for the piece (from allocation).
TermMaximum: int64
// The epoch at which the (first range of the) piece was committed.
TermStart: int64
// ID of the provider's sector in which the data is committed.
Sector uint64
}
Claims are stored in the verified registry actor state, grouped by provider address. A claim's ID is inherited from the allocation that created it.
struct State {
// ... Existing verified registry state ...
// Claims by provider ID, then Claim ID.
Claims HAMT[Address]HAMT[ClaimID]Claim
}
An allocation is claimed when a storage miner actor proves the corresponding piece of data is committed to a sector. Upon converting an allocation to a claim, the verified registry actor burns the associated data cap tokens from its balance.
struct SectorAllocationClaim {
// Client of the allocation being claimed.
Client: ActorID
// The allocation being claimed.
AllocationID: AllocationID
// The piece data committed (either the allocations data, or a part of it).
PieceCID: CID
// The (padded) size of the data committed (up to the allocation's data size).
PieceSize: uint64
// The sector into which the piece has been committed.
SectorID: uint64
// The epoch during which the sector is scheduled to expire.
SectorExpiration: int64
}
struct ClaimAllocationsParams {
Sectors: []SectorAllocationClaim
AllOrNothing: bool
}
struct SectorClaimResult {
BatchInfo: {
// Count of successful claims.
SuccessCount: uint64
// Index and failure code for each unsuccessful claim.
Failures: []{index: uint64, code: ExitCode}
}
ClaimedSpace: BigInt
}
// Called by storage miner actor to claim allocations for data provably committed to storage.
// For each allocation claim, the registry checks that the provided piece CID
// and size match that of the allocation.
fn ClaimAllocations(params: ClaimAllocationsParams) -> ClaimAllocationsResult
The record of a claim may be removed after the term maximum has elapsed.
struct RemoveExpiredClaimsParams {
Provider: ActorID // provider to clean up (need not be the caller)
ClaimIDs: []ClaimID // empty for "all", or specify a set
}
struct RemoveExpiredClaimsResult {
// Ids of the claims that were either specified by the caller or discovered to be expired.
Considered: []ClaimID
// Results for each processed claim.
Results: {
// Count of successful claims.
SuccessCount: uint64
// Index and failure code for each unsuccessful claim.
Failures: []{index: uint64, code: ExitCode}
}
}
// Removes all (or specified) expired claims for a provider.
// Claims which have not yet passed their term maximum cannot be removed.
fn RemoveExpiredClaims(params: RemoveExpiredClaimsParams) -> RemoveExpiredClaimsResult
A client of a claim may increase its maximum term up to MaximumVerifiedAllocationTerm.
struct ClaimTerm {
Provider: ActorID // needed to find claim in state
ClaimID: ClaimID
TermMaximum: int64 // new duration in epochs from TermStart
}
struct ExtendClaimTermsParams{
Claims: []ClaimTerm
}
struct ExtendClaimTermsResult {
// Count of successful extensions.
SuccessCount: uint64
// Index and failure code for each unsuccessful extension.
Failures: []{index: uint64, code: ExitCode}
}
// Called by allocation client to extend the maximum term of their allocations,
// without consuming DataCap.
// The new TermMaximum must be no less than the current one,
// and no greater than MaximumVerifiedAllocationTerm.
fn ExtendClaimTerms(params: ExtendClaimTermsParams) -> ExtendClaimTermsResult
A verified client can extend the term for a claim beyond the initial maximum term by
spending new DataCap.
The claim's term maximum can be extended up to MaximumVerifiedAllocationTerm beyond the current epoch.
The client extending the claim need not be the one that made the original allocation.
This is similar to issuing a new allocation/claim for the same data,
but avoids overheads of re-sealing.
The VerifiedClients map of addreses to balances is removed from the verified registry state.
The balances of DataCap tokens held by verified clients are extracted to a DataCap token actor.
This actor represents un-allocated DataCap as a fungible token.
This token actor is intended to implement the FRC-0046 token standard in the future,
but specification of the full, public API is deferred to a subsequent proposal.
The DataCap token actor should be retroactively updated to support the standard later.
Token balances are represented as BigIntegers with 18 decimal places of precision,
a representation which is likely to be adopted by future token standards.
Token quantities are constrained to be whole multiples of 10^18.
One whole DataCap token corresponds to one unit of DataCap to be allocated,
which today is good for one byte of data.
The DataCap token is generally non-transferable, except to or from the verified registry actor.
The token actor enforces these transfer restrictions.
The verified registry actor can invoke privileged methods to create and destroy tokens.
FRC-0046 supports delegation of control of token balances to specified third-party actors as "operators". The built-in storage market actor is pre-authorized as an operator for all verified clients. Thus, the built-in market actor can allocate DataCap on clients' behalf. This delegation mechanism will support user-programmed market actors dealing in DataCap in the future.
This token delegation mechanism allows this proposal to be implemented without changing any clients' or providers' workflows for existing functionality. Parties will continue to interact with the built-in market actor and verified registry actor directly.
The built-in account and multisig actors are changed to implement a receiver hook method, as specified by FRC-0046. This hook always succeeds, without inspecting the payload.
The verified registry's AddVerifiedClient method is changed to invoke the DataCap token actor to
mint new tokens to the client's balance.
The verified registry's UseBytes method is removed.
To create an allocation, a verified client transfers DataCap tokens to the verified registry actor.
specifying one or more AllocationRequest records in operator data attached to the transfer (below).
The verified registry's RestoreBytes method is also removed.
Instead, the RevokeAllocations method on the verified registry actor transfers any released
DataCap tokens back to the client's balance for re-use.
The verified registry actor implements a token receiver hook method, invoked by the datacap token actor upon any transfer of tokens to the verified registry actor.
The hook payload must specify the creation of allocations or extension of claims corresponding to
the amount of datacap tokens received by the verified registry actor in the transfer.
The registry validates allocations and extensions, and rejects the transfer if any are invalid.
The verified registry checks the same preconditions as the existing UseBytes method.
The registry rejects the transfer if the sum of sizes does not match the amount of datacap received.
// A request to create an allocation with datacap tokens.
// Field semantics match the Allocation structure.
struct AllocationRequest {
// The provider (miner actor) which may claim the allocation.
// (Historical note: network version 17 introduced this type with an Address for this field but
// it was switched to an ActorID for network version 18 and later.)
Provider: ActorID
// Identifier of the data to be committed.
Data: Cid
// The (padded) size of data.
Size: PaddedPieceSize
// The minimum duration which the provider must commit to storing the piece to avoid
// early-termination penalties (epochs).
TermMin: int64
// The maximum period for which a provider can earn quality-adjusted power
// for the piece (epochs).
TermMax: int64
// The latest epoch by which a provider must commit data before the allocation expires.
Expiration: int64
}
// A request to extend the term of an existing claim with datacap tokens.
struct ClaimExtensionRequest {
// The provider (miner actor) which may claim the allocation.
// (Historical note: network version 17 introduced this type with an Address for this field but
// it was switched to an ActorID for network version 18 and later.)
Provider: ActorID
// Identifier of the claim to be extended.
Claim: ClaimID
// The new maximum period for which a provider can earn quality-adjusted power
// for the piece (epochs).
TermMax: int64
}
// Operator-data payload for a datacap token transfer receiver hook.
// The implied client is the sender of the datacap.
struct AllocationRequests {
AllocationRequests: []AllocationRequest,
ClaimExtensionRequests: []ClaimExtensionRequest,
}
// Return value from datacap token transfer receiver hook.
struct AllocationsResponse {
// Result for each allocation request.
AllocationResults: {
SuccessCount: uint64
Failures: []{index: uint64, code: ExitCode}
}
// Result for each extension request.
ExtensionResult: {
SuccessCount: uint64
Failures: []{index: uint64, code: ExitCode}
}
// IDs of new allocations created.
NewAllocations: []AllocationID
}
// Payload accompanying the receiver hook for a FRC46 token.
// (Copied from FRC-46).
struct FRC46TokenReceived {
// The address from which tokens were debited
From: Address,
// The address to which tokens were credited (which will match the hook receiver)
To: Address,
// The actor which initiated the mint or transfer (i.e. the market)
Operator: Address,
// The quantity of tokens received; non-negative
Amount: TokenAmount,
// Serialized AllocationRequests record
OperatorData: Bytes,
// Ignored for FIP-0045.
TokenData: Bytes,
}
A DataCap allocation/claim is valid for a range of commitment terms, between a client-specified minimum and maximum, so long as the allocation is claimed before the allocation expires. The actual term for a claim begins at the epoch the data is committed into a sector, proven either with proof-of-replication or replica-update. There is no distinct "deal start epoch". The actual term for a claim ends the epoch that the sector containing the data expires, or that data is replaced.
A sector's DealWeight and VerifiedDealWeight are calculated as simple sums of the sizes
of the deals, with no multiplication by duration.
The miner actor computes a sector’s power as SectorSize + (9 * VerifedDealWeight).
A sector containing a DataCap claim is thus immediately eligible for a power multiplier of 10
for that fraction of the sector's capacity occupied by VerifiedDealWeight
(subject to the existing requirement to initially prove the sector in Window PoSt).
A claim term cannot end during a sector's lifetime, except by explicit replacement of data. If data is replaced, the sector's verified deal weight is reduced and it immediately loses the power multiplier for that fraction of the sector's capacity. (Replacing non-zero data in a sector is not possible today).
Aligning the claim's term with the sector's expiration exactly in this way removes the "spreading-out" of QA power over a sector's life in the current method of computation. Providers gain 10x power for each byte of verified data exactly while that data is proven.
A sector's pledge requirement is proportional to its share of power, but never decreases, just as today. When the minimum term for a verified piece expires, the pledge requirement does not change. The provider's commitment to storing the verified piece extends until the sector's scheduled expiration. If the sector is terminated early, the termination penalty is based on its boosted power.
If a verified piece is transferred to new sector (see below), the new initial pledge for the new sector is calculated using the network conditions at the time, including the sector's new power after adding the verified piece. The pledge requirement for the old sector is not decreased, but any subsequent penalty is calculated using its current (i.e. reduced) power.
A new field SimpleQAPower: bool is added to SectorOnChainInfo.
During the migration activating this proposal, SimpleQAPower is set for each sector:
trueif the sector hasVerifiedDealWeight = 0andDealWeight = 0,falseotherwise
All sectors committed after this proposal is activated take SimpleQAPower = true.
The value of SimpleQAPower never changes for an individual sector.
A sector with SimpleQAPower = true has its deal weights calculated as simple sums of deal size,
and QA power calculated according to the simple method described above, with no spreading out effect.
A sector with SimpleQAPower = false continues to use the existing QA-power calculation.
Such a sector cannot claim any allocations.
Such a sector may be extended, but, as today, this will result in its power being diluted
to "spread out" the verified reward over the sector's new commitment duration.
The SimpleQAPower field will become redundant and may be removed by a network upgrade once
all sectors with the value false have expired.
The built-in miner actor enforces the term and the continual storage of the data.
The miner actor requires that the sector into which a claimed piece is committed must have a remaining lifespan that is sufficient to fulfil the minimum term. Thus, when a sector terminates at its scheduled expiration epoch, the claim's term has been met.
Similarly, the miner actor requires that sector into which the piece is committed must have a remaining lifespan no longer than that which would reach the maximum term. That is, a sector's commitment expiration epoch must fall between the minimum and maximum terms of any verified claim it holds.
The longest practical minimum term for a verified DataCap allocation is the larger of the maximum initial sector commitment duration, and maximum sector extension duration (which today are identical).
There is no change to termination penalty calculations. If a sector is terminated early (continual faults, or manual termination), the provider will pay a penalty related to the expected reward earned by the sector (up to some limit).
A sector's scheduled commitment term cannot be extended beyond the maximum term of any claim attributed to it. Thus, to extend a sector, a provider must simultaneously drop any verified claims with terms that would be exceeded by the extended commitment. Dropping a verified claim reduces the sector's power accordingly, as well as reducing penalties to be paid on any future fault to reflect the new, lower power. Dropping a claim does not reduce the sector’s pledge requirement.
This facility to drop a claim while extending a sector's commitment can only be exercised:
- for claims that have passed their minimum term,
- for non-faulty sectors, and
- in the final 30 days of the sector's currently committed lifespan
These restrictions prevent a provider using extension as a means of escaping their commitment to store the claimed piece for the sector’s full originally-committed lifetime.
The storage provider operator must provide the IDs of the allocations claimed by the sector when extending.
// The verified data claims currently supported by a sector, to be maintained or dropped
// while processing extension.
// All claims currently supported by the sector must be provided.
struct SectorClaims {
Sector: uint64
MaintainClaims: []ClaimID
DropClaims: []ClaimID
}
struct ExpirationExtension2 {
Deadline: uint64
Partition: uint64
Sectors: BitField // IDs of non-FIL+ sectors (must have zero verified deal weight)
SectorsWithClaims: []SectorClaims
NewExpiration: int64 // epoch
}
struct ExtendSectorExpirationParams2 {
Extensions: []ExpirationExtension2
}
struct ExtendSectorExpirationResult2 {
// Count of successful extensions.
SuccessCount: uint64
// Index and failure code for each unsuccessful extension.
Failures: []{index: uint64, code: ExitCode}
}
// Increases the expiration epoch for some sectors, optionally dropping expired claims.
// For each sector, the provided claim IDs must correspond to claims that refer back to the
// sector, and the sum of their size must equal the sector's verified deal weight.
// The new expiration must be prior to the earliest term maximum for any maintained claims.
fn ExtendSectorExpiration2(params: ExtendSectorExpirationParams2) -> ExtendSectorExpirationResult2
This proposal makes no changes to sector termination. When a sector terminates at the end of its scheduled commitment, no specific action is necessary. When a sector terminates before its scheduled commitment is complete, the provider pays penalties according to the sector's power at that time.
The verified registry actor will continue to hold Claim records for terminated sectors
until the provider removes them with RemoveExpiredClaims.
The workflows described above are independent of any market actor, and take place through interactions by the client and provider with the verified registry actor directly.
However, in order to maintain current client and provider workflows, the built-in market actor can act as a delegate.
The built-in storage market actor is pre-authorized as a DataCap token delegate for all clients.
When a deal proposal is published with VerifiedDeal=true, the market actor transfers
the corresponding DataCap tokens from the client to the verified registry actor.
This transfer is accompanied by an Allocation record populated with data from the deal.
The built-in market specifies an allocation's minimum term to be equal to the deal duration,
and maximum term to be some amount greater than the deal duration.
Parameters
MarketDefaultMaximumAllocationTerm: The built-in market calculates a term maximum of 90 days greater than a deal's specified duration.
let dealAllocation = Allocation {
Client: deal.Client
Provider: deal.Provider
Data: deal.PieceCID
Size: deal.PieceSize
TermMinimum: deal.EndEpoch - deal.StartEpoch
TermMaximum: deal.EndEpoch - deal.StartEpoch + MarketDefaultMaximumAllocationTerm
Expiration: min(deal.StartEpoch, CURRENT_EPOCH + MaximumVerifiedAllocationExpiration)
AllowRanges: false
}
The market will accept verified deals with a StartEpoch beyond the maximum allowed allocation Expiration.
If such a deal is not activated before the allocation expires, it will not confer QA power.
However, the deal may still be activated and sector commitment should proceed (without QA power).
The market stores allocation IDs for not-yet-activated deals in a new mapping, and the claim ID for activated deals in deal state.
struct State {
// Remainder unchanged...
// Allocation IDs for deals that are not yet activated.
PendingDealAllocationIDs: CID // HAMT[DealID]AllocationID
}
struct DealState {
// Existing state
SectorStartEpoch: int64
LastUpdatedEpoch: int64
SlashEpoch: int64
// New
ClaimID: uint64
}
When a verified deal expires without being committed, the DataCap allocation remains. The market does not revoke the allocation during cron handling. Revocation is a manual operation that the client must perform.
When a verified deal is activated by the provider, the market actor returns
the allocation ID and piece metadata to the storage miner actor.
The miner actor then invokes ClaimAllocations and, if successful,
computes quality adjusted power for the sector according to the piece size and QA power multiplier.
A network state migration is required to implement this proposal.
That migration must:
- Implement the new verified registry state schema for allocations and claims,
and set
NextAllocationId = 1. - Migrate the verified registry client balances map to DataCap token balances, with appropriate unit conversion.
- Add a new field
SimpleQAPowerto eachSectorOnChainInfo, taking the valueDealWeight == 0 && VerifiedDealWeight == 0 - Add the pending allocation mapping to market state, and claim ID field to deal states (all
0) - For each pending (not yet activated) deal with
Verified = true, create an Allocation- The maximum term for such allocations is 540 days, in order to support any possible sector duration
- The expiration for such allocations is set to
MaximumVerifiedAllocationExpirationafter the migration epoch, which may be before the deal's start date.
The design rationale is mostly provided by the proposal's motivation. See also #298.
The rationale behind the new flows is to remove system trust in the built-in market actors. This proposal removes trust of the verified registry in the built-in market, by moving network-critical state into the verified registry actor state directly.
Data cap is presented as a fungible token in order to support greater programmability in the future, such as through automated notary actors. Many tools and libraries will emerge to support fungible tokens generally, and conforming to the same standard will allow DataCap to enjoy their transparent support.
This proposal changes the behaviour and internal APIs of several built-in actors, and thus requires a network upgrade. The proposal also changes state schema of several built-in actors, which requires a state migration at that upgrade. Off-chain tools that directly inspect actor state may need to change to handle the post-migration schema.
This proposal leaves all existing externally-invoked actor methods intact and functional. This is acheived through the built-in market actor acting as a delegate for DataCap, without needing to be trusted by the verified registry. Deal clients and providers are not expected to need to change their operations.
Sectors containing verified data claims that are either committed after this upgrade,
and sectors which are migrated to verified data claims,
can only have their lifetimes extended via the new ExtendSectorExpiration2 method specified by this proposal.
Storage providers must invoke this new method in order to extend such sectors.
Sectors with no verified claims, including existing sectors which are not migrated,
can continue to use the existing ExtendSectorExpiration method.
To be provided with implementation, prior to acceptance.
This proposal does not alter security properties of the Filecoin network. This proposal does increase the value of Filecoin Plus data cap by extending its term longer than 1.5 years. This may increase demands on Filecoin Plus governance to ensure appropriate allocations.
This proposal increases the value of Filecoin Plus data cap to storage providers. A storage provider will be able to earn a slightly greater reward per byte (due to reduced spreading-out), and maintain that reward level for up to 5 years. Since the QA power associated with a data cap allocation may be active for a longer period, more of the network's total block reward will be directed toward data cap claim providers than if the status quo were continued.
Where an allocation is extended beyond 1.5 years, this proposal will increase the incentive to the provider to extend the sector storing the associated data until the allocation's maximum term is reached.
Since new allocations are more valuable than existing ones, this proposal may incentivise deal-focussed but capital- or capacity-constrained providers to let existing sectors expires in order to re-use the associated pledge and hardware for newer allocations.
Storage providers may prefer deals associated with long data cap allocation terms to those with shorter expected terms. Clients specifying short maximum terms may find less supply of storage for their deals. This bias is expected to decrease once repeated replica updates are supported for a single sector ("re-snapping"). The 10x reward even for short terms is expected to remain much more profitable for providers than CC sectors.
Once it is possible for clients to specify allocation term ranges directly (rather than via the built-in market's defaults), clients specifying very narrow term ranges for their allocations may find less supply of storage, due to the difficulty of packing such deals into sectors with other deals.
Most of the product considerations of this proposal are directed towards the future product possibilities toward which this proposal works.
This proposal breaks the trust relationship between the verified registry actor (a core network component), and the built-in market actor. This is a step towards removing all network trust in market actors, at which point we can support user-programmed market-like actors on the FVM. Such markets could implement a wide range of marketplace models and rules, and be significantly more efficient. The remaining step toward this goal is to remove the miner actor's trust of the built-in market actor.
After this trust is fully removed, the built-in market actor could transition to an alternative governance model, decoupled from network upgrades.
After the built-in market actor is decoupled from the miner actor, a client will be able to allocate DataCap to a piece of data directly, without necessarily using the built-in or any market actor. Such an allocation would represent the simplest, unconditional deal for verified data possible. This direct route would likely be far more gas-efficient than the built-in market actor.
The "market as DataCap delegate" pattern enables market actors to implement arbitrary deal logic, and make the allocation of FIL+ DataCap conditional upon the provider's acceptance of deal terms. For example, a market could specify a negative deal price (i.e. the provider pays the client), and allocte the DataCap only after the provider has escrowed the necessary payment. Or a market could require simultaneous acceptance of a retrieval assurance policy as part of deal terms, again withholding the DataCap allocation until on-chain conditions are met. Filecoin Plus verified clients will be able to choose markets that implement policies they want, which may extend far beyond what's reasonable to implement in the built-in market actor.
The verified registry actor and its allocations and claims are not coupled to sector size. A future iteration to this schema could support allocations which are larger than a single sector, and may be claimed by a collection of data commitments spread across multiple sectors.
Such an allocation would be far more gas efficient than the multiple per-sector deals required for a large data set today.
The pattern of proxying verified registry interactions through the built-in market actor will initially raise the gas cost of publishing deals, and of committing a sector with verified claims, but this is a transitional state. This proposal is a stepping stone toward much greater potential for gas efficiency after the final decoupling from the miner actor.
- For simple, unconditional verified data allocations, with no additional payment, no deal with a market will be necessary at all. A client need only make an allocation directly with the verified registry. This direct allocation will be much cheaper to make and maintain than a deal.
- Upgrades to the claim schema could support DataCap allocations larger than a single sector. As well as improvements in utility, such allocations would be far more gas efficient than negotiating a separate deal for each sector-sized unit.
- Decoupling FIL+ from markets opens the potential to develop alternative storage markets, including those far more efficient than the built-in one.
By opening the field of competition for market-like actors, we can expect deal costs to decrease over time, while the network enforces just enough state to administer the quality-adjusted power of verified data.
PublishStorageDeals messages that include verified deals will see a large increase in gas usage in this FIP. The increase is about 70-80M gas per deal. This will be drastically reduced in the future by:
-
Aggregating datacap token transfers to the verifreg actor
-
Removing the need for the market actor to be involved at all.
1 deal - Old Gas: ~110M New Gas: ~200M Increase: ~80%
8 deals - Old Gas: ~370M New Gas: ~970M Increase: ~260%
20 deals Old Gas: ~800M New Gas: ~2250M Increase: ~280%
Prove Commit Aggregate calls with no verified deals will see no change. Calls with verified deals will see around a 75% increase.
9 sectors, no verified deals Old Gas: ~280M New Gas: ~284M Increase: ~1%
11 sectors with verified deals Old Gas: ~750M New Gas: ~1400M Increase: ~85%
20 sectors with verified deals Old Gas: ~1900M New Gas: ~3200M Increase: ~65%
Each call to Prove replica Update with verified deals will see a ~30% increase in gas usage.
1 verified deal Old Gas: ~180M New Gas: ~230M Increase: 30%
Deal activation for prove commit it handled in cron, so SPs will not see an increase in gas costs for Prove Commit. This will however add an extra ~50M gas to cron execution, which we may want to compensate for in a future update.
Storage providers choose how to arrange deals into sectors, subject to the constraints on size and start epoch. They may optimise for various goals, such as filling space, or space-time, or latency to commit a sector. This proposal introduces a new constraint of an allocation's maximum term, and removes a possible optimisation target of space-time.
Each allocation constrains the sector's commitment duration to lie between its minimum and maximum term. Two DataCap allocations may only be sealed into the same sector if there is some overlap between the ranges specified by their minimum and maximum term. Storage providers' deal packing algorithms must be updated to respect these constraints.
Prior to this change, co-locating deals with very different durations is merely an inefficient use of space-time, which diluted and delayed a provider's power and rewards. After this change, some allocation will be incompatible, but every compatible collection of allocations will then grant maximal per-byte quality-adjusted power for the sector's full lifetime.
Implementation is in progress on the decouple-fil+ branch of the built-in actors repository:
https://github.com/filecoin-project/builtin-actors/tree/decouple-fil+.
This FIP was originally accepted with "Address" in most places where "ActorID" is now used within the types in this document. However, these were implemented and launched in network version 17 as ActorID.
Both ClaimExtensionRequest and AllocationRequest were originally defined with an Address field
for the provider and this remained in the implementation for network version 17. However, this was
changed to ActorID for network version 18 and later.
Copyright and related rights waived via CC0.