[EPIC] Lotus Miner Roadmap 2023

[rjan90]

Note
This is a project issue for the Lotus Miner 2023 roadmap, milestones and tracking issues. This issue will be updated and are subject to changes and refinements over time.

Table of Contents

• Context
• Current Architecture
• Roadmap
  • 🌌 2023 Q1 - Miner issue backlog sprint
  • 🌌 2023 Q2 - Aggregation/Batching sprint
  • 🌌 2023 Q2 - HA of WindowPoSt Technical Design
  • 🌌 2023 Q2/Q3 - Unsealing Enhancements
  • 🌌 2023 Q3 - Implement nv21 FIPs
  • 🌌 2023 Q3 - Implement Supranational PC2/C2 sealing code
  • 🌌 2023 Q3/Q4 - High Availabilty of WindowPoSt
  • 🌌 2023 Q4 / 2024 Q1 - High Availabilty of WinningPoSt
  • 🌌 2023 Q4 / 2024 Q1 - Multi-MinerID support in Lotus-Provider
• 2024 - Next Steps
• Contact us

Context

In late 2022 the Lotus team wanted to understand what current and aspiring enterprising Storage Providers (SPs) pain points where with the Lotus-Miner software, and what features we could build to offer our users stability in their operations, and give them confidence to further grow their businesses using what is currently called the Lotus-Miner.

We gathered all the high-level problem areas we identified, and potential solutions we had looked into in the Architectural/System-level problems for enterprise-level Storage Providers post. We encouraged everyone to provide their feedback, concerns, prioritization and other ideas they might have to make the Lotus-Miner experience better. We got a tremendous amount of great ideas and feedback, both asynchronous, and during Storage Provider Working Group (SPWG) calls and the Decentralized Storage Alliance (DSA) calls.

Based on this community feedback, we have created a roadmap that aims to address pain points and build features that have the largest overall impact for both storage providers and ecosystem collaborators. We believe that this roadmap will help ease the pain points that have been identified and lead to a better experience for our awesome users. 💙

Current Lotus-Miner Architecture

Roadmap

🌌 2023 Q1 - Lotus-Miner issue backlog sprint

User story:

As a storage provider, I want the development team to prioritize fixing top bugs. This is because these bugs are causing operational instability and requiring extra work hours to fix. By addressing these bugs, I can reduce the time spent on debugging and fixing general issues, allowing me to focus more on providing quality storage solutions to our customers.

Impact:

• Reduced downtime and lower penalties due to fewer windowPoSt failures. This is achieved by fixing instability and edge-case bugs, which also reduces the gas paid for recovery.
• Improved general performance by fixing bugs in the sealing pipeline, a critical part of how storage providers are onboarding data.
• Enhanced user experience with more human-readable outputs and workflows.
• Reduced user confusion by differentiating similar looking commands in the lotus-worker binary.

Milestone

Preview Give feedback

1. Lotus-Miner Backlog Sprint #10338
   12 of 13
   +0

---

🌌 2023 Q2 - Aggregation/Batching sprint

User story:

As a storage provider on the Filecoin network, I want the lotus-miner team to focus on fixing bugs in the current aggregation and batching logic. This need arose in late Q1 when the network experienced a more stable higher base fee than previous periods. This change uncovered bugs that led to failed batches and aggregation messages because the gas used in these messages exceeded the block limit. By addressing these issues, we can prevent such failures and improve the stability and efficiency of the network.

Impact:

• Improved stability: By fixing the bugs in the aggregation and batching logic, storage providers need to spend less time on debugging failed batching and aggregation messages.
• Increased efficiency: With the bugs fixed, the aggregation and batching processes should run more smoothly, reducing the gas used.
• Enhanced user confidence: By addressing these issues promptly, we demonstrate our commitment to address bugs that arise from changes in the network, which can increase user confidence and satisfaction.

Milestone

Preview Give feedback

1. feat: sealing: Split PCA/PCB batches if gas used exceeds block limit #10647
   +0
2. Set default for MaxSectorProveCommitsSubmittedPerEpoch #10728
   +0
3. Add feature to stagger sector prove commit submission #10543
   +0

---

🌌 2023 Q2 - V2 Technical Design

User story:

As a storage provider, I want a High Availability (HA) solution for WindowPoSt/WinningPoSt. This is because software upgrades around network upgrades and Lotus releases often cause failures in WindowPoSt and downtime, impacting my earnings.

Impact:

• Enhanced Understanding: Provides a clear view into the technical components that are part of the new Lotus-miner/Storage Provider (SP) stack design. This transparency can help users better understand the system's workings and potential improvements.
• Blueprint for Future Enhancements: The technical design document developed during this process will serve as a comprehensive blueprint for future enhancements. This will streamline the implementation of additional features and improvements, making the SP stack more adaptable and efficient in the long run.

Milestone

Preview Give feedback

1. HA of WindowPoSt - Techincal Design #10379
   2 of 2
   +0
   

---

🌌 2023 Q2/Q3 - Unsealing Enhancements

User story:

• As a storage provider (SP) who stores sealed sectors, I need an efficient and reliable unsealing process which does not require manual work to attach workers to where the sealed sector is located. This will enable me to handle a large volume of sealed sectors being unsealed for clients effectively, so I can focus on other critical tasks.

Impact:

• Enhanced Efficiency and Scalability: By addressing the unsealing limitations, we enable storage providers to easily spin up more unsealing workers. This improvement allows for the efficient unsealing of larger amounts of sealed sectors.

Milestone

Preview Give feedback

1. Unsealing enhancements #10316
   8 of 9
   area/sealing kind/enhancement +2

---

🌌 2023 Q3 - Implement nv21 FIPs

User story:

As a storage provider (SP) I want to leverage SynthethicPoReps which has been accepted for the nv21 upgrade to reduce the temporary data stored between PreCommit and ProveCommit from ~400GiB to ≤ 25GiB.

Dependent on FIPs passing:

FIP-0070:
As a Storage Provider (SP) maintaining storage for the Filecoin network, I face challenges by having to be always prepared for any failed WindowPoSt. To better manage these challenges, I need greater flexibility in controlling the period to prove data integrity to the network, while still adhering to the rule of proving once every 24 hours. Having the ability to create user-defined maintenance windows will allow me to optimize my operations, reduce costs, and improve overall performance, while maintaining the integrity of the data I store.

Impact:

• Increased throughput: SynthPoRep reduces the cache storage requirements between PC2 and C2 by 98%, increasing the potential sealing throughput.
• Enabling batching PreCommit: SynthPoRep enables SPs to amass more PreCommit message due to the reduced cache size after PreCommit2 before batch publishing them to save gas costs.

Tracking issues

Preview Give feedback

1. Implement SynthPoRep #10984
   18 of 18
   +0
2. Implementation plan for allowing SPs to move partitions proposal' in Lotus-Miner #11129 (FIP decision dependent)
3. Implementation plan for direct data onboarding proposal' in Lotus-Miner #11125 (FIP decision dependent)

---

🌌 2023 Q3 - Implement Supranational PC2/C2 sealing code

User story:

• As a storage provider (SP) I want to utilize the highly optimized brownfield PC2/C2 sealing proofs from SupraNational, so that I can experience reduced energy consumption and hardware needs while achieving significant improvements in sealing times for the PC2/C2 tasks without compromising compatibility with deal-sectors, CC-sectors or sector sizes applicable on the Filecoin mainnet.

Impact:

• Improved sealing speed: Reduce sealing times for PC2 and C2 by ≈70+%
• Lower the energy usage: PC2 and C2-tasks are energy intensive GPU-tasks. Lowering the sealing times reduces OPEX by reducing electricy usage for a given sealingthroughput per day.

Milestone

Preview Give feedback

1. Supranational brownfield PC2 <> Lotus-Miner #10983
   7 of 8
   +0
   

   
2. Supranational brownfield C2 <> Lotus-Miner #10976
   8 of 8
   +0

---

🌌 2023 Q3/Q4 - High Availability of WindowPoSt

User story:

As a storage provider (SP) I need a high availability solution for windowPoSt to ensure that I don't miss any proving deadlines, scheduling, computations and submissions. This will allow me to maintain and maximize the reliability of my services, and not lose block rewards or get slashed due to unnecessary failed proving deadlines.

Impact:

• Improved Reliability: By ensuring high availability for windowPoSt, storage providers can maintain consistent service, reducing the risk of missed proving deadlines.
• Increased Earnings: With fewer failures in WindowPoSt, storage providers can avoid disruptions that could impact their earnings.
• Enhanced Reputation: Consistent service and reduced failures can improve the reputation of storage providers, potentially attracting more users to the Filecoin network.
• Future Enhancements: The common code developed as part of this milestone will serve as the foundation for future enhancements, making it easier to implement additional features and improvements.

Milestone

Preview Give feedback

1. High availability of WindowPoSt #10991
   10 of 10
   area/MinerV2 +1
   

---

🌌 2023 Q4 - High Availabilty of WinningPoSt

User story:

As a storage provider (SP), I need a high availability solution for WinningPoSt to ensure that I don't miss any opportunities to win block rewards. This includes the efficient scheduling, computation, and submission of WinningPoSt proofs. This will allow me to maintain and maximize the profitability of my services, and not lose potential block rewards due to missed computations or submissions.

Impact:

• Increased Earnings: By ensuring high availability for WinningPoSt, storage providers can maximize their opportunities to win block rewards, leading to increased earnings.

Milestone

Preview Give feedback

1. WinningPoSt work #11383
   0 of 6
   +0

---

🌌 2023 Q4 - Multi-MinerID support in Lotus-Provider

User story:

Impact:

Milestone

Preview Give feedback

1. Multi-Miner #11393
   0 of 3
   area/MinerV2 team/curio +2
   

---

Future architecture

Contact us

You can find us easily at the Filecoin Slack workspace(filecoin.io/slack) - #fil-lotus-dev & #fil-lotus-help