Integrate Storage Client Deal List Pagination #5880

aarshkshah1992 · 2021-03-25T11:51:48Z

For #4185.

Depends on #5853

TODO

Merge update to markets v1.2.x #5853
Update deps.

Otherwise, this won't work on 32bit ARM.

Can specify miner actor address when terminate sectors, so that you can terminate sectors without miner api.

( cherry-pick of bc5ed96 )

… the time

…eset_to_1_5_3 Chore/backport nerpa reset to 1 5 3

better logging when unsealing fails

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>

api: Document StateReplay replaced message behavior

perf: add cache for gas permium estimation

- allows tipset apply and revert to be observed

- motivating use case running a lotus-sentinel node

Update generated docs

Implement Event observer and Settings for 3rd party dep injection

* mostly working api proxy gen * api: Consistent api names * fix docsgen * regenerate api struct * api: expand external interfaces * Add missing gen files * apigen: fix perm detection * api: Move perm tags to the interface * gofmt * worker perms * docsgen * docsgen: ignore tag comments * apigen: add codegen warning * gofmt * missing actor type * docsgen * make linter happy * fix lint * apigen: use directives for tags * docsgen * regen openrpc docs

add godocs to miner objects.

* force packer build * disable ami temporarially * apt and ufw * initial lotus config * rename correctly * apt-get instead of apt * undo temporary edits * install before upgrade * tmp - force digitalocean * reenable filters * uncomment ebs * tmp, remove filter for digitalocean build * doc * install ufw (for iams where it isn't installed) * more amazon regions * missing comma * missing comma * remove unavailable regions * switch to ubuntu, not minimal * pro server * change owner * change image * uncomment circleci filters * remove api from firewall * tmp: remove circleci filter * dpkg options * confold * try export * re-add filter

expose NextID from nice market actor interface

fix: StateManager.Replay

fix: make sure atomic 64bit fields are 64bit aligned

dirkmc · 2021-03-25T12:30:28Z

cli/client.go

+			DefaultText: "no-op",
+		},
+		&cli.IntFlag{
+			Name:        "deals-per-page",


Let's have a default for deals-per-page. 50 seems like a reasonable default.

…s-for-perm-on-error chore: add available options for perm on error

Add --actor flag in lotus-shed sectors terminate

Import secp sigs in paych tests

aarshkshah1992 · 2021-03-26T09:36:54Z

@dirkmc @jennijuju

All possible options with which we can filter

arshshah@Aarshs-MacBook-Pro lotus % lotus client list-deals --help                                                      
NAME:
   lotus client list-deals - List storage market deals

USAGE:
   also supports pagination, please see supported options for more details

CATEGORY:
   STORAGE

OPTIONS:
   --verbose, -v                 print verbose deal details (default: false)
   --color                       use color in display output (default: true)
   --watch                       watch deal updates in real-time, rather than a one time list (default: false)
   --hide-failed                 hide failed/failing deals (default: false)
   --creation-time-offset value  minimum creation time offset of the deals to include in the deal list page (default: no-op)
   --limit value                 number of deals to show in the deal list page (default: 50)
   --min-start-epoch value       minimum start epoch of the deals to include in the deal list page (default: no-op)
   --max-end-epoch value         maximum end epoch of the deals to include in the deal list page (default: no-op)
   --help, -h                    show help (default: false)

All Deals

aarshshah@Aarshs-MacBook-Pro lotus % lotus client list-deals -v

Created                    DealCid                                                      DealId  Provider  State                         On Chain?       Slashed?  PieceCID                                                          Size       Price                     Duration  TransferChannelID                                                                                                              TransferStatus  Verified  Message
2021-03-26T13:24:26+05:30  bafyreibcacybtma7rdfycjkognrl6wdty3lkmrdvys4jtoqbiwi2uziuaa  10      f01000    StorageDealAwaitingPreCommit  Y (epoch 1629)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.000402281072888688 FIL  1300994   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884001  Completed       false     
2021-03-26T13:26:21+05:30  bafyreic33ishws3swia2j4wwajlavv7ullpii237pqygtm4o4fwttsdayu  11      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.00040227210578268 FIL   1300965   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884002  Completed       false     
2021-03-26T13:26:26+05:30  bafyreib4yuxx6jmtjonlfdvpi3wk7vmvtvhi6d6teo2qhqscpkznt7gvbq  13      f01000    StorageDealActive    Y (epoch 1628)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.000402271796572128 FIL  1300964   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884004  Completed       false     
2021-03-26T13:26:31+05:30  bafyreibj47awsvzkblc4r6xln7elgh5r6gr4ftbvqo67i3sbjyfhk75mr4  14      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.000402271487361576 FIL  1300963   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884005  Completed       false     
2021-03-26T13:26:34+05:30  bafyreignj7x7smarnhd7ndbgrsi6xtwwnanxndhtww7fsc2tadnnrwnhru  12      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.000402271178151024 FIL  1300962   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884003  Completed       false

Just show the first two deals

aarshshah@Aarshs-MacBook-Pro lotus % lotus client list-deals -v --limit=2

Created                    DealCid                                                      DealId  Provider  State                         On Chain?       Slashed?  PieceCID                                                          Size       Price                     Duration  TransferChannelID                                                                                                              TransferStatus  Verified  Message
2021-03-26T13:24:26+05:30  bafyreibcacybtma7rdfycjkognrl6wdty3lkmrdvys4jtoqbiwi2uziuaa  10      f01000    StorageDealAwaitingPreCommit  Y (epoch 1629)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.000402281072888688 FIL  1300994   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884001  Completed       false     
2021-03-26T13:26:21+05:30  bafyreic33ishws3swia2j4wwajlavv7ullpii237pqygtm4o4fwttsdayu  11      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         baga6ea4seaqkmjd7eh6ypntkvtnfp65fjgqhhta44vik7ths7hkrrp32scruujy  1.984 KiB  0.00040227210578268 FIL   1300965   12D3KooWHxkZnb2UHS23QAE6Hs2GptYQpziQXaHwpHjnZGHdXtkf-12D3KooWGBzFUDEw14UtJg1MeYrN5b8UNnyNBgqjumrB9p3wfqQ7-1616743172515884002  Completed       false

Show deals after a certain time offset

aarshshah@Aarshs-MacBook-Pro lotus % lotus client list-deals --creation-time-offset="2021-03-26T13:26:21+05:30"
DealCid      DealId  Provider  State                         On Chain?       Slashed?  PieceCID     Size       Price                     Duration  Verified  
...wttsdayu  11      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         ...2scruujy  1.984 KiB  0.00040227210578268 FIL   1300965   false     
...znt7gvbq  13      f01000    StorageDealActive             Y (epoch 1628)  N         ...2scruujy  1.984 KiB  0.000402271796572128 FIL  1300964   false     
...fhk75mr4  14      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         ...2scruujy  1.984 KiB  0.000402271487361576 FIL  1300963   false     
...nnrwnhru  12      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         ...2scruujy  1.984 KiB  0.000402271178151024 FIL  1300962   false

Show the first two deals after a certain time offset

aarshshah@Aarshs-MacBook-Pro lotus % lotus client list-deals --creation-time-offset="2021-03-26T13:26:21+05:30" --limit=2
DealCid      DealId  Provider  State                         On Chain?       Slashed?  PieceCID     Size       Price                     Duration  Verified  
...wttsdayu  11      f01000    StorageDealAwaitingPreCommit  Y (epoch 1630)  N         ...2scruujy  1.984 KiB  0.00040227210578268 FIL   1300965   false     
...znt7gvbq  13      f01000    StorageDealActive             Y (epoch 1628)  N         ...2scruujy  1.984 KiB  0.000402271796572128 FIL  1300964   false

aarshkshah1992 · 2021-03-26T11:47:23Z

Closed in favour of #5886.

Stebalien and others added 30 commits March 11, 2021 20:10

fix: make sure atomic 64bit fields are 64bit aligned

1c490f3

Otherwise, this won't work on 32bit ARM.

bugfix: StateManager.Replay

6f64d5d

chore: add available options for perm on error

9120d6f

Add --actor flag in lotus-shed sectors terminate

7417a13

Can specify miner actor address when terminate sectors, so that you can terminate sectors without miner api.

Lotus 1.5.3-rc2

9afb5ff

Nerpa reset backport/cherry-pick from 73eabc3

c340750

Build more binaries by default when a devnet tag is active

b54a58c

( cherry-pick of bc5ed96 )

api: Document StateReplay replaced message behavior

e74aa7e

api: Note that ChainGetBlockMessages is not the method to use most of…

82e019b

… the time

api: Better StateCompute docs

da10ef2

expose NextID from nice market actor interface

6bcf92f

fix: better logging when unsealing fails

73d5225

Fix unseal comment

14ede9b

Merge pull request #5839 from filecoin-project/chore/backport_nerpa_r…

b55ca92

…eset_to_1_5_3 Chore/backport nerpa reset to 1 5 3

Merge pull request #5851 from filecoin-project/fix/rtrvl-log

897c5c7

better logging when unsealing fails

perf: add cache for gas permium estimation

5f672c2

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>

Merge pull request #5840 from filecoin-project/docs/replay-replace-note

caa1d5b

api: Document StateReplay replaced message behavior

Merge pull request #5709 from filecoin-project/feat/optim-gas-premium

4c7df0a

perf: add cache for gas permium estimation

feat(events): define Observer intreface for events

0b579c1

- allows tipset apply and revert to be observed

polish(node): define settings opts for 3rd party dep injection

6caafae

- motivating use case running a lotus-sentinel node

circle: output generated openrpc diffs

2f37b62

docsgen

be704cf

Merge pull request #5860 from filecoin-project/fix/update-openrpc-doc

4cae530

Update generated docs

Merge pull request #5693 from filecoin-project/sentinel/binary

f145d4f

Implement Event observer and Settings for 3rd party dep injection

chore: export Net commands

6827f2d

chore: export log cli commands

1062117

chore: export chain commands

8f6ea05

chore: export state cli commands

22329b9

chore: export sync cli commands

89c45fe

raulk and others added 3 commits March 25, 2021 13:02

add godocs to miner objects.

87efc9d

Merge pull request #2184 from filecoin-project/godocs/miner

4d01f8f

add godocs to miner objects.

dirkmc mentioned this pull request Mar 25, 2021

Add paging to deal list #4185

Closed

4 tasks

magik6k added 3 commits March 25, 2021 13:26

Merge pull request #5850 from filecoin-project/feat/markets-dealid

fc4f60e

expose NextID from nice market actor interface

Merge pull request #5804 from zl03jsj/fix/StateManager.Replay

53cba15

fix: StateManager.Replay

Merge pull request #5794 from filecoin-project/fix/atomic-first

5f80869

fix: make sure atomic 64bit fields are 64bit aligned

dirkmc reviewed Mar 25, 2021

View reviewed changes

magik6k added 3 commits March 25, 2021 13:56

Merge pull request #5814 from vasco-santos/chore/add-available-option…

312c69f

…s-for-perm-on-error chore: add available options for perm on error

Merge pull request #5819 from chadwick2143/lotus-shed

6d9ae1e

Add --actor flag in lotus-shed sectors terminate

Merge pull request #5879 from filecoin-project/feat/paychtest-secp

885ecb9

Import secp sigs in paych tests

aarshkshah1992 force-pushed the feat/integrate-pagination branch from dbb95ad to 3ece9d7 Compare March 26, 2021 04:38

aarshkshah1992 changed the base branch from master to feat/markets-v1.2.0 March 26, 2021 09:28

dirkmc and others added 12 commits March 26, 2021 17:01

feat: update to markets v1.2.0

8675e30

feat: update go-data-transfer and go-fil-markets

c828912

fix: disable pull channel monitoring for now

d0b5d62

feat: markets v1.2.3

dd00c0a

increase client polling interval

053ddf2

integrate fil markets pagination

e5c8a5a

remove line

845f28e

fix line

6778b29

changes as per review

70bdcf7

remove var-arg and run makegen

5f4eb2a

fix timestamp handling

11b27e3

description changes

ac4faae

aarshkshah1992 force-pushed the feat/integrate-pagination branch from 9ce9ce5 to ac4faae Compare March 26, 2021 11:32

aarshkshah1992 closed this Mar 26, 2021

Kubuxu deleted the feat/integrate-pagination branch November 25, 2021 18:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Storage Client Deal List Pagination #5880

Integrate Storage Client Deal List Pagination #5880

aarshkshah1992 commented Mar 25, 2021 •

edited by dirkmc

Loading

dirkmc Mar 25, 2021

aarshkshah1992 commented Mar 26, 2021 •

edited

Loading

aarshkshah1992 commented Mar 26, 2021

Integrate Storage Client Deal List Pagination #5880

Integrate Storage Client Deal List Pagination #5880

Conversation

aarshkshah1992 commented Mar 25, 2021 • edited by dirkmc Loading

TODO

dirkmc Mar 25, 2021

Choose a reason for hiding this comment

aarshkshah1992 commented Mar 26, 2021 • edited Loading

aarshkshah1992 commented Mar 26, 2021

aarshkshah1992 commented Mar 25, 2021 •

edited by dirkmc

Loading

aarshkshah1992 commented Mar 26, 2021 •

edited

Loading