Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjust actor event API after review #11649

Merged
merged 7 commits into from
Mar 1, 2024
Merged

Adjust actor event API after review #11649

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
90f1988
Adjust actor event API after review
rvagg Feb 22, 2024
0e20f14
s/ActorEvents/Events/g in global config
rvagg Feb 26, 2024
8a3a461
Manage event sending rate for SubscribeActorEvents
rvagg Feb 26, 2024
28389ff
Terminate SubscribeActorEvents chan when at max height
rvagg Feb 26, 2024
d200a43
Document future API changes
rvagg Feb 26, 2024
99ecc47
More clarity in actor event API docs
rvagg Feb 26, 2024
9fd3e02
More post-review changes, lots of tests for SubscribeActorEvents
rvagg Mar 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions api/api_full.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -890,15 +890,23 @@ type FullNode interface {

// GetActorEvents returns all FVM and built-in Actor events that match the given filter.
// This is a request/response API.
// Results available from this API may be limited by the MaxFilterResults and MaxFilterHeightRange
// configuration options and also the amount of historical data available in the node.
GetActorEvents(ctx context.Context, filter *types.ActorEventFilter) ([]*types.ActorEvent, error) //perm:read

// SubscribeActorEvents returns a long-lived stream of all FVM and built-in Actor events that match the given filter.
// Events that match the given filter are written to the stream in real-time as they are emitted from the FVM.
// The response stream is closed when the client disconnects or if there is an error while writing an event to the stream.
// This API also allows clients to read all historical events matching the given filter before
// any real-time events are written to the response stream.
// SubscribeActorEvents returns a long-lived stream of all FVM and built-in Actor events that
// match the given filter.
// Events that match the given filter are written to the stream in real-time as they are emitted
// from the FVM.
// The response stream is closed when the client disconnects, when a ToHeight is specified and is
// reached, or if there is an error while writing an event to the stream.
// This API also allows clients to read all historical events matching the given filter before any
// real-time events are written to the response stream if the filter specifies an earlier
// FromHeight.
// Results available from this API may be limited by the MaxFilterResults and MaxFilterHeightRange
// configuration options and also the amount of historical data available in the node.
// NOTE: THIS API IS ONLY SUPPORTED OVER WEBSOCKETS FOR NOW
SubscribeActorEvents(ctx context.Context, filter *types.SubActorEventFilter) (<-chan *types.ActorEvent, error) //perm:read
SubscribeActorEvents(ctx context.Context, filter *types.ActorEventFilter) (<-chan *types.ActorEvent, error) //perm:read
}

// reverse interface to the client, called after EthSubscribe
Expand Down
2 changes: 1 addition & 1 deletion api/api_gateway.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,5 +131,5 @@ type Gateway interface {
EthTraceReplayBlockTransactions(ctx context.Context, blkNum string, traceTypes []string) ([]*ethtypes.EthTraceReplayBlockTransaction, error)

GetActorEvents(ctx context.Context, filter *types.ActorEventFilter) ([]*types.ActorEvent, error)
SubscribeActorEvents(ctx context.Context, filter *types.SubActorEventFilter) (<-chan *types.ActorEvent, error)
SubscribeActorEvents(ctx context.Context, filter *types.ActorEventFilter) (<-chan *types.ActorEvent, error)
}
26 changes: 7 additions & 19 deletions api/docgen/docgen.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -430,25 +430,8 @@ func init() {
},
},
},
FromEpoch: "earliest",
ToEpoch: "latest",
})

addExample(&types.SubActorEventFilter{
Filter: types.ActorEventFilter{
Addresses: []address.Address{addr},
Fields: map[string][]types.ActorEventBlock{
"abc": {
{
Codec: 0x51,
Value: []byte("ddata"),
},
},
},
FromEpoch: "earliest",
ToEpoch: "latest",
},
Prefill: true,
FromHeight: epochPtr(1010),
ToHeight: epochPtr(1020),
})
}

Expand Down Expand Up @@ -555,6 +538,11 @@ func exampleStruct(method string, t, parent reflect.Type) interface{} {
return ns.Interface()
}

func epochPtr(ei int64) *abi.ChainEpoch {
ep := abi.ChainEpoch(ei)
return &ep
}

type Visitor struct {
Root string
Methods map[string]ast.Node
Expand Down
2 changes: 1 addition & 1 deletion api/mocks/mock_full.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions api/proxy_gen.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified build/openrpc/full.json.gz
View file
Open in desktop
Binary file not shown.
Binary file modified build/openrpc/gateway.json.gz
View file
Open in desktop
Binary file not shown.
37 changes: 16 additions & 21 deletions chain/types/actor_event.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,6 @@ type ActorEventBlock struct {
Value []byte `json:"value"`
}

type SubActorEventFilter struct {
Filter ActorEventFilter `json:"filter"`

// If true, all available matching historical events will be written to the response stream
// before any new real-time events that match the given filter are written.
// If `Prefill` is true and `FromEpoch` is set to latest, the pre-fill operation will become a no-op.
// if `Prefill` is false and `FromEpoch` is set to earliest, historical events will still be sent to the client.
Prefill bool `json:"prefill"`
}

type ActorEventFilter struct {
// Matches events from one of these actors, or any actor if empty.
// For now, this MUST be a Filecoin address.
Expand All @@ -37,25 +27,30 @@ type ActorEventFilter struct {
// If the value is an empty slice, the filter will match on the key only, accepting any value.
Fields map[string][]ActorEventBlock `json:"fields,omitempty"`

// Interpreted as an epoch (in hex) or one of "latest" for last mined block, "earliest" for first,
// Optional, default: "latest".
FromEpoch string `json:"fromEpoch,omitempty"`
// The height of the earliest tipset to include in the query. If empty, the query starts at the
// last finalized tipset.
// NOTE: In a future upgrade, this will be strict when set and will result in an error if a filter
// cannot be fulfilled by the depth of history available in the node. Currently, the node will
// nott return an error, but will return starting from the epoch it has data for.
FromHeight *abi.ChainEpoch `json:"fromHeight,omitempty"`

// Interpreted as an epoch (in hex) or one of "latest" for last mined block, "earliest" for first,
// Optional, default: "latest".
ToEpoch string `json:"toEpoch,omitempty"`
// The height of the latest tipset to include in the query. If empty, the query ends at the
// latest tipset.
ToHeight *abi.ChainEpoch `json:"toHeight,omitempty"`

// Restricts events returned to those emitted from messages contained in this tipset.
// If `TipSetCid` is present in the filter criteria, then neither `FromEpoch` nor `ToEpoch` are allowed.
TipSetCid *cid.Cid `json:"tipsetCid,omitempty"`
// If `TipSetKey` is legt empty in the filter criteria, then neither `FromHeight` nor `ToHeight` are allowed.
TipSetKey *TipSetKey `json:"tipsetKey,omitempty"`
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointer here may not be idiomatic; I just wanted to avoid an unnecessary [] in the json form, mainly. I think maybe if [] is omitted on the input side that it may still resolve to the zero-value here, I'm not sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointer is fine. I'm actually kind of surprised we don't marshal these to strings... but ok.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if [] is omitted on the input side that it may still resolve to the zero-value here, I'm not sure.

It will.

}

type ActorEvent struct {
// Event entries in log form.
Entries []EventEntry `json:"entries"`

// Filecoin address of the actor that emitted this event.
EmitterAddr address.Address `json:"emitter"`
// NOTE: In a future upgrade, this will change to always be an ID address. Currently this will be
// either the f4 address, or ID address if an f4 is not available for this actor.
Emitter address.Address `json:"emitter"`

// Reverted is set to true if the message that produced this event was reverted because of a network re-org
// in that case, the event should be considered as reverted as well.
Expand All @@ -64,8 +59,8 @@ type ActorEvent struct {
// Height of the tipset that contained the message that produced this event.
Height abi.ChainEpoch `json:"height"`

// CID of the tipset that contained the message that produced this event.
TipSetCid cid.Cid `json:"tipsetCid"`
// The tipset that contained the message that produced this event.
TipSetKey TipSetKey `json:"tipsetKey"`

// CID of message that produced this event.
MsgCid cid.Cid `json:"msgCid"`
Expand Down
28 changes: 15 additions & 13 deletions chain/types/actor_event_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"github.com/stretchr/testify/require"

"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
builtintypes "github.com/filecoin-project/go-state-types/builtin"
)

Expand All @@ -29,11 +30,11 @@ func TestActorEventJson(t *testing.T) {
Value: []byte("value2"),
},
},
EmitterAddr: randomF4Addr(t, rng),
Reverted: false,
Height: 1001,
TipSetCid: randomCid(t, rng),
MsgCid: randomCid(t, rng),
Emitter: randomF4Addr(t, rng),
Reverted: false,
Height: 1001,
TipSetKey: NewTipSetKey(randomCid(t, rng)),
MsgCid: randomCid(t, rng),
}

bz, err := json.Marshal(in)
Expand All @@ -46,7 +47,7 @@ func TestActorEventJson(t *testing.T) {
require.Equal(t, in, out)

s := `
{"entries":[{"Flags":0,"Key":"key1","Codec":81,"Value":"dmFsdWUx"},{"Flags":0,"Key":"key2","Codec":82,"Value":"dmFsdWUy"}],"emitter":"f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua","reverted":false,"height":1001,"tipsetCid":{"/":"bafkqacx3dag26sfht3qlcdi"},"msgCid":{"/":"bafkqacrziziykd6uuf4islq"}}
{"entries":[{"Flags":0,"Key":"key1","Codec":81,"Value":"dmFsdWUx"},{"Flags":0,"Key":"key2","Codec":82,"Value":"dmFsdWUy"}],"emitter":"f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua","reverted":false,"height":1001,"tipsetKey":[{"/":"bafkqacx3dag26sfht3qlcdi"}],"msgCid":{"/":"bafkqacrziziykd6uuf4islq"}}
`
var out2 ActorEvent
err = json.Unmarshal([]byte(s), &out2)
Expand Down Expand Up @@ -77,9 +78,9 @@ func TestActorEventBlockJson(t *testing.T) {
}

func TestSubActorEventFilterJson(t *testing.T) {
c := randomCid(t, pseudo.New(pseudo.NewSource(0)))
from := "earliest"
to := "latest"
tsk := NewTipSetKey(randomCid(t, pseudo.New(pseudo.NewSource(0))))
from := abi.ChainEpoch(0)
to := abi.ChainEpoch(100)
f := ActorEventFilter{
Addresses: []address.Address{
randomF4Addr(t, pseudo.New(pseudo.NewSource(0))),
Expand All @@ -99,16 +100,17 @@ func TestSubActorEventFilterJson(t *testing.T) {
},
},
},
FromEpoch: from,
ToEpoch: to,
TipSetCid: &c,
FromHeight: &from,
ToHeight: &to,
TipSetKey: &tsk,
}

bz, err := json.Marshal(f)
require.NoError(t, err)
require.NotEmpty(t, bz)
t.Logf("%s", bz)

s := `{"addresses":["f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua","f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua"],"fields":{"key1":[{"codec":81,"value":"dmFsdWUx"}],"key2":[{"codec":82,"value":"dmFsdWUy"}]},"fromEpoch":"earliest","toEpoch":"latest","tipsetCid":{"/":"bafkqacqbst64f6rp7taeduy"}}`
s := `{"addresses":["f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua","f410fagkp3qx2f76maqot74jaiw3tzbxe76k76zrkl3xifk67isrnbn2sll3yua"],"fields":{"key1":[{"codec":81,"value":"dmFsdWUx"}],"key2":[{"codec":82,"value":"dmFsdWUy"}]},"fromHeight":0,"toHeight":100,"tipsetKey":[{"/":"bafkqacqbst64f6rp7taeduy"}]}`
var out ActorEventFilter
err = json.Unmarshal([]byte(s), &out)
require.NoError(t, err)
Expand Down
71 changes: 43 additions & 28 deletions documentation/en/api-v1-unstable-methods.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3394,6 +3394,8 @@ Response:
### GetActorEvents
GetActorEvents returns all FVM and built-in Actor events that match the given filter.
This is a request/response API.
Results available from this API may be limited by the MaxFilterResults and MaxFilterHeightRange
configuration options and also the amount of historical data available in the node.


Perms: read
Expand All @@ -3413,8 +3415,8 @@ Inputs:
}
]
},
"fromEpoch": "earliest",
"toEpoch": "latest"
"fromHeight": 1010,
"toHeight": 1020
}
]
```
Expand All @@ -3434,9 +3436,14 @@ Response:
"emitter": "f01234",
"reverted": true,
"height": 10101,
"tipsetCid": {
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
},
"tipsetKey": [
{
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
},
{
"/": "bafy2bzacebp3shtrn43k7g3unredz7fxn4gj533d3o43tqn2p2ipxxhrvchve"
}
],
"msgCid": {
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
}
Expand Down Expand Up @@ -8780,11 +8787,17 @@ Response:


### SubscribeActorEvents
SubscribeActorEvents returns a long-lived stream of all FVM and built-in Actor events that match the given filter.
Events that match the given filter are written to the stream in real-time as they are emitted from the FVM.
The response stream is closed when the client disconnects or if there is an error while writing an event to the stream.
This API also allows clients to read all historical events matching the given filter before
any real-time events are written to the response stream.
SubscribeActorEvents returns a long-lived stream of all FVM and built-in Actor events that
match the given filter.
Events that match the given filter are written to the stream in real-time as they are emitted
from the FVM.
The response stream is closed when the client disconnects, when a ToHeight is specified and is
reached, or if there is an error while writing an event to the stream.
This API also allows clients to read all historical events matching the given filter before any
real-time events are written to the response stream if the filter specifies an earlier
FromHeight.
Results available from this API may be limited by the MaxFilterResults and MaxFilterHeightRange
configuration options and also the amount of historical data available in the node.
NOTE: THIS API IS ONLY SUPPORTED OVER WEBSOCKETS FOR NOW


Expand All @@ -8794,22 +8807,19 @@ Inputs:
```json
[
{
"filter": {
"addresses": [
"f01234"
],
"fields": {
"abc": [
{
"codec": 81,
"value": "ZGRhdGE="
}
]
},
"fromEpoch": "earliest",
"toEpoch": "latest"
"addresses": [
"f01234"
],
"fields": {
"abc": [
{
"codec": 81,
"value": "ZGRhdGE="
}
]
},
"prefill": true
"fromHeight": 1010,
"toHeight": 1020
}
]
```
Expand All @@ -8828,9 +8838,14 @@ Response:
"emitter": "f01234",
"reverted": true,
"height": 10101,
"tipsetCid": {
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
},
"tipsetKey": [
{
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
},
{
"/": "bafy2bzacebp3shtrn43k7g3unredz7fxn4gj533d3o43tqn2p2ipxxhrvchve"
}
],
"msgCid": {
"/": "bafy2bzacea3wsdh6y3a36tb3skempjoxqpuyompjbmfeyf34fi3uy6uue42v4"
}
Expand Down
Loading