Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1918144 - Document submitting Glean events in "real-time" #2953

Merged
merged 2 commits into from
Sep 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .dictionary
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
personal_ws-1.1 en 289 utf-8
personal_ws-1.1 en 290 utf-8
AAR
AARs
ABI
Expand Down Expand Up @@ -160,6 +160,7 @@ enqueue
enqueued
enum
envs
etl
exe
experimentId
ffi
Expand Down
3 changes: 3 additions & 0 deletions docs/user/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@
- [JavaScript](user/debugging/javascript.md)
- [Walkthroughs and How-tos](user/howto/index.md)
- [Server Knobs Walkthrough](user/howto/server-knobs-walkthrough/server-knobs-walkthrough.md)
- ["Real-Time" Events](user/howto/real-time-events/real-time-events.md)

# API Reference

- [YAML Registry Format](reference/yaml/index.md)
Expand Down Expand Up @@ -101,6 +103,7 @@
- [Command Line Interface](language-bindings/javascript/cli.md)
- [Plugins](language-bindings/javascript/plugins/index.md)
- [Ping Encryption Plugin](language-bindings/javascript/plugins/encryption.md)

# Appendix

- [Glossary](appendix/glossary.md)
Expand Down
37 changes: 37 additions & 0 deletions docs/user/user/howto/real-time-events/real-time-events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# "Real-Time" Events

## Defining "Real-Time" Glean Events

"Real-Time" in the context of Glean [Events](../../../reference/metrics/event.md) directly relates to configuring Glean to send each event in an [Events Ping](../../pings/events.md) as soon as it is recorded.
travis79 marked this conversation as resolved.
Show resolved Hide resolved

## Configuring Glean For "Real-Time" Events
travis79 marked this conversation as resolved.
Show resolved Hide resolved

Glean event ping submission can either be configured at [initialization](../../../reference/general/initializing.md) or through [Server Knobs](../../../user/server-knobs/other/max-events.md).

Setting the maximum event threshold to a value of `1` will configure Glean to submit an Events Ping for every event recorded.

## Considerations

### What "Real-Time" Glean Events Are _Not_

Configuring Glean to submit events as soon as they are recorded does not mean to imply that the event data is available for analysis in
real-time. There are networks to traverse, ingestion pipelines, etl, etc. that are all factors to keep in mind when considering how soon
travis79 marked this conversation as resolved.
Show resolved Hide resolved
the data is available for analysis purposes. This documentation only purports to cover configuring Glean to send the data in a real-time
fashion and does not make any assumptions about the analysis of data in real-time.

### More Network Requests

There are also other trade-offs to consider when sending event pings with each and every event. For every event recorded a network request
travis79 marked this conversation as resolved.
Show resolved Hide resolved
will be generated when the event is submitted for ingestion. By default, Glean batches up to 500 events per event ping, so this has the
travis79 marked this conversation as resolved.
Show resolved Hide resolved
potential to generate up to 500 times as many network requests than the current default.

### More Ingestion Endpoint Traffic

As a result of the increased network requests, the ingestion endpoint will need to handle this additional traffic. This increases the load
of all the processing steps that are involved with ingesting event data from an application.

### Storage Space

Typically the raw dataset for Glean events contains 1-500 events in a single row of the database. This row also includes metadata such as
information about the client application and the ping itself. With only a single event per events ping, the replication of this metadata
across the database will use additional space to house this repeated information that should rarely if ever change between events
2 changes: 1 addition & 1 deletion docs/user/user/pings/events.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The `events` ping is automatically submitted under the following circumstances:
1. If there are any recorded events to send when the application becomes inactive (on mobile, this means going to [background](sent-by-glean.md#defining-foreground-and-background-state)).

2. When the queue of events exceeds `Glean.configuration.maxEvents` (default 1 for Glean.js, 500 for all other SDKs). This configuration
option can be changed at [initialization](../../reference/general/initializing.md).
option can be changed at [initialization](../../reference/general/initializing.md) or through [Server Knobs](../../user/server-knobs/other/max-events.md).

3. If there are any unsent events found on disk when starting the application. _(This results in this ping never containing the [`glean.restarted`](./custom.md#the-gleanrestarted-event) event.)_

Expand Down
Loading